Robust optimization

The origins of robust optimization date back to the establishment of modern decision theory in the 1950s and the use of worst case analysis and Wald's maximin model as a tool for the treatment of severe uncertainty. It became a discipline of its own in the 1970s with parallel developments in several scientific and technological fields. Over the years, it has been applied in statistics, but also in operations research,{{cite journal|last=Bertsimas|first=Dimitris|author2=Sim, Melvyn |title=The Price of Robustness|journal=Operations Research|year=2004|volume=52|issue=1|pages=35–53|doi=10.1287/opre.1030.0065|hdl=2268/253225 |s2cid=8946639 |hdl-access=free}} electrical engineering,{{Cite journal |last1=Giraldo |first1=Juan S. |last2=Castrillon |first2=Jhon A. |last3=Lopez |first3=Juan Camilo |last4=Rider |first4=Marcos J. |last5=Castro |first5=Carlos A. |date=July 2019 |title=Microgrids Energy Management Using Robust Convex Programming |url=https://ieeexplore.ieee.org/document/8424876 |journal=IEEE Transactions on Smart Grid |volume=10 |issue=4 |pages=4520–4530 |doi=10.1109/TSG.2018.2863049 |s2cid=115674048 |issn=1949-3053|url-access=subscription }}{{Cite journal| title = The design of a risk-hedging tool for virtual power plants via robust optimization approach | journal= Applied Energy | date = October 2015 | doi = 10.1016/j.apenergy.2015.06.059 | author = Shabanzadeh M | volume = 155 | pages = 766–777 | last2 = Sheikh-El-Eslami | first2 = M-K |last3 = Haghifam | first3 = P|last4 = M-R| bibcode= 2015ApEn..155..766S }}{{Cite book| pages= 1504–1509 | date = July 2015 | doi = 10.1109/IranianCEE.2015.7146458 | author = Shabanzadeh M | last2 = Fattahi | first2 = M | title= 2015 23rd Iranian Conference on Electrical Engineering | chapter= Generation Maintenance Scheduling via robust optimization | isbn= 978-1-4799-1972-7 | s2cid= 8774918 }} control theory,{{cite journal|last=Khargonekar|first=P.P.|author2=Petersen, I.R. |author3=Zhou, K. |title=Robust stabilization of uncertain linear systems: quadratic stabilizability and H/sup infinity / control theory|journal=IEEE Transactions on Automatic Control|volume=35|issue=3|pages=356–361|doi=10.1109/9.50357|year=1990}} finance,[https://books.google.com/books?id=p6UHHfkQ9Y8C&dq=economics%20robust%20optimization&pg=PR11 Robust portfolio optimization] portfolio managementMd. Asadujjaman and Kais Zaman, "Robust Portfolio Optimization under Data Uncertainty" 15th National Statistical Conference, December 2014, Dhaka, Bangladesh. logistics,{{cite journal|last=Yu|first=Chian-Son|author2=Li, Han-Lin |title=A robust optimization model for stochastic logistic problems|journal=International Journal of Production Economics|volume=64|issue=1–3|pages=385–397|doi=10.1016/S0925-5273(99)00074-2|year=2000}} manufacturing engineering,{{cite journal|last=Strano|first=M|title=Optimization under uncertainty of sheet-metal-forming processes by the finite element method|journal=Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture|volume=220|issue=8|pages=1305–1315|doi=10.1243/09544054JEM480|year=2006|s2cid=108843522}} chemical engineering,{{cite journal|last=Bernardo|first=Fernando P.|author2=Saraiva, Pedro M. |title=Robust optimization framework for process parameter and tolerance design|journal=AIChE Journal|year=1998|volume=44|issue=9|pages=2007–2017|doi=10.1002/aic.690440908|bibcode=1998AIChE..44.2007B |hdl=10316/8195|hdl-access=free}} medicine,{{cite journal|last=Chu|first=Millie|author2=Zinchenko, Yuriy |author3=Henderson, Shane G |author4= Sharpe, Michael B |title=Robust optimization for intensity modulated radiation therapy treatment planning under uncertainty|journal=Physics in Medicine and Biology|year=2005|volume=50|issue=23|pages=5463–5477|doi=10.1088/0031-9155/50/23/003|pmid=16306645|bibcode=2005PMB....50.5463C |s2cid=15713904 }} and computer science. In engineering problems, these formulations often take the name of "Robust Design Optimization", RDO or "Reliability Based Design Optimization", RBDO.

Consider the following linear programming problem

:$\backslash max\_\{x,y\}\; \backslash \; \backslash \{3x\; +\; 2y\backslash \}\; \backslash \; \backslash \; \backslash mathrm\; \{\; subject\; \backslash \; to\; \}\backslash \; \backslash \; x,y\backslash ge\; 0;\; cx\; +\; dy\; \backslash le\; 10,\; \backslash forall\; (c,d)\backslash in\; P$

where $P$ is a given subset of $\backslash mathbb\{R\}^\{2\}$.

What makes this a 'robust optimization' problem is the $\backslash forall\; (c,d)\backslash in\; P$ clause in the constraints. Its implication is that for a pair $(x,y)$ to be admissible, the constraint $cx\; +\; dy\; \backslash le\; 10$ must be satisfied by the worst $(c,d)\backslash in\; P$ pertaining to $(x,y)$, namely the pair $(c,d)\backslash in\; P$ that maximizes the value of $cx\; +\; dy$ for the given value of $(x,y)$.

If the parameter space $P$ is finite (consisting of finitely many elements), then this robust optimization problem itself is a linear programming problem: for each $(c,d)\backslash in\; P$ there is a linear constraint $cx\; +\; dy\; \backslash le\; 10$.

If $P$ is not a finite set, then this problem is a linear semi-infinite programming problem, namely a linear programming problem with finitely many (2) decision variables and infinitely many constraints.

There are a number of classification criteria for robust optimization problems/models. In particular, one can distinguish between problems dealing with local and global models of robustness; and between probabilistic and non-probabilistic models of robustness. Modern robust optimization deals primarily with non-probabilistic models of robustness that are worst case oriented and as such usually deploy Wald's maximin models.

There are cases where robustness is sought against small perturbations in a nominal value of a parameter. A very popular model of local robustness is the radius of stability model:

: $\backslash hat\{\backslash rho\}(x,\backslash hat\{u\}):=\; \backslash max\_\{\backslash rho\backslash ge\; 0\}\backslash \; \backslash \{\backslash rho:\; u\backslash in\; S(x),\; \backslash forall\; u\backslash in\; B(\backslash rho,\backslash hat\{u\})\backslash \}$

where $\backslash hat\{u\}$ denotes the nominal value of the parameter, $B(\backslash rho,\backslash hat\{u\})$ denotes a ball of radius $\backslash rho$ centered at $\backslash hat\{u\}$ and $S(x)$ denotes the set of values of $u$ that satisfy given stability/performance conditions associated with decision $x$.

In words, the robustness (radius of stability) of decision $x$ is the radius of the largest ball centered at $\backslash hat\{u\}$ all of whose elements satisfy the stability requirements imposed on $x$. The picture is this:

500px

where the rectangle $U(x)$ represents the set of all the values $u$ associated with decision $x$.

Consider the simple abstract robust optimization problem

: $\backslash max\_\{x\backslash in\; X\}\backslash \; \backslash \{f(x):\; g(x,u)\backslash le\; b,\; \backslash forall\; u\backslash in\; U\backslash \}$

where $U$ denotes the set of all possible values of $u$ under consideration.

This is a global robust optimization problem in the sense that the robustness constraint $g(x,u)\backslash le\; b,\; \backslash forall\; u\backslash in\; U$ represents all the possible values of $u$.

The difficulty is that such a "global" constraint can be too demanding in that there is no $x\backslash in\; X$ that satisfies this constraint. But even if such an $x\backslash in\; X$ exists, the constraint can be too "conservative" in that it yields a solution $x\backslash in\; X$ that generates a very small payoff $f(x)$ that is not representative of the performance of other decisions in $X$. For instance, there could be an $x\text{'}\backslash in\; X$ that only slightly violates the robustness constraint but yields a very large payoff $f(x\text{'})$. In such cases it might be necessary to relax a bit the robustness constraint and/or modify the statement of the problem.

Consider the case where the objective is to satisfy a constraint $g(x,u)\backslash le\; b,$. where $x\backslash in\; X$ denotes the decision variable and $u$ is a parameter whose set of possible values in $U$. If there is no $x\backslash in\; X$ such that $g(x,u)\backslash le\; b,\backslash forall\; u\backslash in\; U$, then the following intuitive measure of robustness suggests itself:

: $\backslash rho(x):=\; \backslash max\_\{Y\backslash subseteq\; U\}\; \backslash \; \backslash \{size(Y):\; g(x,u)\backslash le\; b,\; \backslash forall\; u\backslash in\; Y\backslash \}\; \backslash \; ,\; \backslash \; x\backslash in\; X$

where $size(Y)$ denotes an appropriate measure of the "size" of set $Y$. For example, if $U$ is a finite set, then $size(Y)$ could be defined as the cardinality of set $Y$.

In words, the robustness of decision is the size of the largest subset of $U$ for which the constraint $g(x,u)\backslash le\; b$ is satisfied for each $u$ in this set. An optimal decision is then a decision whose robustness is the largest.

This yields the following robust optimization problem:

: $\backslash max\_\{x\backslash in\; X,\; Y\backslash subseteq\; U\}\; \backslash \; \backslash \{size(Y):\; g(x,u)\; \backslash le\; b,\; \backslash forall\; u\backslash in\; Y\backslash \}$

This intuitive notion of global robustness is not used often in practice because the robust optimization problems that it induces are usually (not always) very difficult to solve.

Consider the robust optimization problem

:$z(U):=\; \backslash max\_\{x\backslash in\; X\}\backslash \; \backslash \{f(x):\; g(x,u)\backslash le\; b,\; \backslash forall\; u\backslash in\; U\backslash \}$

where $g$ is a real-valued function on $X\backslash times\; U$, and assume that there is no feasible solution to this problem because the robustness constraint $g(x,u)\backslash le\; b,\; \backslash forall\; u\backslash in\; U$ is too demanding.

To overcome this difficulty, let $\backslash mathcal\{N\}$ be a relatively small subset of $U$ representing "normal" values of $u$ and consider the following robust optimization problem:

:$z(\backslash mathcal\{N\}):=\; \backslash max\_\{x\backslash in\; X\}\backslash \; \backslash \{f(x):\; g(x,u)\backslash le\; b,\; \backslash forall\; u\backslash in\; \backslash mathcal\{N\}\backslash \}$

Since $\backslash mathcal\{N\}$ is much smaller than $U$, its optimal solution may not perform well on a large portion of $U$ and therefore may not be robust against the variability of $u$ over $U$.

One way to fix this difficulty is to relax the constraint $g(x,u)\backslash le\; b$ for values of $u$ outside the set $\backslash mathcal\{N\}$ in a controlled manner so that larger violations are allowed as the distance of $u$ from $\backslash mathcal\{N\}$ increases. For instance, consider the relaxed robustness constraint

: $g(x,u)\; \backslash le\; b\; +\; \backslash beta\; \backslash cdot\; dist(u,\backslash mathcal\{N\})\; \backslash \; ,\; \backslash \; \backslash forall\; u\backslash in\; U$

where $\backslash beta\; \backslash ge\; 0$ is a control parameter and $dist(u,\backslash mathcal\{N\})$ denotes the distance of $u$ from $\backslash mathcal\{N\}$. Thus, for $\backslash beta\; =0$ the relaxed robustness constraint reduces back to the original robustness constraint.

This yields the following (relaxed) robust optimization problem:

:$z(\backslash mathcal\{N\},U):=\; \backslash max\_\{x\backslash in\; X\}\backslash \; \backslash \{f(x):\; g(x,u)\backslash le\; b\; +\; \backslash beta\; \backslash cdot\; dist(u,\backslash mathcal\{N\})\; \backslash \; ,\; \backslash \; \backslash forall\; u\backslash in\; U\backslash \}$

The function $dist$ is defined in such a manner that

:$dist(u,\backslash mathcal\{N\})\backslash ge\; 0,\backslash forall\; u\backslash in\; U$

and

: $dist(u,\backslash mathcal\{N\})=\; 0,\backslash forall\; u\backslash in\; \backslash mathcal\{N\}$

and therefore the optimal solution to the relaxed problem satisfies the original constraint $g(x,u)\backslash le\; b$ for all values of $u$ in $\backslash mathcal\{N\}$. It also satisfies the relaxed constraint

: $g(x,u)\backslash le\; b\; +\; \backslash beta\; \backslash cdot\; dist(u,\backslash mathcal\{N\})$

outside $\backslash mathcal\{N\}$.

The dominating paradigm in this area of robust optimization is Wald's maximin model, namely

: $\backslash max\_\{x\backslash in\; X\}\backslash min\_\{u\backslash in\; U(x)\}\; f(x,u)$

where the $\backslash max$ represents the decision maker, the $\backslash min$ represents Nature, namely uncertainty, $X$ represents the decision space and $U(x)$ denotes the set of possible values of $u$ associated with decision $x$. This is the classic format of the generic model, and is often referred to as minimax or maximin optimization problem. The non-probabilistic (deterministic) model has been and is being extensively used for robust optimization especially in the field of signal processing.{{cite journal | last1 = Verdu | first1 = S. | last2 = Poor | first2 = H. V. | year = 1984 | title = On Minimax Robustness: A general approach and applications | journal = IEEE Transactions on Information Theory | volume = 30 | issue = 2| pages = 328–340 | doi=10.1109/tit.1984.1056876| citeseerx = 10.1.1.132.837 }}{{cite journal | last1 = Kassam | first1 = S. A. | last2 = Poor | first2 = H. V. | year = 1985 | title = Robust Techniques for Signal Processing: A Survey | journal = Proceedings of the IEEE | volume = 73 | issue = 3| pages = 433–481 | doi=10.1109/proc.1985.13167| hdl = 2142/74118 | s2cid = 30443041 | hdl-access = free }}M. Danish Nisar. [https://www.shaker.eu/shop/978-3-8440-0332-1 "Minimax Robustness in Signal Processing for Communications"], Shaker Verlag, {{ISBN|978-3-8440-0332-1}}, August 2011.

The equivalent mathematical programming (MP) of the classic format above is

:$\backslash max\_\{x\backslash in\; X,v\backslash in\; \backslash mathbb\{R\}\}\; \backslash \; \backslash \{v:\; v\backslash le\; f(x,u),\; \backslash forall\; u\backslash in\; U(x)\backslash \}$

Constraints can be incorporated explicitly in these models. The generic constrained classic format is

: $\backslash max\_\{x\backslash in\; X\}\backslash min\_\{u\backslash in\; U(x)\}\; \backslash \; \backslash \{f(x,u):\; g(x,u)\backslash le\; b,\backslash forall\; u\backslash in\; U(x)\backslash \}$

The equivalent constrained MP format is defined as:

:$\backslash max\_\{x\backslash in\; X,v\backslash in\; \backslash mathbb\{R\}\}\; \backslash \; \backslash \{v:\; v\backslash le\; f(x,u),\; g(x,u)\backslash le\; b,\; \backslash forall\; u\backslash in\; U(x)\backslash \}$

These models quantify the uncertainty in the "true" value of the parameter of interest by probability distribution functions. They have been traditionally classified as stochastic programming and stochastic optimization models. Recently, probabilistically robust optimization has gained popularity by the introduction of rigorous theories such as scenario optimization able to quantify the robustness level of solutions obtained by randomization. These methods are also relevant to data-driven optimization methods.

The solution method to many robust program involves creating a deterministic equivalent, called the robust counterpart. The practical difficulty of a robust program depends on if its robust counterpart is computationally tractable.Ben-Tal A., El Ghaoui, L. and Nemirovski, A. (2009). Robust Optimization. Princeton Series in Applied Mathematics, Princeton University Press, 9-16.Leyffer S., Menickelly M., Munson T., Vanaret C. and Wild S. M (2020). A survey of nonlinear robust optimization. INFOR: Information Systems and Operational Research, Taylor \& Francis.

• Stability radius
• Minimax
• Minimax estimator
• Minimax regret
• Robust statistics
• Robust decision making
• Robust fuzzy programming
• Stochastic programming
• Stochastic optimization
• Info-gap decision theory
• Taguchi methods

{{Reflist}}

• H.J. Greenberg. Mathematical Programming Glossary. World Wide Web, http://glossary.computing.society.informs.org/, 1996-2006. Edited by the INFORMS Computing Society.
• {{cite journal | last1 = Ben-Tal | first1 = A. | last2 = Nemirovski | first2 = A. | year = 1998 | title = Robust Convex Optimization | journal = Mathematics of Operations Research | volume = 23 | issue = 4| pages = 769–805 | doi=10.1287/moor.23.4.769| citeseerx = 10.1.1.135.798 | s2cid = 15905691 }}
• {{cite journal | last1 = Ben-Tal | first1 = A. | last2 = Nemirovski | first2 = A. | year = 1999 | title = Robust solutions to uncertain linear programs | journal = Operations Research Letters | volume = 25 | pages = 1–13 | doi=10.1016/s0167-6377(99)00016-4| citeseerx = 10.1.1.424.861 }}
• {{cite journal | last1 = Ben-Tal | first1 = A. | last2 = Arkadi Nemirovski | first2 = A. | year = 2002 | title = Robust optimization—methodology and applications | journal = Mathematical Programming, Series B | volume = 92 | issue = 3| pages = 453–480 | doi=10.1007/s101070100286| citeseerx = 10.1.1.298.7965 | s2cid = 1429482 }}
• Ben-Tal A., El Ghaoui, L. and Nemirovski, A. (2006). Mathematical Programming, Special issue on Robust Optimization, Volume 107(1-2).
• Ben-Tal A., El Ghaoui, L. and Nemirovski, A. (2009). Robust Optimization. Princeton Series in Applied Mathematics, Princeton University Press.
• {{cite journal | last1 = Bertsimas | first1 = D. | last2 = Sim | first2 = M. | year = 2003 | title = Robust Discrete Optimization and Network Flows | journal = Mathematical Programming | volume = 98 | issue = 1–3| pages = 49–71 | doi=10.1007/s10107-003-0396-4| citeseerx = 10.1.1.392.4470 | s2cid = 1279073 }}
• {{cite journal | last1 = Bertsimas | first1 = D. | last2 = Sim | first2 = M. | year = 2006 | title = Tractable Approximations to Robust Conic Optimization Problems Dimitris Bertsimas | journal = Mathematical Programming | volume = 107 | issue = 1| pages = 5–36 | doi=10.1007/s10107-005-0677-1| citeseerx = 10.1.1.207.8378 | s2cid = 900938 }}
• {{cite journal | last1 = Chen | first1 = W. | last2 = Sim | first2 = M. | year = 2009 | title = Goal Driven Optimization | journal = Operations Research | volume = 57 | issue = 2| pages = 342–357 | doi=10.1287/opre.1080.0570}}
• {{cite journal | last1 = Chen | first1 = X. | last2 = Sim | first2 = M. | last3 = Sun | first3 = P. | last4 = Zhang | first4 = J. | year = 2008 | title = A Linear-Decision Based Approximation Approach to Stochastic Programming | journal = Operations Research | volume = 56 | issue = 2| pages = 344–357 | doi=10.1287/opre.1070.0457}}
• {{cite journal | last1 = Chen | first1 = X. | last2 = Sim | first2 = M. | last3 = Sun | first3 = P. | year = 2007 | title = A Robust Optimization Perspective on Stochastic Programming | journal = Operations Research | volume = 55 | issue = 6| pages = 1058–1071 | doi=10.1287/opre.1070.0441}}
• {{cite journal | last1 = Dembo | first1 = R | year = 1991 | title = Scenario optimization | journal = Annals of Operations Research | volume = 30 | issue = 1| pages = 63–80 | doi=10.1007/bf02204809| s2cid = 44126126 }}
• Dodson, B., Hammett, P., and Klerx, R. (2014) Probabilistic Design for Optimization and Robustness for Engineers John Wiley & Sons, Inc. {{ISBN|978-1-118-79619-1}}
• {{cite journal | last1 = Gupta | first1 = S.K. | last2 = Rosenhead | first2 = J. | year = 1968 | title = Robustness in sequential investment decisions | doi = 10.1287/mnsc.15.2.B18 | journal = Management Science | volume = 15 | issue = 2| pages = 18–29 }}
• Kouvelis P. and Yu G. (1997). Robust Discrete Optimization and Its Applications, Kluwer.
• {{cite journal | last1 = Mutapcic | first1 = Almir | last2 = Boyd | first2 = Stephen | year = 2009 | title = Cutting-set methods for robust convex optimization with pessimizing oracles | journal = Optimization Methods and Software | volume = 24 | issue = 3| pages = 381–406 | doi=10.1080/10556780802712889| citeseerx = 10.1.1.416.4912 | s2cid = 16443437 }}
• {{cite journal | last1 = Mulvey | first1 = J.M. | last2 = Vanderbei | first2 = R.J. | last3 = Zenios | first3 = S.A. | year = 1995 | title = Robust Optimization of Large-Scale Systems | journal = Operations Research | volume = 43 | issue = 2| pages = 264–281 | doi=10.1287/opre.43.2.264}}
• Nejadseyfi, O., Geijselaers H.J.M, van den Boogaard A.H. (2018). "Robust optimization based on analytical evaluation of uncertainty propagation". Engineering Optimization 51 (9): 1581-1603. doi:10.1080/0305215X.2018.1536752.
• {{cite journal | last1 = Rosenblat | first1 = M.J. | year = 1987 | title = A robust approach to facility design | journal = International Journal of Production Research | volume = 25 | issue = 4| pages = 479–486 | doi = 10.1080/00207548708919855 }}
• {{cite journal | last1 = Rosenhead | first1 = M.J | last2 = Elton | first2 = M | last3 = Gupta | first3 = S.K. | year = 1972 | title = Robustness and Optimality as Criteria for Strategic Decisions | journal = Operational Research Quarterly | volume = 23 | issue = 4| pages = 413–430 | doi=10.2307/3007957| jstor = 3007957 }}
• Rustem B. and Howe M. (2002). Algorithms for Worst-case Design and Applications to Risk Management, Princeton University Press.
• {{cite journal | last1 = Sniedovich | first1 = M | year = 2007 | title = The art and science of modeling decision-making under severe uncertainty | journal = Decision Making in Manufacturing and Services| volume = 1 | issue = 1–2| pages = 111–136 | doi = 10.7494/dmms.2007.1.2.111 | doi-access = free }}
• {{cite journal | last1 = Sniedovich | first1 = M | year = 2008 | title = Wald's Maximin Model: a Treasure in Disguise! | journal = Journal of Risk Finance | volume = 9 | issue = 3| pages = 287–291 | doi=10.1108/15265940810875603}}
• {{cite journal | last1 = Sniedovich | first1 = M | year = 2010 | title = A bird's view of info-gap decision theory | journal = Journal of Risk Finance | volume = 11 | issue = 3| pages = 268–283 | doi=10.1108/15265941011043648}}
• {{cite journal | last1 = Wald | first1 = A | year = 1939 | title = Contributions to the theory of statistical estimation and testing hypotheses | journal = The Annals of Mathematical Statistics| volume = 10 | issue = 4| pages = 299–326 | doi=10.1214/aoms/1177732144| doi-access = free }}
• {{cite journal | last1 = Wald | first1 = A | year = 1945 | title = Statistical decision functions which minimize the maximum risk | journal = The Annals of Mathematics | volume = 46 | issue = 2| pages = 265–280 | doi=10.2307/1969022| jstor = 1969022 }}
• Wald, A. (1950). Statistical Decision Functions, John Wiley, NY.
• {{cite book |doi=10.1109/IranianCEE.2015.7146458|isbn=978-1-4799-1972-7|chapter=Generation Maintenance Scheduling via robust optimization|title=2015 23rd Iranian Conference on Electrical Engineering|year=2015|last1=Shabanzadeh|first1=Morteza|last2=Fattahi|first2=Mohammad|pages=1504–1509|s2cid=8774918 }}

• [https://www.robustopt.com ROME: Robust Optimization Made Easy]
• [http://robust.moshe-online.com: Robust Decision-Making Under Severe Uncertainty]
• [https://robustimizer.com/ Robustimizer: Robust optimization software]

{{Major subfields of optimization}}

Category:Mathematical optimization