Optimization and Control
- [1] arXiv:2405.20498 [pdf, ps, html, other]
-
Title: Robust Decentralized Control of Coupled Systems via Risk Sensitive Control of Decoupled or Simple Models with Measure ChangeSubjects: Optimization and Control (math.OC); Probability (math.PR)
Decentralized stochastic control problems with local information involve problems where multiple agents and subsystems which are coupled via dynamics and/or cost are present. Typically, however, the dynamics of such couplings is complex and difficult to precisely model, leading to questions on robustness in control design. Additionally, when such a coupling can be modeled, the problem arrived at is typically challenging and non-convex, due to decentralization of information. In this paper, we develop a robustness framework for optimal decentralized control of interacting agents, where we show that a decentralized control problem with interacting agents can be robustly designed by considering a risk-sensitive version of non-interacting agents/particles. This leads to a tractable robust formulation where we give a bound on the value of the cost function in terms of the risk-sensitive cost function for the non-interacting case plus a term involving the ``strength" of the interaction as measured by relative entropy. We will build on Gaussian measure theory and an associated variational equality. A particular application includes mean-field models consisting of (a generally large number of) interacting agents which are often hard to solve for the case with small or moderate numbers of agents, leading to an interest in effective approximations and robustness. By adapting a risk-sensitivity parameter, we also robustly control a non-symmetrically interacting problem with mean-field cost by one which is symmetric with a risk-sensitive criterion, and in the limit of small interactions, show the stability of optimal solutions to perturbations.
- [2] arXiv:2405.20500 [pdf, ps, html, other]
-
Title: Hybrid Reinforcement Learning Framework for Mixed-Variable ProblemsSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG)
Optimization problems characterized by both discrete and continuous variables are common across various disciplines, presenting unique challenges due to their complex solution landscapes and the difficulty of navigating mixed-variable spaces effectively. To Address these challenges, we introduce a hybrid Reinforcement Learning (RL) framework that synergizes RL for discrete variable selection with Bayesian Optimization for continuous variable adjustment. This framework stands out by its strategic integration of RL and continuous optimization techniques, enabling it to dynamically adapt to the problem's mixed-variable nature. By employing RL for exploring discrete decision spaces and Bayesian Optimization to refine continuous parameters, our approach not only demonstrates flexibility but also enhances optimization performance. Our experiments on synthetic functions and real-world machine learning hyperparameter tuning tasks reveal that our method consistently outperforms traditional RL, random search, and standalone Bayesian optimization in terms of effectiveness and efficiency.
- [3] arXiv:2405.20528 [pdf, ps, html, other]
-
Title: Convergence Analysis of the Sinkhorn Algorithm with Sparse Cost MatricesSubjects: Optimization and Control (math.OC); Statistics Theory (math.ST)
This paper presents a theoretical analysis of the convergence rate of the Sinkhorn algorithm when the cost matrix is sparse. We derive bounds on the convergence rate that depend on the sparsity pattern and the degree of sparsity of the cost matrix. We also explore whether existing convergence results for dense cost matrices can be adapted or improved for the sparse case. Our analysis provides new insights into the behavior of the Sinkhorn algorithm in the presence of sparsity and highlights potential avenues for algorithmic improvements.
- [4] arXiv:2405.20737 [pdf, ps, html, other]
-
Title: Regular Subgradients of Marginal Functions with Applications to Calculus and Bilevel ProgrammingSubjects: Optimization and Control (math.OC)
The paper addresses the study and applications of a broad class of extended-real-valued functions, known as optimal value or marginal functions, which are frequently appeared in variational analysis, parametric optimization, and a variety of applications. Functions of this type are intrinsically nonsmooth and require the usage of tools of generalized differentiation. The main results of this paper provide novel evaluations and exact calculations of regular/Fréchet subgradients and their singular counterparts for general classes of marginal functions via their given data. The obtained results are applied to establishing new calculus rules for such subgradients and necessary optimality conditions in bilevel programming
- [5] arXiv:2405.20744 [pdf, ps, html, other]
-
Title: On the sequential convergence of Lloyd's algorithmsSubjects: Optimization and Control (math.OC)
Lloyd's algorithm is an iterative method that solves the quantization problem, i.e. the approximation of a target probability measure by a discrete one, and is particularly used in digital applications.This algorithm can be interpreted as a gradient method on a certain quantization functional which is given by optimal transport. We study the sequential convergence (to a single accumulation point) for two variants of Lloyd's method: (i) optimal quantization with an arbitrary discrete measure and (ii) uniform quantization with a uniform discrete measure. For both cases, we prove sequential convergence of the iterates under an analiticity assumption on the density of the target measure. This includes for example analytic densities truncated to a compact semi-algebraic set. The argument leverages the log analytic nature of globally subanalytic integrals, the interpretation of Lloyd's method as a gradient method and the convergence analysis of gradient algorithms under Kurdyka-Lojasiewicz assumptions. As a by-product, we also obtain definability results for more general semi-discrete optimal transport losses such as transport distances with general costs, the max-sliced Wasserstein distance and the entropy regularized optimal transport loss.
- [6] arXiv:2405.21023 [pdf, ps, html, other]
-
Title: Compact Optimality Verification for Optimization ProxiesComments: International Conference on Machine Learning 2024Subjects: Optimization and Control (math.OC); Artificial Intelligence (cs.AI)
Recent years have witnessed increasing interest in optimization proxies, i.e., machine learning models that approximate the input-output mapping of parametric optimization problems and return near-optimal feasible solutions. Following recent work by (Nellikkath & Chatzivasileiadis, 2021), this paper reconsiders the optimality verification problem for optimization proxies, i.e., the determination of the worst-case optimality gap over the instance distribution. The paper proposes a compact formulation for optimality verification and a gradient-based primal heuristic that brings substantial computational benefits to the original formulation. The compact formulation is also more general and applies to non-convex optimization problems. The benefits of the compact formulation are demonstrated on large-scale DC Optimal Power Flow and knapsack problems.
New submissions for Monday, 3 June 2024 (showing 6 of 6 entries )
- [7] arXiv:2405.19499 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous EnvironmentsJournal-ref: Proceedings of the 41st International Conference on Machine Learning, 2024 LearningSubjects: Machine Learning (cs.LG); Multiagent Systems (cs.MA); Optimization and Control (math.OC)
We explore a Federated Reinforcement Learning (FRL) problem where $N$ agents collaboratively learn a common policy without sharing their trajectory data. To date, existing FRL work has primarily focused on agents operating in the same or ``similar" environments. In contrast, our problem setup allows for arbitrarily large levels of environment heterogeneity. To obtain the optimal policy which maximizes the average performance across all potentially completely different environments, we propose two algorithms: FedSVRPG-M and FedHAPG-M. In contrast to existing results, we demonstrate that both FedSVRPG-M and FedHAPG-M, both of which leverage momentum mechanisms, can exactly converge to a stationary point of the average performance function, regardless of the magnitude of environment heterogeneity. Furthermore, by incorporating the benefits of variance-reduction techniques or Hessian approximation, both algorithms achieve state-of-the-art convergence results, characterized by a sample complexity of $\mathcal{O}\left(\epsilon^{-\frac{3}{2}}/N\right)$. Notably, our algorithms enjoy linear convergence speedups with respect to the number of agents, highlighting the benefit of collaboration among agents in finding a common policy.
- [8] arXiv:2405.20387 (cross-list from eess.SY) [pdf, ps, other]
-
Title: Sensitivity Analysis for Piecewise-Affine Approximations of Nonlinear Programs with Polytopic ConstraintsComments: 6 pages, 4 figures, accepted for publication in IEEE Control Systems LettersSubjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
Nonlinear Programs (NLPs) are prevalent in optimization-based control of nonlinear systems. Solving general NLPs is computationally expensive, necessitating the development of fast hardware or tractable suboptimal approximations. This paper investigates the sensitivity of the solutions of NLPs with polytopic constraints when the nonlinear continuous objective function is approximated by a PieceWise-Affine (PWA) counterpart. By leveraging perturbation analysis using a convex modulus, we derive guaranteed bounds on the distance between the optimal solution of the original polytopically-constrained NLP and that of its approximated formulation. Our approach aids in determining criteria for achieving desired solution bounds. Two case studies on the Eggholder function and nonlinear model predictive control of an inverted pendulum demonstrate the theoretical results.
- [9] arXiv:2405.20390 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Quantitative Convergences of Lie Group Momentum OptimizersSubjects: Machine Learning (cs.LG); Numerical Analysis (math.NA); Optimization and Control (math.OC); Machine Learning (stat.ML)
Explicit, momentum-based dynamics that optimize functions defined on Lie groups can be constructed via variational optimization and momentum trivialization. Structure preserving time discretizations can then turn this dynamics into optimization algorithms. This article investigates two types of discretization, Lie Heavy-Ball, which is a known splitting scheme, and Lie NAG-SC, which is newly proposed. Their convergence rates are explicitly quantified under $L$-smoothness and local strong convexity assumptions. Lie NAG-SC provides acceleration over the momentumless case, i.e. Riemannian gradient descent, but Lie Heavy-Ball does not. When compared to existing accelerated optimizers for general manifolds, both Lie Heavy-Ball and Lie NAG-SC are computationally cheaper and easier to implement, thanks to their utilization of group structure. Only gradient oracle and exponential map are required, but not logarithm map or parallel transport which are computational costly.
- [10] arXiv:2405.20449 (cross-list from eess.SY) [pdf, ps, html, other]
-
Title: Optimization, guidance, and control of low-thrust transfers from the Lunar Gateway to low lunar orbitComments: 19 pages, 12 figures, IAC 2023, ACTA ASTRONAUTICA 2024Subjects: Systems and Control (eess.SY); Dynamical Systems (math.DS); Optimization and Control (math.OC); Space Physics (physics.space-ph)
The Gateway will represent a primary space system useful for the Artemis program, Earth-Moon transportation, and deep space exploration. It is expected to serve as a staging location on the way to the lunar surface. This study focuses on low-thrust transfer dynamics, from the Near-Rectilinear Halo Orbit traveled by Gateway to a specified Low-altitude Lunar Orbit (LLO). This research addresses: (i) determination of the minimum-time low-thrust trajectory and (ii) design, implementation, and testing of a guidance and control architecture, for a space vehicle that travels from Gateway to LLO. Orbit dynamics is described in terms of modified equinoctial elements, in the context of a high-fidelity ephemeris model. The minimum-time trajectory from Gateway to a specified lunar orbit is detected through an indirect heuristic approach, which uses the analytical conditions arising in optimal control theory in conjunction with a heuristic technique. However, future missions will pursue a growing level of autonomy, and this circumstance implies the mandatory design of an efficient feedback guidance scheme, capable of compensating for nonnominal flight conditions. This research proposes nonlinear orbit control as a viable option for autonomous explicit guidance of low-thrust transfers from Gateway to LLO. This approach allows defining a feedback law that enjoys quasi-global stability properties without requiring any offline reference trajectory. The overall spacecraft dynamics is modeled including attitude control and actuation. The latter is demanded to an array of reaction wheels, arranged in a pyramidal configuration. Guidance, attitude control, and actuation are implemented in an iterative scheme. Monte Carlo simulations demonstrate that the guidance and control architecture is effective with random starting points from Gateway and the temporary unavailability of the propulsion system.
- [11] arXiv:2405.20451 (cross-list from stat.ML) [pdf, ps, html, other]
-
Title: Statistical Properties of Robust SatisficingSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC)
The Robust Satisficing (RS) model is an emerging approach to robust optimization, offering streamlined procedures and robust generalization across various applications. However, the statistical theory of RS remains unexplored in the literature. This paper fills in the gap by comprehensively analyzing the theoretical properties of the RS model. Notably, the RS structure offers a more straightforward path to deriving statistical guarantees compared to the seminal Distributionally Robust Optimization (DRO), resulting in a richer set of results. In particular, we establish two-sided confidence intervals for the optimal loss without the need to solve a minimax optimization problem explicitly. We further provide finite-sample generalization error bounds for the RS optimizer. Importantly, our results extend to scenarios involving distribution shifts, where discrepancies exist between the sampling and target distributions. Our numerical experiments show that the RS model consistently outperforms the baseline empirical risk minimization in small-sample regimes and under distribution shifts. Furthermore, compared to the DRO model, the RS model exhibits lower sensitivity to hyperparameter tuning, highlighting its practicability for robustness considerations.
- [12] arXiv:2405.20467 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Performance of NPG in Countable State-Space Average-Cost RLComments: 23 pagesSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
We consider policy optimization methods in reinforcement learning settings where the state space is arbitrarily large, or even countably infinite. The motivation arises from control problems in communication networks, matching markets, and other queueing systems. We consider Natural Policy Gradient (NPG), which is a popular algorithm for finite state spaces. Under reasonable assumptions, we derive a performance bound for NPG that is independent of the size of the state space, provided the error in policy evaluation is within a factor of the true value function. We obtain this result by establishing new policy-independent bounds on the solution to Poisson's equation, i.e., the relative value function, and by combining these bounds with previously known connections between MDPs and learning from experts.
- [13] arXiv:2405.20502 (cross-list from eess.SY) [pdf, ps, html, other]
-
Title: Reach-Avoid Control Synthesis for a Quadrotor UAV with Formal Safety GuaranteesSubjects: Systems and Control (eess.SY); Dynamical Systems (math.DS); Optimization and Control (math.OC)
Reach-avoid specifications are one of the most common tasks in autonomous aerial vehicle (UAV) applications. Despite the intensive research and development associated with control of aerial vehicles, generating feasible trajectories though complex environments and tracking them with formal safety guarantees remain challenging. In this paper, we propose a control framework for a quadrotor UAV that enables accomplishing reach-avoid tasks with formal safety guarantees. In this proposed framework, we integrate geometric control theory for tracking and polynomial trajectory generation using Bezier curves, where tracking errors are accounted for in the trajectory synthesis process. To estimate the tracking errors, we revisit the stability analysis of the closed-loop quadrotor system, when geometric control is implemented. We show that the tracking error dynamics exhibit local exponential stability when geometric control is implemented with any positive control gains, and we derive tight uniform bounds of the tracking error. We also introduce sufficient conditions to be imposed on the desired trajectory utilizing the derived uniform bounds to ensure the well-definedness of the closed-loop system. For the trajectory synthesis, we present an efficient algorithm that enables constructing a safe tube by means of sampling-based planning and safe hyper-rectangular set computations. Then, we compute the trajectory, given as a piecewise continuous Bezier curve, through the safe tube, where a heuristic efficient approach that utilizes iterative linear programming is employed. We present extensive numerical simulations with a cluttered environment to illustrate the effectiveness of the proposed framework in reach-avoid planning scenarios.
- [14] arXiv:2405.20540 (cross-list from cs.LG) [pdf, ps, other]
-
Title: Fully Unconstrained Online LearningSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
We provide an online learning algorithm that obtains regret $G\|w_\star\|\sqrt{T\log(\|w_\star\|G\sqrt{T})} + \|w_\star\|^2 + G^2$ on $G$-Lipschitz convex losses for any comparison point $w_\star$ without knowing either $G$ or $\|w_\star\|$. Importantly, this matches the optimal bound $G\|w_\star\|\sqrt{T}$ available with such knowledge (up to logarithmic factors), unless either $\|w_\star\|$ or $G$ is so large that even $G\|w_\star\|\sqrt{T}$ is roughly linear in $T$. Thus, it matches the optimal bound in all cases in which one can achieve sublinear regret, which arguably most "interesting" scenarios.
- [15] arXiv:2405.20623 (cross-list from cs.LG) [pdf, ps, other]
-
Title: Prune at the Clients, Not the Server: Accelerated Sparse Training in Federated LearningSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
In the recent paradigm of Federated Learning (FL), multiple clients train a shared model while keeping their local data private. Resource constraints of clients and communication costs pose major problems for training large models in FL. On the one hand, addressing the resource limitations of the clients, sparse training has proven to be a powerful tool in the centralized setting. On the other hand, communication costs in FL can be addressed by local training, where each client takes multiple gradient steps on its local data. Recent work has shown that local training can provably achieve the optimal accelerated communication complexity [Mishchenko et al., 2022]. Hence, one would like an accelerated sparse training algorithm. In this work we show that naive integration of sparse training and acceleration at the server fails, and how to fix it by letting the clients perform these tasks appropriately. We introduce Sparse-ProxSkip, our method developed for the nonconvex setting, inspired by RandProx [Condat and Richtárik, 2022], which provably combines sparse training and acceleration in the convex setting. We demonstrate the good performance of Sparse-ProxSkip in extensive experiments.
- [16] arXiv:2405.20682 (cross-list from eess.SY) [pdf, ps, html, other]
-
Title: Impact of Phase Selection on Accuracy and Scalability in Calculating Distributed Energy Resources Hosting CapacitySubjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
Hosting capacity (HC) and dynamic operating envelopes (DOEs), defined as dynamic, time-varying HC, are calculated using three-phase optimal power flow (OPF) formulations. Due to the computational complexity of such optimisation problems, HC and DOE are often calculated by introducing certain assumptions and approximations, including the linearised OPF formulation, which we implement in the Python-based tool ppOPF. Furthermore, we investigate how assumptions of the distributed energy resource (DER) connection phase impact the objective function value and computational time in calculating HC and DOE in distribution networks of different sizes. The results are not unambiguous and show that it is not possible to determine the optimal connection phase without introducing binary variables since, no matter the case study, the highest objective function values are calculated with mixed integer OPF formulations. The difference is especially visible in a real-world low-voltage network in which the difference between different scenarios is up to 14 MW in a single day. However, binary variables make the problem computationally complex and increase computational time to several hours in the DOE calculation, even when the optimality gap different from zero is set.
- [17] arXiv:2405.20747 (cross-list from q-bio.QM) [pdf, ps, html, other]
-
Title: Generalized Inverse Optimal Control and its Application in BiologySubjects: Quantitative Methods (q-bio.QM); Optimization and Control (math.OC)
Living organisms exhibit remarkable adaptations across all scales, from molecules to ecosystems. We believe that many of these adaptations correspond to optimal solutions driven by evolution, training, and underlying physical and chemical laws and constraints. While some argue against such optimality principles due to their potential ambiguity, we propose generalized inverse optimal control to infer them directly from data. This novel approach incorporates multi-criteria optimality, nestedness of objective functions on different scales, the presence of active constraints, the possibility of switches of optimality principles during the observed time horizon, maximization of robustness, and minimization of time as important special cases, as well as uncertainties involved with the mathematical modeling of biological systems. This data-driven approach ensures that optimality principles are not merely theoretical constructs but are firmly rooted in experimental observations. Furthermore, the inferred principles can be used in forward optimal control to predict and manipulate biological systems, with possible applications in bio-medicine, biotechnology, and agriculture. As discussed and illustrated, the well-posed problem formulation and the inference are challenging and require a substantial interdisciplinary effort in the development of theory and robust numerical methods.
- [18] arXiv:2405.20763 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Improving Generalization and Convergence by Enhancing Implicit RegularizationMingze Wang, Haotian He, Jinbo Wang, Zilin Wang, Guanhua Huang, Feiyu Xiong, Zhiyu Li, Weinan E, Lei WuComments: 35 pagesSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
In this work, we propose an Implicit Regularization Enhancement (IRE) framework to accelerate the discovery of flat solutions in deep learning, thereby improving generalization and convergence. Specifically, IRE decouples the dynamics of flat and sharp directions, which boosts the sharpness reduction along flat directions while maintaining the training stability in sharp directions. We show that IRE can be practically incorporated with {\em generic base optimizers} without introducing significant computational overload. Experiments show that IRE consistently improves the generalization performance for image classification tasks across a variety of benchmark datasets (CIFAR-10/100, ImageNet) and models (ResNets and ViTs). Surprisingly, IRE also achieves a $2\times$ {\em speed-up} compared to AdamW in the pre-training of Llama models (of sizes ranging from 60M to 229M) on datasets including Wikitext-103, Minipile, and Openwebtext. Moreover, we provide theoretical guarantees, showing that IRE can substantially accelerate the convergence towards flat minima in Sharpness-aware Minimization (SAM).
- [19] arXiv:2405.20912 (cross-list from econ.GN) [pdf, ps, html, other]
-
Title: A Branch-Price-Cut-And-Switch Approach for Optimizing Team Formation and Routing for Airport Baggage Handling Tasks with Stochastic Travel TimesSubjects: General Economics (econ.GN); Optimization and Control (math.OC)
In airport operations, optimally using dedicated personnel for baggage handling tasks plays a crucial role in the design of resource-efficient processes. Teams of workers with different qualifications must be formed, and loading or unloading tasks must be assigned to them. Each task has a time window within which it can be started and should be finished. Violating these temporal restrictions incurs severe financial penalties for the operator. In practice, various components of this process are subject to uncertainties. We consider the aforementioned problem under the assumption of stochastic travel times across the apron. We present two binary program formulations to model the problem at hand and solve it with a Branch-Price-Cut-and-Switch approach, in which we dynamically switch between two master problem formulations. Furthermore, we use an exact separation method to identify violated rank-1 Chvátal-Gomory cuts and utilize an efficient branching rule relying on task finish times. We test the algorithm on instances generated based on real-world data from a major European hub airport with a planning horizon of up to two hours, 30 flights per hour, and three available task execution modes to choose from. Our results indicate that our algorithm is able to significantly outperform existing solution approaches. Moreover, an explicit consideration of stochastic travel times allows for solutions that utilize the available workforce more efficiently, while simultaneously guaranteeing a stable service level for the baggage handling operator.
- [20] arXiv:2405.21064 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Recurrent neural networks: vanishing and exploding gradients are not the end of the storySubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC)
Recurrent neural networks (RNNs) notoriously struggle to learn long-term memories, primarily due to vanishing and exploding gradients. The recent success of state-space models (SSMs), a subclass of RNNs, to overcome such difficulties challenges our theoretical understanding. In this paper, we delve into the optimization challenges of RNNs and discover that, as the memory of a network increases, changes in its parameters result in increasingly large output variations, making gradient-based learning highly sensitive, even without exploding gradients. Our analysis further reveals the importance of the element-wise recurrence design pattern combined with careful parametrizations in mitigating this effect. This feature is present in SSMs, as well as in other architectures, such as LSTMs. Overall, our insights provide a new explanation for some of the difficulties in gradient-based learning of RNNs and why some architectures perform better than others.
Cross submissions for Monday, 3 June 2024 (showing 14 of 14 entries )
- [21] arXiv:2210.16678 (replaced) [pdf, ps, html, other]
-
Title: Curse of Scale-Freeness: Intractability of Large-Scale Combinatorial Optimization with Multi-Start MethodsComments: Submission to Extremes, March 14th, 2024Subjects: Optimization and Control (math.OC)
This paper investigates the intractability of large-scale combinatorial optimization with multi-start methods. For the theoretical performance analysis, we focus on the random multi-start (RMS) method, one of the representative multi-start methods including the RMS local search and the greedy randomized adaptive search procedure (GRASP). Our main theoretical contribution is to derive two power-law formulas using extreme value theory. One is for the expected improvement rate (EIR) of the best empirical objective value (EOV), and the other is for the expected relative gap (ERG) of the best EOV to the supremum of empirical objective values. Notably, the ERG has scale-freeness as a function of the number of iterations. Thus, the ``half-life" of the ERG is eventually proportional to the number of iterations completed by an RMS algorithm. This result can be understood as the curse of scale-freeness: a Zeno's paradox-like phenomenon of ``trying to get to the goal pushes it far away," Through numerical experiments, we observe that applying several RMS algorithms to traveling salesman problems is subject to the curse of scale-freeness. Furthermore, we show that overcoming the curse of scale-freeness requires the development of an algorithm with exponentially accelerated performance compared to the RMS method.
- [22] arXiv:2307.00490 (replaced) [pdf, ps, html, other]
-
Title: Riemannian Trust Region Methods for SC$^1$ MinimizationSubjects: Optimization and Control (math.OC)
Manifold optimization has recently gained significant attention due to its wide range of applications in various areas. This paper introduces the first Riemannian trust region method for minimizing an SC$^1$ function, which is a differentiable function that has a semismooth gradient vector field, on manifolds with convergence guarantee. We provide proof of both global and local convergence results, along with demonstrating the local superlinear convergence rate of our proposed method. As an application and to demonstrate our motivation, we utilize our trust region method as a subproblem solver within an augmented Lagrangian method for minimizing nonsmooth nonconvex functions over manifolds. This represents the first approach that fully explores the second-order information of the subproblem in the context of augmented Lagrangian methods on manifolds. Numerical experiments confirm that our method outperforms existing methods.
- [23] arXiv:2307.16128 (replaced) [pdf, ps, html, other]
-
Title: Online Interior-point Methods for Time-varying Equality-constrained OptimizationSubjects: Optimization and Control (math.OC); Systems and Control (eess.SY)
An important challenge in the online convex optimization (OCO) setting is to incorporate generalized inequalities and time-varying constraints. The inclusion of constraints in OCO widens the applicability of such algorithms to dynamic and safety-critical settings such as the online optimal power flow (OPF) problem. In this work, we propose the first projection-free OCO algorithm admitting time-varying linear constraints and convex generalized inequalities: the online interior-point method for time-varying equality constraints (OIPM-TEC). We derive simultaneous sublinear dynamic regret and constraint violation bounds for OIPM-TEC under standard assumptions. For applications where a given tolerance around optima is accepted, we employ an alternative OCO performance metric -- the epsilon-regret -- and a more computationally efficient algorithm, the epsilon-OIPM-TEC, that possesses sublinear bounds under this metric. Finally, we showcase the performance of these two algorithms on an online OPF problem and compare them to another OCO algorithm from the literature.
- [24] arXiv:2308.09310 (replaced) [pdf, ps, html, other]
-
Title: Variance reduction techniques for stochastic proximal point algorithmsSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG)
In the context of finite sums minimization, variance reduction techniques are widely used to improve the performance of state-of-the-art stochastic gradient methods. Their practical impact is clear, as well as their theoretical properties. Stochastic proximal point algorithms have been studied as an alternative to stochastic gradient algorithms since they are more stable with respect to the choice of the stepsize but their variance reduced versions are not as studied as the gradient ones. In this work, we propose the first unified study of variance reduction techniques for stochastic proximal point algorithms. We introduce a generic stochastic proximal algorithm that can be specified to give the proximal version of SVRG, SAGA, and some of their variants for smooth and convex functions. We provide several convergence results for the iterates and the objective function values. In addition, under the Polyak-Łojasiewicz (PL) condition, we obtain linear convergence rates for the iterates and the function values. Our numerical experiments demonstrate the advantages of the proximal variance reduction methods over their gradient counterparts, especially about the stability with respect to the choice of the stepsize for difficult problems.
- [25] arXiv:2308.14143 (replaced) [pdf, ps, html, other]
-
Title: Ensemble-localized Kernel Density Estimation with Applications to the Ensemble Gaussian Mixture FilterSubjects: Optimization and Control (math.OC); Numerical Analysis (math.NA); Applications (stat.AP)
The ensemble Gaussian mixture filter (EnGMF) is a non-linear filter suited to data assimilation of highly non-Gaussian and non-linear models that has practical utility in the case of a small number of samples, and theoretical convergence to full Bayesian inference in the ensemble limit. We aim to increase the utility of the EnGMF by introducing an ensemble-local notion of covariance into the kernel density estimation (KDE) step for the prior distribution. We prove that in the Gaussian case, our new ensemble-localized KDE technique is exactly the same as more traditional KDE techniques. We also show an example of a non-Gaussian distribution that can fail to be approximated by canonical KDE methods, but can be approximated well by our new KDE technique. We showcase our new KDE technique on a simple bivariate problem, showing that it has nice qualitative and quantitative properties, and significantly improves the estimate of the prior and posterior distributions for all ensemble sizes tested. We additionally show the utility of the proposed methodology for sequential filtering for the Lorenz '63 equations, achieving a significant reduction in error, and less conservative behavior in the uncertainty estimate with respect to traditional techniques.
- [26] arXiv:2311.07847 (replaced) [pdf, ps, html, other]
-
Title: Approximate Bregman Proximal Gradient Algorithm for Relatively Smooth Nonconvex OptimizationComments: 31 pages, 20 figuresSubjects: Optimization and Control (math.OC)
In this paper, we propose the approximate Bregman proximal gradient algorithm (ABPG) for solving composite nonconvex optimization problems. ABPG employs a new distance that approximates the Bregman distance, making the subproblem of ABPG simpler to solve compared to existing Bregman-type algorithms. The subproblem of ABPG is often expressed in a closed form. Similarly to existing Bregman-type algorithms, ABPG does not require the global Lipschitz continuity for the gradient of the smooth part. Instead, assuming the smooth adaptable property, we establish the global subsequential convergence under standard assumptions. Additionally, assuming that the Kurdyka--Łojasiewicz property holds, we prove the global convergence for a special case. Our numerical experiments on the $\ell_p$ regularized least squares problem, the $\ell_p$ loss problem, and the nonnegative linear system show that ABPG outperforms existing algorithms especially when the gradient of the smooth part is not globally Lipschitz or even local Lipschitz continuous.
- [27] arXiv:2311.13167 (replaced) [pdf, ps, html, other]
-
Title: Regularity Properties of Optimization-Based ControllersSubjects: Optimization and Control (math.OC)
This paper studies regularity properties of optimization-based controllers, which are obtained by solving optimization problems where the parameter is the system state and the optimization variable is the input to the system. Under a wide range of assumptions on the optimization problem data, we provide an exhaustive collection of results about their regularity, and examine their implications on the existence and uniqueness of solutions and the forward invariance guarantees for the resulting closed-loop systems. We discuss the broad relevance of the results in different areas of systems and controls.
- [28] arXiv:2402.08097 (replaced) [pdf, ps, html, other]
-
Title: An Accelerated Gradient Method for Convex Smooth Simple Bilevel OptimizationSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)
In this paper, we focus on simple bilevel optimization problems, where we minimize a convex smooth objective function over the optimal solution set of another convex smooth constrained optimization problem. We present a novel bilevel optimization method that locally approximates the solution set of the lower-level problem using a cutting plane approach and employs an accelerated gradient-based update to reduce the upper-level objective function over the approximated solution set. We measure the performance of our method in terms of suboptimality and infeasibility errors and provide non-asymptotic convergence guarantees for both error criteria. Specifically, when the feasible set is compact, we show that our method requires at most $\mathcal{O}(\max\{1/\sqrt{\epsilon_{f}}, 1/\epsilon_g\})$ iterations to find a solution that is $\epsilon_f$-suboptimal and $\epsilon_g$-infeasible. Moreover, under the additional assumption that the lower-level objective satisfies the $r$-th Hölderian error bound, we show that our method achieves an iteration complexity of $\mathcal{O}(\max\{\epsilon_{f}^{-\frac{2r-1}{2r}},\epsilon_{g}^{-\frac{2r-1}{2r}}\})$, which matches the optimal complexity of single-level convex constrained optimization when $r=1$.
- [29] arXiv:2404.07402 (replaced) [pdf, ps, html, other]
-
Title: An excursion onto Schr\"odinger's bridges: Stochastic flows with spatio-temporal marginalsComments: 6 pages, 2 figuresSubjects: Optimization and Control (math.OC); Systems and Control (eess.SY); Probability (math.PR)
The purpose of the present work is to expand substantially the type of control and estimation problems that can be addressed following Schrödinger's dictum, by incorporating termination (killing) of stochastic flows. Specifically, in the context of estimation, we seek the most likely evolution realizing measured spatio-temporal marginals of killed particles. In the context of control, we seek a suitable control action directing the killed process toward spatio-temporal probabilistic constraints. To this end, we derive a new Schrödinger system of coupled, in space and time, partial differential equations to construct the solution of the proposed problem. Further, we show that a Fortet-Sinkhorn type of algorithm is, once again, available to attain the associated bridge. A key feature of our framework is that the obtained bridge retains the Markovian structure in the prior process, and thereby, the corresponding controller takes the form of state feedback.
- [30] arXiv:2404.19392 (replaced) [pdf, ps, html, other]
-
Title: Convergence analysis of the transformed gradient projection algorithms on compact matrix manifoldsComments: 45 pages, 5 figures, 4 tablesSubjects: Optimization and Control (math.OC); Numerical Analysis (math.NA)
In this paper, to address the optimization problem on a compact matrix manifold, we introduce a novel algorithmic framework called the Transformed Gradient Projection (TGP) algorithm, using the projection onto this compact matrix manifold. Compared with the existing algorithms, the key innovation in our approach lies in the utilization of a new class of search directions and various stepsizes, including the Armijo, nonmonotone Armijo, and fixed stepsizes, to guide the selection of the next iterate. Our framework offers flexibility by encompassing the classical gradient projection algorithms as special cases, and intersecting the retraction-based line-search algorithms. Notably, our focus is on the Stiefel or Grassmann manifold, revealing that many existing algorithms in the literature can be seen as specific instances within our proposed framework, and this algorithmic framework also induces several new special cases. Then, we conduct a thorough exploration of the convergence properties of these algorithms, considering various search directions and stepsizes. To achieve this, we extensively analyze the geometric properties of the projection onto compact matrix manifolds, allowing us to extend classical inequalities related to retractions from the literature. Building upon these insights, we establish the weak convergence, convergence rate, and global convergence of TGP algorithms under three distinct stepsizes. In cases where the compact matrix manifold is the Stiefel or Grassmann manifold, our convergence results either encompass or surpass those found in the literature. Finally, through a series of numerical experiments, we observe that the TGP algorithms, owing to their increased flexibility in choosing search directions, outperform classical gradient projection and retraction-based line-search algorithms in several scenarios.
- [31] arXiv:2405.15552 (replaced) [pdf, ps, html, other]
-
Title: Stability and Performance Analysis of Model Predictive Control of Uncertain Linear SystemsComments: 14 pages, 3 figures, the full version of the short paper that has been submitted to CDC2024 for potential publication (some minor changes made)Subjects: Optimization and Control (math.OC)
Model mismatch often poses challenges in model-based controller design. This paper investigates model predictive control (MPC) of uncertain linear systems with input constraints, focusing on stability and closed-loop infinite-horizon performance. The uncertainty arises from a parametric mismatch between the true and the estimated system under the matrix Frobenius norm. We examine a simple MPC controller that exclusively uses the estimated system model and establishes sufficient conditions under which the MPC controller can stabilize the true system. Moreover, we derive a theoretical performance bound based on relaxed dynamic programming, elucidating the impact of prediction horizon and modeling errors on the suboptimality gap between the MPC controller and the Oracle infinite-horizon optimal controller with knowledge of the true system. Simulations of a numerical example validate the theoretical results. Our theoretical analysis offers guidelines for obtaining the desired modeling accuracy and choosing a proper prediction horizon to develop certainty-equivalent MPC controllers for uncertain linear systems.
- [32] arXiv:2205.00825 (replaced) [pdf, ps, other]
-
Title: Stochastic Online Fisher Markets: Static Pricing Limits and Adaptive EnhancementsSubjects: Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG); Theoretical Economics (econ.TH); Optimization and Control (math.OC)
Fisher markets are one of the most fundamental models for resource allocation. However, the problem of computing equilibrium prices in Fisher markets typically relies on complete knowledge of users' budgets and utility functions and requires transactions to happen in a static market where all users are present simultaneously. Motivated by these practical considerations, we study an online variant of Fisher markets, wherein users with privately known utility and budget parameters, drawn i.i.d. from a distribution, arrive sequentially. In this setting, we first study the limitations of static pricing algorithms, which set uniform prices for all users, along two performance metrics: (i) regret, i.e., the optimality gap in the objective of the Eisenberg-Gale program between an online algorithm and an oracle with complete information, and (ii) capacity violations, i.e., the over-consumption of goods relative to their capacities. Given the limitations of static pricing, we design adaptive posted-pricing algorithms, one with knowledge of the distribution of users' budget and utility parameters and another that adjusts prices solely based on past observations of user consumption, i.e., revealed preference feedback, with improved performance guarantees. Finally, we present numerical experiments to compare our revealed preference algorithm's performance to several benchmarks.
- [33] arXiv:2405.15682 (replaced) [pdf, ps, html, other]
-
Title: The Road Less ScheduledSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC); Machine Learning (stat.ML)
Existing learning rate schedules that do not require specification of the optimization stopping step T are greatly out-performed by learning rate schedules that depend on T. We propose an approach that avoids the need for this stopping time by eschewing the use of schedules entirely, while exhibiting state-of-the-art performance compared to schedules across a wide family of problems ranging from convex problems to large-scale deep learning problems. Our Schedule-Free approach introduces no additional hyper-parameters over standard optimizers with momentum. Our method is a direct consequence of a new theory we develop that unifies scheduling and iterate averaging. An open source implementation of our method is available (this https URL).