Stochastic Approximation: A Dynamical Systems Viewpoint

2022/01/01 by Vivek S. Borkar · 3 citations
Decision Sciences · Economics, Econometrics and Finance · #Simulation Techniques and Applications #Stochastic processes and financial applications

paper · doi:10.1007/978-81-951961-1-1

openalex publication_date 2022/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

This second edition presents a comprehensive view of the ODE-based approach for the analysis of stochastic approximation algorithms.

Cited by

• Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic\n Approximation under Markovian Noise
• Truncated Stochastic Approximation with Moving Bounds: Convergence
• On the fast convergence of random perturbations of the gradient flow
• On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games
• Robust Reinforcement Learning with Wasserstein Constraint
• Lyapunov Robust Constrained-MDPs: Soft-Constrained Robustly Stable Policy Optimization under Model Uncertainty
• Variational Policy Gradient Method for Reinforcement Learning with General Utilities
• Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signals
• Convergence Analysis of Gradient-Based Learning with Non-Uniform Learning Rates in Non-Cooperative Multi-Agent Settings
• Real-Time Stochastic Predictive Control for Hybrid Vehicle Energy Management
• Convergence of the Continuous Time Trajectories of Isotropic Evolution Strategies on Monotonic C²-composite Functions
• Optimal Matrix Momentum Stochastic Approximation and Applications to Q-learning
• Monitoring State Transitions in Markovian Systems with Sampling Cost
• Resource Allocation Optimization for Delay-Sensitive Traffic in Fronthaul Constrained Cloud Radio Access Networks
• Cumulative Prospect Theory Meets Reinforcement Learning: Prediction and\n Control
• Optimal Dynamic Sensor Subset Selection for Tracking a Time-Varying Stochastic Process
• Risk-Constrained Reinforcement Learning with Percentile Risk Criteria
• Distributed adaptive steplength stochastic approximation schemes for\n Cartesian stochastic variational inequality problems
• Finite-Sample Analysis of Proximal Gradient TD Algorithms
• Normal Approximation for Stochastic Gradient Descent via Non-Asymptotic Rates of Martingale CLT
• Robust adaptive Metropolis algorithm with coerced acceptance rate
• Gradient-Based Adaptive Stochastic Search for Non-Differentiable Optimization
• Tight High Probability Bounds for Linear Stochastic Approximation with Fixed Stepsize
• Who Leads and Who Follows in Strategic Classification?
• Stochastic approximation with cone-contractive operators: Sharp\n \ℓ_\∞-bounds for Q-learning
• Ergodic Randomized Algorithms and Dynamics over Networks
• Concentration bounds for temporal difference learning with linear function approximation: The case of batch data and uniform sampling
• Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learning
• Decentralized Control via Dynamic Stochastic Prices: The Independent System Operator Problem
• Online Advertisement, Optimization and Stochastic Networks
• Coordination of autonomic functionalities in communications networks
• Finite Sample Analysis of Two-Timescale Stochastic Approximation with Applications to Reinforcement Learning
• Sparse Q-learning with Mirror Descent
• Average-Reward Learning and Planning with Options
• Model-Free Primal-Dual Methods for Network Optimization with Application to Real-Time Optimal Power Flow
• Algorithms for CVaR Optimization in MDPs
• Teaching Precommitted Agents: Model-Free Policy Evaluation and Control in Quasi-Hyperbolic Discounted MDPs
• Finite-Time Error Bounds For Linear Stochastic Approximation and TD\n Learning
• Explicit Mean-Square Error Bounds for Monte-Carlo and Linear Stochastic Approximation
• Stochastic Online Feedback Optimization for Networks of Non-Compliant Agents
• Pseudo-perturbation-based Broadcast Control of Multi-agent Systems
• Some Limit Properties of Markov Chains Induced by Stochastic Recursive Algorithms
• Stochastic Recursive Inclusions with Non-Additive Iterate-Dependent Markov Noise
• Approachability in Stackelberg Stochastic Games with Vector Costs
• Recurrent Value Functions
• Dynamic Partial Cooperative MIMO System for Delay-Sensitive Applications with Limited Backhaul Capacity
• A Unified Framework for Stochastic Matrix Factorization via Variance Reduction
• Distributed Policy Evaluation Under Multiple Behavior Strategies
• On the diffusion approximation of nonconvex stochastic gradient descent

Related

• New Algorithms And Fast Implementations To Approximate Stochastic Processes
• Stochastic Approximation with Discontinuous Dynamics, Differential Inclusions, and Applications
• A probability approximation framework: Markov process approach
• Analysis of nonsmooth stochastic approximation: the differential inclusion approach
• Functional Central Limit Theorem for Two Timescale Stochastic Approximation