Mathematical Finance, Stochastic Analysis, and Machine Learning Seminar by Harrison Waldon: The Deep Adaptive Regulator for Closed-Loop Predictive Control

Speaker:

Harrison Waldon, post-doctoral researcher, Oxford Man Institute at Oxford University

Title:

The Deep Adaptive Regulator for Closed-Loop Predictive Control

Abstract: 

Traditional methods in optimal control (OC) for deriving optimal policies face challenges in scalability and adaptability due to the curse-of-dimensionality and the reliance on fixed prior models of the environment. Model Predictive Control (MPC) addresses these issues but is limited to open-loop controls, i.e., policies without feedback to adapt, and faces difficulties when implemented in environments demanding high-frequency control. Another approach is Reinforcement Learning (RL) which can scale well to high-dimensional applications but is often computationally expensive and can be unreliable in highly stochastic, continuous-time setups. In this talk, we discuss our recent work which combines deep learning with OC to compute closed-loop adaptive policies by solving continuously updated OC problems that explicitly trade off exploration with exploitation. We show that our method effectively transfers learning to unseen environments and is suited for online decision-making in environments that change in real time. We test our method in various setups and demonstrate its superior performance over traditional methods, especially in scenarios with misspecified priors and nonstationary dynamics.

Mathematical Finance, Stochastic Analysis, and Machine Learning Seminar