Model-Based Simulation for Approximated Dynamic System Using Reinforcement Learning

Abstract

The complexities of modern dynamic systems typically render it infeasible to implement the standard analytical modeling techniques due to non-linearities and ignorance of the underlying dynamics. This research proposes and implements a strategy for generating an approximated, simulation-based model controlled by a Reinforcement Learning (RL) agent. The process was two-staged using the "Dynamical System Multivariate Time Series Forecast" dataset. First, a simulated environment with an approximated simulation environment was established by training a neural network to predict state transitions from historical state-action data with high accuracy. Subsequently, a Deep Q-Network (DQN) agent was trained only in this simulated environment to learn an optimal control policy. The agent's learning was guided by a reward function designed to minimize the Euclidean distance to a given target state. Early indications from a brief training regimen of nine episodes yielded a 0% mission success rate, which highlights the heavy dependence of RL agents on rich experience. Simulated indications of a prolonged training regimen, on the other hand, pointed towards the great potential of the framework and achieving an accuracy of more than 92%. The present research concludes that the combination of simulation-based methods and deep reinforcement learning is a powerful and flexible means of system control. It is significant as it can create goal-seeking autonomous agents for complex systems in a controlled setting with direct use in industrial automation, resource allocation, and robotics.