(345r) An Efficient Reinforcement Algorithm Approach to Stochastic Optimal Control with Application to Biodiesel Production.

Stochastic optimal control problems are one of the most difficult problems in optimization. There are very few approaches to solving stochastic optimal control problems in the literature. This paper presents a new and efficient Reinforcement Learning approach to solving stochastic optimal control problems based on the Batch Q-learning algorithm. To improve the convergence of the RL algorithm, we use k-dimensional uniformity of advanced sampling procedures, namely employing the Halton and Hamersley sequences (HSS). These sequences are used to randomly sample the discrete controls from the action space for the RL optimal control problem. The Neural-fitted Q-iterative algorithm is applied to solve an optimal control problem for a first-order state dynamical system. We found that the HSS-based RL improves the convergence significantly as compared to the current RL practice. We extend this scheme for stochastic optimal control. We illustrate the approach using a case study of biodiesel production where kinetics uncertainties result in stochastic optimal control. We present the comparison of our HSS-RL algorithm with that of the stochastic maximum principle.