Interpretable Control and Robust Stability Analysis for Discrete-Time Nonlinear Systems via Fuzzy Actor-Critic Learning

Abstract

The deployment of advanced reinforcement learning algorithms in physical control systems has been severely hindered by their inherent black-box nature and the lack of theoretical stability guarantees. To address these critical shortcomings, this paper proposes a novel framework integrating fuzzy logic systems with actor-critic reinforcement learning for discrete-time nonlinear systems. By leveraging the universal approximation capabilities and linguistic transparency of fuzzy logic, the proposed approach replaces conventional deep neural networks in the actor and critic structures, thereby providing a clear, rule-based interpretation of the learned control policies and value functions. Furthermore, we provide a rigorous robust stability analysis using Lyapunov theory to ensure the closed-loop system remains uniformly ultimately bounded despite the presence of external disturbances and approximation errors. The optimal control problem is formulated utilizing the discrete-time Hamilton-Jacobi-Bellman equation, and an adaptive weight update mechanism is derived to iteratively optimize the fuzzy rule parameters. Comprehensive empirical evaluations are conducted on standard highly nonlinear benchmark environments. The experimental results demonstrate that the proposed fuzzy actor-critic framework not only achieves competitive convergence speeds and steady-state performance compared to state-of-the-art deep reinforcement learning methods but also significantly enhances the interpretability and safety of the control synthesis.