# Neurodynamic Programming and Zero-Sum Games for Constrained Control Systems

@article{AbuKhalaf2008NeurodynamicPA, title={Neurodynamic Programming and Zero-Sum Games for Constrained Control Systems}, author={Murad Abu-Khalaf and Frank L. Lewis and Jie Huang}, journal={IEEE Transactions on Neural Networks}, year={2008}, volume={19}, pages={1243-1252} }

In this paper, neural networks are used along with two-player policy iterations to solve for the feedback strategies of a continuous-time zero-sum game that appears in L2-gain optimal control, suboptimal Hinfin control, of nonlinear systems affine in input with the control policy having saturation constraints. The result is a closed-form representation, on a prescribed compact set chosen a priori, of the feedback strategies and the value function that solves the associated Hamilton-Jacobi… Expand

#### Figures and Topics from this paper

#### 153 Citations

Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP

- Mathematics, Computer Science
- IEEE Transactions on Cybernetics
- 2013

The novel weight tuning laws for critic neural networks are proposed, which not only ensure the Nash equilibrium to be reached but also guarantee the system to be stable and demonstrate the uniform ultimate boundedness of the closed-loop system. Expand

Constrained online optimal control for continuous-time nonlinear systems using neuro-dynamic programming

- Proceedings of the 33rd Chinese Control Conference
- 2014

An online adaptive optimal control scheme to solve the infinite-horizon optimal control problem of continuous-time nonlinear systems with control constraints by using only a critic neural network to derive the optimal control instead of typical action-critic dual networks employed in neuro-dynamic programming methods. Expand

An Approximate Control Algorithm for Zero-Sum Differential Games Using Adaptive Critic Technique

- Computer Science
- 2018 37th Chinese Control Conference (CCC)
- 2018

A novel adaptive dynamic programming (ADP) algorithm is proposed to approximate the optimal value function through a critic neural network (NN) to obtain the approximate optimal control policy and the worst-base disturbance policy. Expand

Zero-sum differential games for nonlinear systems using adaptive dynamic programming with input constraint

- Mathematics
- 2017 36th Chinese Control Conference (CCC)
- 2017

In this paper, the zero-sum differential game problem for a class of nonlinear system with input constraints is investigated via adaptive dynamic programming (ADP). A suitable non-quadratic… Expand

Adaptive dynamic programming for H∞ control of constrained-input nonlinear systems

- Mathematics
- 2015 34th Chinese Control Conference (CCC)
- 2015

This paper presents a novel adaptive/approximate dynamic programming algorithm to solve the H∞ control problem of constrained-input continuous-time nonlinear systems. The developed algorithm employs… Expand

Robust control design for multi-player nonlinear systems with input disturbances via adaptive dynamic programming

- Computer Science
- Neurocomputing
- 2019

It is proved that the robust control strategy can guarantee the multi-player nonlinear systems to be stable in the sense of uniform ultimate boundedness (UUB) with disturbance rejection. Expand

Integral Reinforcement Learning for Linear Continuous-Time Zero-Sum Games With Completely Unknown Dynamics

- Computer Science, Mathematics
- IEEE Transactions on Automation Science and Engineering
- 2014

An integral reinforcement learning algorithm based on policy iteration to learn online the Nash equilibrium solution for a two-player zero-sum differential game with completely unknown linear continuous-time dynamics is developed. Expand

Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm

- Computer Science, Mathematics
- Neurocomputing
- 2013

A greedy heuristic dynamic programming iteration algorithm is developed to solve the zero-sum game problems, which can be used to solves the Hamilton-Jacobi-Isaacs equation associated with H"~ optimal regulation control problems. Expand

Adaptive near-optimal neuro controller for continuous-time nonaffine nonlinear systems with constrained input

- Computer Science, Medicine
- Neural Networks
- 2017

By employing two Neural Networks (NNs), the solution of Hamilton-Jacobi-Bellman (HJB) equation associated with the cost function is derived without requiring a priori knowledge about system dynamics. Expand

Model-Free Adaptive Control for Unknown Nonlinear Zero-Sum Differential Game

- Computer Science, Medicine
- IEEE Transactions on Cybernetics
- 2018

The online learning algorithm is proposed based on the GDHP method to solve the Hamilton–Jacobi–Isaacs equation associated with optimal regulation control problem and three neural networks are established to approximate the optimal saddle point feedback control law, the disturbance law, and the performance index. Expand

#### References

SHOWING 1-10 OF 51 REFERENCES

Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach

- Mathematics, Computer Science
- Autom.
- 2005

It is shown that the constrained optimal control law has the largest region of asymptotic stability (RAS) and the result is a nearly optimal constrained state feedback controller that has been tuned a priori off-line. Expand

Policy Iterations on the Hamilton–Jacobi–Isaacs Equation for $H_{\infty}$ State Feedback Control With Input Saturation

- Mathematics, Computer Science
- IEEE Transactions on Automatic Control
- 2006

An Hinfin suboptimal state feedback controller for constrained input systems is derived using the Hamilton-Jacobi-Isaacs (HJI) equation of a corresponding zero-sum game that uses a special quasi-norm… Expand

Neural approximations for infinite-horizon optimal control of nonlinear stochastic systems

- Computer Science, Mathematics
- IEEE Trans. Neural Networks
- 1998

Simulation results show that the proposed feedback control law may constitute an effective tool for solving a wide class of control problems traditionally regarded as difficult ones, to a sufficient degree of accuracy. Expand

H∞-0ptimal Control and Related Minimax Design Problems: A Dynamic Game Approach

- Computer Science
- IEEE Trans. Autom. Control.
- 1996

The authors believe that the H-infinity-optimal control theory is now at a stage where it can easily be incorporated into a second-level graduate course in a control curriculum, that would follow a basic course in linear control theory covering LQ and LQG designs. Expand

A preliminary control design for the nonlinear benchmark problem

- Computer Science
- Proceeding of the 1996 IEEE International Conference on Control Applications IEEE International Conference on Control Applications held together with IEEE International Symposium on Intelligent Contro
- 1996

Analysis of the design shows that in the absence of disturbances and uncertainties, the SDRE nonlinear feedback solution compares very favorably to the optimal open-loop solution obtained via numerical optimization. Expand

Role of performance functionals in control laws design

- Mathematics
- Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148)
- 2001

The research in optimal control has been concentrated on design of control laws in order to optimize the dynamic performance of systems. These optimization problems were explicitly formulated and… Expand

Adaptively controlling nonlinear continuous-time systems using multilayer neural networks

- Computer Science
- IEEE Trans. Autom. Control.
- 1994

Multilayer neural networks are used in a nonlinear adaptive control problem to control the plant to track a reference command and a local convergence result is provided, which says that if the initial parameter errors are small enough, then the tracking error will converge to a bounded area. Expand

Stable adaptive neural control scheme for nonlinear systems

- Computer Science
- IEEE Trans. Autom. Control.
- 1996

A design methodology is developed that expands the class of nonlinear systems that adaptive neural control schemes can be applied to and relaxes some of the restrictive assumptions that are usually made. Expand

Stable Adaptive Neural Network Control

- Engineering
- The Springer International Series on Asian Studies in Computer and Information Science
- 2002

While neural network control has been successfully applied in various practical applications, many important issues, such as stability, robustness, and performance, have not been extensively… Expand

Output feedback control of nonlinear systems using RBF neural networks

- Mathematics, Computer Science
- IEEE Trans. Neural Networks Learn. Syst.
- 2000

An adaptive output feedback control scheme for the output tracking of a class of continuous-time nonlinear plants is presented and it is shown that by using adaptive control in conjunction with robust control, it is possible to tolerate larger approximation errors resulting from the use of lower order networks. Expand