Author: Kyriakos G. Vamvoudakis Publisher: ISBN: Category : Adaptive control systems Languages : en Pages :
Book Description
Optimal control deals with the problem of finding a control law for a given system that a certain optimality criterion is achieved. It can be derived using Pontryagin's maximum principle (a necessary condition), or by solving the Hamilton-Jacobi-Bellman equation (a sufficient condition). Major drawback of optimal control is that it is offline. Adaptive control involves modifying the control law used by a controller to cope with the facts that the system is unknown or uncertain. Adaptive controllers are not optimal. Adaptive optimal controllers have been proposed by adding optimality criteria to an adaptive controller, or adding adaptive characteristics to an optimal controller. In this work, online adaptive learning algorithms are developed for optimal control and differential dynamic games by using measurements along the trajectory or input/output data. These algorithms are based on actor/critic schemes and involve simultaneous tuning of the actor/critic neural networks and provide online solutions to complex Hamilton-Jacobi equations, along with convergence and Lyapunov stability proofs. The research begins with the development of an online algorithm based on policy iteration for learning the continuous-time (CT) optimal control solution with infinite horizon cost for nonlinear systems with known dynamics. That is, the algorithm learns online in real-time the solution to the optimal control design Hamilton-Jacobi (HJ) equation. This is called 'synchronous' policy iteration. Then it became interesting to develop an online learning algorithm to solve the continuous-time two-player zero-sum game with infinite horizon cost for nonlinear systems. The algorithm learns online in real-time the solution to the game design Hamilton-Jacobi-Isaacs equation. This algorithm is called online gaming algorithm 'synchronous' zero-sum game policy iteration. One of the major outcomes of this work is the online learning algorithm to solve the continuous time multi player non-zero sum games with infinite horizon for linear and nonlinear systems. The adaptive algorithm learns online the solution of coupled Riccati and coupled Hamilton-Jacobi equations for linear and nonlinear systems respectively. The optimal-adaptive algorithm is implemented as a separate actor/critic parametric network approximator structure for every player, and involves simultaneous continuous-time adaptation of the actor/critic networks. The next result shows how to implement Approximate Dynamic Programming methods using only measured input/output data from the systems. Policy and value iteration algorithms have been developed that converge to an optimal controller that requires only output feedback. The notion of graphical games is developed for dynamical systems, where the dynamics and performance indices for each node depend only on local neighbor information. A cooperative policy iteration algorithm, is given for graphical games, that converges to the best response when the neighbors of each agent do not update their policies and to the cooperative Nash equilibrium when all agents update their policies simultaneously. Finally, a synchronous policy iteration algorithm based on integral reinforcement learning is given. This algorithm does not need the drift dynamics.
Author: Draguna L. Vrabie Publisher: IET ISBN: 1849194890 Category : Computers Languages : en Pages : 305
Book Description
The book reviews developments in the following fields: optimal adaptive control; online differential games; reinforcement learning principles; and dynamic feedback control systems.
Author: Frank L. Lewis Publisher: John Wiley & Sons ISBN: 1118453972 Category : Technology & Engineering Languages : en Pages : 498
Book Description
Reinforcement learning (RL) and adaptive dynamic programming (ADP) has been one of the most critical research fields in science and engineering for modern complex systems. This book describes the latest RL and ADP techniques for decision and control in human engineered systems, covering both single player decision and control and multi-player games. Edited by the pioneers of RL and ADP research, the book brings together ideas and methods from many fields and provides an important and timely guidance on controlling a wide variety of systems, such as robots, industrial processes, and economic decision-making.
Author: Leon A. Petrosjan Publisher: World Scientific ISBN: 9789810209797 Category : Mathematics Languages : en Pages : 342
Book Description
The classical optimal control theory deals with the determination of an optimal control that optimizes the criterion subjects to the dynamic constraint expressing the evolution of the system state under the influence of control variables. If this is extended to the case of multiple controllers (also called players) with different and sometimes conflicting optimization criteria (payoff function) it is possible to begin to explore differential games. Zero-sum differential games, also called differential games of pursuit, constitute the most developed part of differential games and are rigorously investigated. In this book, the full theory of differential games of pursuit with complete and partial information is developed. Numerous concrete pursuit-evasion games are solved (?life-line? games, simple pursuit games, etc.), and new time-consistent optimality principles in the n-person differential game theory are introduced and investigated.
Author: Peiliang An Publisher: ISBN: Category : Languages : en Pages : 27
Book Description
This work studies a multi-player H∞ differential game for systems of general linear dynamics. In this game, multiple players design their control inputs to minimize their cost functions in the presence of worst-case disturbances. We first derive the optimal control and disturbance policies using the solutions to Hamilton-Jacobi-Isaacs (HJI) equations. We then prove that the derived optimal policies stabilize the system and constitute a Nash equilibrium solution. Two integral reinforcement learning (IRL) -based algorithms, including the policy iteration IRL and off-policy IRL, are developed to solve the differential game online. We show that the off-policy IRL can solve the multi-player H∞ differential game online without using any system dynamics information. Simulation studies are conducted to validate the theoretical analysis and demonstrate the effectiveness of the developed learning algorithms.
Author: Derong Liu Publisher: Springer ISBN: 3319508156 Category : Technology & Engineering Languages : en Pages : 609
Book Description
This book covers the most recent developments in adaptive dynamic programming (ADP). The text begins with a thorough background review of ADP making sure that readers are sufficiently familiar with the fundamentals. In the core of the book, the authors address first discrete- and then continuous-time systems. Coverage of discrete-time systems starts with a more general form of value iteration to demonstrate its convergence, optimality, and stability with complete and thorough theoretical analysis. A more realistic form of value iteration is studied where value function approximations are assumed to have finite errors. Adaptive Dynamic Programming also details another avenue of the ADP approach: policy iteration. Both basic and generalized forms of policy-iteration-based ADP are studied with complete and thorough theoretical analysis in terms of convergence, optimality, stability, and error bounds. Among continuous-time systems, the control of affine and nonaffine nonlinear systems is studied using the ADP approach which is then extended to other branches of control theory including decentralized control, robust and guaranteed cost control, and game theory. In the last part of the book the real-world significance of ADP theory is presented, focusing on three application examples developed from the authors’ work: • renewable energy scheduling for smart power grids;• coal gasification processes; and• water–gas shift reactions. Researchers studying intelligent control methods and practitioners looking to apply them in the chemical-process and power-supply industries will find much to interest them in this thorough treatment of an advanced approach to control.
Author: Tamer Başar Publisher: ISBN: Category : Control theory Languages : en Pages : 220
Book Description
This volume contains fifteen articles on the topic of differential and dynamic games, focusing on both theory and applications. It covers a variety of areas and presents recent developments on topics of current interest. It should be useful to researchers in differential and dynamic games, systems and control, operations research and mathematical economics.
Author: Myer Kutz Publisher: John Wiley & Sons ISBN: 1118930800 Category : Technology & Engineering Languages : en Pages : 1008
Book Description
Full coverage of electronics, MEMS, and instrumentation and control in mechanical engineering This second volume of Mechanical Engineers' Handbook covers electronics, MEMS, and instrumentation and control, giving you accessible and in-depth access to the topics you'll encounter in the discipline: computer-aided design, product design for manufacturing and assembly, design optimization, total quality management in mechanical system design, reliability in the mechanical design process for sustainability, life-cycle design, design for remanufacturing processes, signal processing, data acquisition and display systems, and much more. The book provides a quick guide to specialized areas you may encounter in your work, giving you access to the basics of each and pointing you toward trusted resources for further reading, if needed. The accessible information inside offers discussions, examples, and analyses of the topics covered, rather than the straight data, formulas, and calculations you'll find in other handbooks. Presents the most comprehensive coverage of the entire discipline of Mechanical Engineering anywhere in four interrelated books Offers the option of being purchased as a four-book set or as single books Comes in a subscription format through the Wiley Online Library and in electronic and custom formats Engineers at all levels will find Mechanical Engineers' Handbook, Volume 2 an excellent resource they can turn to for the basics of electronics, MEMS, and instrumentation and control.
Author: Magdi S. Mahmoud Publisher: Elsevier ISBN: 0081012624 Category : Technology & Engineering Languages : en Pages : 400
Book Description
Microgrids: Advanced Control Methods and Renewable Energy System Integration demonstrates the state-of-art of methods and applications of microgrid control, with eleven concise and comprehensive chapters. The first three chapters provide an overview of the control methods of microgrid systems that is followed by a review of distributed control and management strategies for the next generation microgrids. Next, the book identifies future research directions and discusses the hierarchical power sharing control in DC Microgrids. Chapter 4 investigates the demand side management in microgrid control systems from various perspectives, followed by an outline of the operation and controls of the smart microgrids in Chapter 5. Chapter 6 deals with control of low-voltage microgrids with master/slave architecture. The final chapters explain the load-Frequency Controllers for Distributed Power System Generation Units and the issue of robust control design for VSIs, followed by a communication solution denoted as power talk. Finally, in Chapter 11, real-time implementation of distributed control for an autonomous microgrid system is performed. Addresses issues of contemporary interest to practitioners in the power engineering and management fields Focuses on the role of microgrids within the overall power system structure and attempts to clarify the main findings relating to primary and secondary control and management at the microgrid level Provides results from a quantified assessment of benefits from economic, environmental, operational, and social point-of-views Presents the hierarchical control levels manifested in microgrid operations and evaluates the principles and main functions of centralized and decentralized control
Author: Jacob Engwerda Publisher: John Wiley & Sons ISBN: 9780470015247 Category : Business & Economics Languages : en Pages : 514
Book Description
Game theory is the theory of social situations, and the majority of research into the topic focuses on how groups of people interact by developing formulas and algorithms to identify optimal strategies and to predict the outcome of interactions. Only fifty years old, it has already revolutionized economics and finance, and is spreading rapidly to a wide variety of fields. LQ Dynamic Optimization and Differential Games is an assessment of the state of the art in its field and the first modern book on linear-quadratic game theory, one of the most commonly used tools for modelling and analysing strategic decision making problems in economics and management. Linear quadratic dynamic models have a long tradition in economics, operations research and control engineering; and the author begins by describing the one-decision maker LQ dynamic optimization problem before introducing LQ differential games. Covers cooperative and non-cooperative scenarios, and treats the standard information structures (open-loop and feedback). Includes real-life economic examples to illustrate theoretical concepts and results. Presents problem formulations and sound mathematical problem analysis. Includes exercises and solutions, enabling use for self-study or as a course text. Supported by a website featuring solutions to exercises, further examples and computer code for numerical examples. LQ Dynamic Optimization and Differential Games offers a comprehensive introduction to the theory and practice of this extensively used class of economic models, and will appeal to applied mathematicians and econometricians as well as researchers and senior undergraduate/graduate students in economics, mathematics, engineering and management science.
Author: Kyriakos G. Vamvoudakis
Author: Draguna L. Vrabie
Author: Frank L. Lewis
Author: Leon A. Petrosjan
Author: Peiliang An
Author: Derong Liu
Author: Tamer Başar
Author: Myer Kutz
Author: Magdi S. Mahmoud
Author: Jacob Engwerda