Control Systems

Machine Learning and Control Systems

        

Machine learning being a subfield of artificial intelligence which is an undeniably a vast and the most influential and powerful technology in the world today. And this technology is being implemented in every other field of technology if possible with the use of unlimitedly infinite amount of data available in the world. While control system theory manages, controls and regulates the behavior of the systems of the devices using control loops. In short we can say that control theory is optimization that is constrained by its system dynamics. So by merging the use of these two, the classic control theory and machine learning to optimize the linear as well as complex hard to control nonlinear system and with the use of data we can build some interesting modern real world system. This blog is to get thinking about how these powerful techniques in machine learning can be used to tackle real world control system problems based on the amount of data.

Introduction

       Last few decades ago essentially we used to write down the equations of physics for any system. If the equation was observed to be nonlinear ,we would linearize it as they were difficult to control and then apply a control law on the system. For example, say a pendulum on a moving cart, this cart is moving left and right and we want to stabilize the pendulum. In past we would write down the equations which would be nonlinear and we would linearize a lot of systems about some equilibrium and get the required linear equations and use it with linear state space theory that can be a filter with a linear quadratic regulator like robust controller.

In this feedback control diagram, we have some kind of systems, some outputs and a controller. The controller will have some measurements which its going obtain from the sensors and going to control the system like moving the cart left and right from the above example. So the pendulum is a traditional problem and it uses some common techniques to solve the problem. These techniques are what vast majority of modern control systems are based on. But in real world, the dynamics that we are actually interested are systems having the equation in form of

where,

f(x,u)=nonlinear function of x and actuation is u

  

In non-linear control systems things get really difficult. The things that we actually want to solve today are systems in neuroscience where we are trying to understand model and control what the brain is doing so that we can prevent seizures;  a turbulent fluid system like if we have a wing that is under attack so the flow is stalled behind we might want to control that to get higher lift and drag; In disease modelling we want to suppress the spread of pandemic disease like covid -19 on a continent or self-driving cars use specific algorithms for navigation and control, etc. Hence we started making use of machine learning control.

Feedback Control Diagram

Dynamic system: The system maybe linear, nonlinear, high dimensional,etc

Controller: Component of a machine responsible for controlling a system

Sensors: Provide measurements to the system

Cost function: High level function that you need to optimise the system

Disturbances: Some unmeasurable parameter that may change the dynamics.

Fundamental limitations:

Challenges we face while building machine learning control systems are

a)    Nonlinear Equations

b)    Unknown dynamics

c)     High Dimensions i.e. high degree of freedom

d)    Limited measurements

System identification:

Generally, we don’t have equations of motion given to us so that we can design a control law. So we are going to find how to discover the system model to design a control law. If we’re assuming that we have full state measurements i.e. y =x is the entire state of the system that maybe high dimensional or low dimensional system but we can measure the full system. In this kind of age of big data there is likely an increasing chance that you might actually have the full state measurements. Control used here is model predictive control(MPC) which is a very powerful control framework where we run a bunch of models for different inputs u for control signal that will maximize the objective. Now let’s see how we can build system identification model for different input output dynamics that we can then use with MPC. The key tool that we use here is data driven regression.

a)    Linear dynamics:

For linear dynamics we use linear regression.

In system identification with full state measurement we are not just going to use regression to build model of dynamic system but also to figure out how inputs effect system. Therefore,

This is dynamic mode decomposition method with control(DMBc).

b)    Non Linear dynamics:

We can use sparse identification for nonlinear dynamic i.e. the SINDY algorithm.

This algorithm builds library of all possible right hand side functions that could decide

             the dynamics and uses sparse regression to select the fewest terms that agrees with,

The sparsity promotes constraint regularities in the problem and prevents overfitting.

c)    Linear Representation of nonlinear system:

In this we can take the measurement of x from y=x and enrich it into even larger value and by doing so the system becomes linear .here we use the koopman operator theory. The equation to represent a nonlinear equation in linear form is

where,
 k=Koopman operator

    

    This are the identification techniques for systems having full state measurements. For limited measurements system identification techniques are balanced truncation, balanced POD, Eigen system realization, etc.

Machine learning for control:

Identifying dynamics or system is hard to characterize and it turns out that the model you need to predict and understand physical system is a lot more sophisticated than the model you need to control the system. And we can use robust control to handle the missing dynamics and the uncertainty. The whole point is to measure the system and drive it to change the dynamics. For example, if we want to land a SpaceX rocket on a moving platform on sea so here we are going to try to change the natural dynamics which unstable ad try to stabilize it. While doing this we’ll observe a new phenomenon that we’ve never seen before and this may cause the system to breakdown. So here we are going to focus on control laws. Evolutionary genetic algorithms and genetic programming are some of the methods for machine learning in control.

Genetic Algorithm:

Genetic algorithm is based on a biological process call the natural selection i.e. it will make the system adapt with the change in the environment which will prevent breakdown of the system. These algorithms are used for random search that can be used to solve nonlinear system equations and optimize the complex problems. It uses probabilistic rules and handles a population of potential solutions known as individuals or chromosomes. There are three main operations in genetic algorithms that is reproduction, crossover and mutation. This algorithm can be used in PID controller, turbulence, robotics, aeronautics, etc.

Applying genetic algorithm to tune PID controller:

Firstly, the genetic algorithm will generate a random population, which is implemented with small population size in order to allow the controller to be optimized and converge at faster rate.The initial population is set by encoding the PID parameters Kc, Ki and Kd into binary strings known as a chromosome. The fitness of each chromosome can be calculated by converting its binary string into real value which represents the PID parameters. Each of these PID parameters is passed to PID controller.  Then we are going to compute and evaluate the fitness function and proceed with crossover and mutation to make new cluster. This process continues until the end of the generations where the best fitness value is achieved. The ultimate aim of GA is to seek global PID values with minimum fitness value. So we can say that we can apply genetic algorithm for high dimensional search spaces of parameter once we have a fixed structure.

Genetic programming control:

Genetic programming uses considerably more machinery. It simultaneously learns the structure and parameters of a control law. The basic idea here is if we don’t know the control law then we have to learn the structure of effective control laws for that we need bigger machinery.

Here we have a dynamical system, this maybe an experiment or simulation or something to test control laws. It can be written down as the dynamical system with an actual function but generally this is a black box i.e. we may not have a model, but we can control it and measure the cost function. We can write down many control laws and try them until we have the expected cost function ‘j’. Then we apply some genetic operations. The first operation is crossover. Take two good performing individuals and pick some position and swap them.it has the exploitative property i.e if both of these are good then it permutes and exploit good favorable properties of both of these. Second operation is mutation. Mutation takes a random segment of the tee and snips it and replaces it with randomly generated new sub tree. It has the ability to explore the parameter space and tries new things that was never tried before. Third operation is replication that if the control law is good enough it will copy it so that the information wont be lost. This is how genetic programming works in general. And then like genetic algorithm we have to pick the probabilities of crossover, mutation and replication.
People have also been using genetic programming not only to design control law but also for system identification.This is also used by software developers to develop new softwares and applications.

Conclusion

The fundamental challenges mentioned above motivates the entire modern machine learning control. The systems in neuroscience, turbulence, etc. are still under research and developments are being made. This is all possible due to explosion of data and massive data collection capabilities.  Hence machine learning control system is a new and emerging field in applied mathematics and engineering which makes the control system theory more accessible to the real world systems of interest.

References

• Book:Data-Driven science and engineering:Machine Learning,Dynamical systems and Control-By Steven L.Brunton, J.Nathon Kutz https://www.amazon.com/Data-Driven-Science-Engineering-Learning-Dynamical-ebook/dp/B07N4BK4CZ
• Machine learning Control:Taming non linear dynamics and Turbulence-By Thomas Duriez, Steven L.Brunton, Bernd R.Noak  https://www.amazon.com/Machine-Learning-Control-Turbulence-Applications-dp-3319821407/dp/3319821407/ref=mt_paperback?_encoding=UTF8&me=&qid=1586371625
• https://www.hindawi.com/journals/aai/2014/791230/
• https://link.springer.com/chapter/10.1007/978-3-662-03423-1_19

Authors: Anuradha Lohar

                 Swaraj Mane

                 Shaurya Mhaske 

                 Mitul