Model Predictive Controller

Hongbo Zhang

14 Model Predictive Controller

Hongbo Zhang

1) Overview of Model Predictive Control

A Model Predicative Controller (MPC) is an optimal controller. This controller is used in practical control applications to not only control a system but also thoroughly consider the constraints of the system. MPC has been extensively used for process industries, power systems, automotive, and aerospace engineering. Interestingly, MPC also accommodates filtering the noise in a system to ensure noises would not adversely impact the system control performance.

$\newline$

A discrete state-space model for MPC is typically represented as:

$x(k+1) &= Ax(k) + Bu(k) + B_d d(k) + w(k)$

$y(k) &= Cx(k) + v(k)$

where:

x(k) is the state vector at time step k .
u(k) is the control input vector.
d(k) is the disturbance vector.
y(k) is the output vector.
A, B, Bd, and C are matrices that define the system dynamics.
w(k) and v(k) are process and measurement noise, respectively.

Figure 14.1 shows an example of random noises of a system.

Figure 14.1: A control system with random noises.

Such a discrete state space model is similar to the state space modeled previously. A notable difference, however, does exist. In contrast to the previous method, where the noises of the system are not explicitly considered, with this model, the noises of the system (including w(k), the process noise, and v(k), the measurement noise), are well considered.

$\newline$

Of these two different types of noises considered, the process noises w(k) are the noises embedded in the system. These can be illustrated through an example. Given an autonomous driving system, the system noise refers to the road surface variation, wind gusts, sensor inaccuracies, and actuator imperfections. On the other hand, an example of the measurement noises v(k) include the random errors or disturbances that affect the accuracy of measurements taken by sensors or instruments. More specifically, the measurements noises include thermal measurement noise, quantization noise (for the digital systems when continuous signals are converted to the discrete digital values.), the shot noise (noises associated with electric charge), environmental noise (such as electromagnetic interference), interference from other signals (such as radio frequency interference (RFI) and electromagnetic interference (EMI)), and human error.

2) Using a Kalman Filter to Reduce Noise

In order for MPC to perform optimal control, the noise of the system needs to be removed. MPC achieves noise filtering through the use of a Kalman filter or extended state observer (ESO). In our book, we will focus on the use of a Kalman filter to estimate the states and disturbances. To use a Kalman filter to extract noise from the system, the following the steps are needed.

$\newline$

The MPC is an optimal controller. Therefore, it involves a few steps to solve the optimization problem.

(A) Prediction Step:

Predict the next state:

$\hat{x}(k+1|k) = A \hat{x}(k|k) + B u(k)$

Predict the error covariance:

$P(k+1|k) = A P(k|k) A^T + Q$

Here, P is the error covariance matrix and Q is the process noise covariance.

(B) Update Step:

Compute the Kalman gain:

$K(k) = P(k|k-1) C^T (C P(k|k-1) C^T + R)^{-1}$

Update the state estimate:

$\hat{x}(k|k) = \hat{x}(k|k-1) + K(k) (y(k) - C \hat{x}(k|k-1))$

Update the error covariance matrix:

$P(k|k) = (I - K(k) C) P(k|k-1)$

Here, R is the measurement noise covariance.

$\newline$

The key idea of the Kalman filter is to compute the Kalman gain K and update the state estimate by the introduction of the error of covariance. Following the precise estimate of the state, it becomes possible to reject the noises from the system.

Figure 14.2: The control system with random noises removed. The orange curve shown in the figure represents the filtered noises from the system.

3) Model Predictive Control (MPC) Modeling

In this section, let us explore MPC modeling without noise. Following the use of the Kalman filter to filter out the noise from the system and estimate the true states, the MPC solver starts to solve the quadratic optimization problem in order to perform the control of the system.

$\newline$

At each time step, MPC solves an optimization problem to determine the optimal control input u(k). The optimization problem can be formulated with cost function J.

$J = \sum_{i=0}^{N-1} \left( (x(k+i) - x_{\text{ref}}(k+i))^T Q (x(k+i) - x_{\text{ref}}(k+i)) \right) + \lambda \sum_{i=0}^{N-1} \left( u(k+i)^T R u(k+i) \right)$

subject to the following constraints:

$x(k+i+1|k) = A x(k+i|k) + B u(k+i)$

$y(k+i|k) = C x(k+i|k)$

$x_{\text{min}} \leq x(k+i|k) \leq x_{\text{max}}$

$u_{\text{min}} \leq u(k+i) \leq u_{\text{max}}$

where:

N is the prediction horizon
$x_{\text{ref}}$ is the reference state.
Q and R are the weighting matrices for the state and control input, respectively.
$x_{\text{min}}, x_{\text{max}}, u_{\text{min}}, u_{\text{max}}$ are the state constraints and the control inputs.

(1) First Example: MPC with Kalman Filter Modeling

Given the state space listed as follows, let’s define a simple system with:

$A = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}, \quad B = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad C = \begin{bmatrix} 1 & 0 \end{bmatrix}$

The discrete state space is subject to the following constraints:

Control input constraints: $-2 \leq u(k) \leq 2$
State constraints: $-20 \leq x(k) \leq 20$

With the above parameters, we will solve the problem using MPC. Note that this example does not consider noises.

With the above formulation, we need to use a Kalman filter to estimate the state $\hat{x}(k)$ by observing the values of the measurements $y(k)$ .

The above equation can be easily formulated through the following MPC formulation:

$J = \sum_{i=0}^{N-1} \left( (\hat{x}(k+i) - x_{\text{ref}}(k+i))^T Q (\hat{x}(k+i) - x_{\text{ref}}(k+i)) \right) + \lambda \sum_{i=0}^{N-1} \left( u(k+i)^T R u(k+i) \right)$

The solution steps:

Initialization: start with the initial state $x(0)$ and the initial state estimate $\hat{x}(0)$
Kalman Filter Update: For each step, at each time step k, update the state estimate using the Kalman filter.
Prediction: For each step, predict future states using the state-space model and the updated state estimate.
Optimization: Solve the optimization problem to find the optimal control inputs.
Apply Control: Apply the first control input $u(k)$ to the system.
Update: Update the state $x(k+1)$ using the applied control input and repeat the process.

The solution code is shown below in part (3) Solutions

(2) Second Example: MPC with Kalman Filter Modeling

Given the same system with noises, the state space can presented as follows.

$A = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}, \quad B = \begin{bmatrix} 0 \\ 1 \end{bmatrix}, \quad C = \begin{bmatrix} 1 & 0 \end{bmatrix}$

We’ll define the process noise covariance $Q_kf$ and measurement noise covariance $R_kf$ as follows:

$Q_kf = \begin{bmatrix} 0.01 & 0 \\ 0 & 0.01 \end{bmatrix}, \quad R_kf = \begin{bmatrix} 0.01 \end{bmatrix}$

For the noisy system condition, once the process noises and measurement noises are in existence in the system, the cost function should consider it as well. The noises are the process noise covariance $Q_kf$ and measurement noise covariance $R_kf$ as follows:

$J = \sum_{i=0}^{N-1} \left( (x(k+i) - x_{\text{ref}}(k+i))^T Q (x(k+i) - x_{\text{ref}}(k+i)) + \text{tr}(Q_kf) \right) +... \\ ...+\lambda \sum_{i=0}^{N-1} \left( u(k+i)^T R u(k+i) + \text{tr}(R_kf) \right)$

where:

$x(k+i)$ is the predicted state at time step $k+i$
$x_{\text{ref}}(k+i)$ is the reference state at time step $k+i$
Q is the state weighting matrix
$u(k+i)$ is the control input at time step $k+i$
R is the control input weighting matrix
$\lambda$ is a weighting factor for the control effort
$\text{tr}(Q_{kf})$ is the trace of the process noise covariance matrix
$\text{tr}(R_{kf})$ is the trace of the measurement noise covariance matrix

The solution code is shown below in part (3) Solutions

(3) Solutions

Solution to first example: The code used to implement the MPC system without noises is shown in below. More specifically, the given state space without noises can be presented as follows.

Figure 14.3: The cost function of the noiseless MPC system.

Solution to second example: The noises are added to the system as shown below, both process noise and measurement noise. Note that, the process noise is added to the state equation since the process noise is more related to the physical conditions (states) of the system. However, the measurement noise is added to the output of the state space, since it is not state-related.

Figure 14.4: The cost function of the noisy MPC system.

Now that the system has noises, a Kalman filter is needed to filter the noises from the system. The Kalman filter predicates the states, updates the error covariance matrix, and calculates the Kalman gain.

Figure 14.5: Kalman filter to filter the noises

The new cost function considering the noises is then used to replace the original cost function. Note that, the new cost function includes Q, the process noise covariance, and R, the measurement noise covariance.

Figure 14.6: The cost function including the added process and measurement noises.

Following the process above, we can obtain the following state results. The two results correspond to the two different conditions where either no noises exist (Figure 14.7), or process and measurement noises are in existence (Figure 14.8).

Evidently, the results of the states and the input have both satisfied the constraints. Therefore, we can confidently say that the proposed MPC is effective in solving the state space while considering both process and measurement noises in the system.

Figure 14.7: The states of the noiseless system, X1 and X2.

Figure 14.8: The states of the noisy system, X1 and X2, and noises of the system.

Figure 14.9: The control input u of the system.

4) Class Engagement

In this class engagement, your goal is to balance the inverted pendulum on a moving cart. More details of the problem please refer to D2L

https://elearn.mtsu.edu/d2l/le/content/9996674/Home

You can use MPC controller to control the robot to balance the robot. The key thing to implement MPC controller is to have optimal controller to solve the cost function a shown in below. Specifically, we use python package cvxpy to minimize the cost function.

$J = \sum_{i=0}^{N-1} \left( (x(k+i) - x_{\text{ref}}(k+i))^T Q (x(k+i) - x_{\text{ref}}(k+i)) \right) + \lambda \sum_{i=0}^{N-1} \left( u(k+i)^T R u(k+i) \right)$

Python Code:

cost += cvxpy.quad_form(x[:, t + 1], Q)
cost += cvxpy.quad_form(u[:, t], R)

The constrains needed to be satisfied is

$x(k+i+1|k) = A x(k+i|k) + B u(k+i)$

[x[:, t + 1] == A @ x[:, t] + B @ u[:, t]]

————— The Key Code of MPC in Python ————–

for t in range(T):
cost += cvxpy.quad_form(x[:, t + 1], Q)
cost += cvxpy.quad_form(u[:, t], R)
constr += [x[:, t + 1] == A @ x[:, t] + B @ u[:, t]]

Other Constains

$x_{\text{min}} \leq x(k+i|k) \leq x_{\text{max}}$

$u_{\text{min}} \leq u(k+i) \leq u_{\text{max}}$

where:

Python Code:

constr += [x[:, 0] == x0[:, 0]]

The below is to solve it with cvxpy.Minimize function

prob = cvxpy.Problem(cvxpy.Minimize(cost), constr)

License

Icon for the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Robotics and Controls Engineering Copyright © by Hongbo Zhang; Elissa Ledoux; and Vishwas Bedekar is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted.

1) Overview of Model Predictive Control

2) Using a Kalman Filter to Reduce Noise

(A) Prediction Step:

(B) Update Step:

3) Model Predictive Control (MPC) Modeling

(1) First Example: MPC with Kalman Filter Modeling

(2) Second Example: MPC with Kalman Filter Modeling

(3) Solutions

4) Class Engagement

License

Share This Book