UC Berkeley

In control design, the goal is to synthesize policies which map observations to control

actions. Two key elements characterize today's modern design problems: an abundance

of historical data and tasks which are in full or in part repetitive. The requirements are

state and input constraint satisfaction, and performance is assessed by evaluating the cost

associated with the closed-loop trajectories.

In iterative control design, the policy is updated using historical data from past executions

of the control task. The policy update strategy should guarantee

(a) recursive constrain satisfaction,

(b) iterative performance improvement with respect to previous executions, and

(c) locally optimal behavior at convergence.

At present few methodologies are available to iteratively design predictive control policies,

which satisfy the above requirements. The most common approaches resorts to update

the policy after performing system identification. Guarantees are provided for constraint

satisfaction, but not for iterative performance improvements and/or optimality.

This thesis introduces an iterative control design methodology for constrained linear

and nonlinear systems performing iterative tasks. It leads to algorithms which iteratively

synthesize predictive control policies for classes of systems, where there are few, or no tools,

currently available.

We will focus on three classes of discrete time dynamical systems: (i) constrained linear

systems, (ii) constrained nonlinear systems, and (iii) constrained uncertain linear systems.

For these three classes of systems we study iterative optimal control problems and we exploit

knowledge of the system dynamics and historical closed-loop data in the synthesis process.

After each iteration of the control task, we construct a policy which uses forecast to compute

safe control actions, and it is guaranteed to improve the closed-loop performance associated

with stored historical data.

We call this approach the Learning Model Predictive Control (LMPC) framework. For

the above systems, we introduce the policy design which exploits historical data to compute

(i) a control invariant set that represents a safe set of states from which the control task

can be completed and (ii) a control Lyapunov function which for each state of the safe set

approximates the closed-loop cost of completing the task. By using the propose syntheses,

we prove that properties (a),(b) and (c) can be guaranteed for the three classes of discrete

time dynamical systems under consideration.

We start by presenting the LMPC design for linear and nonlinear system subject to convex

and nonconvex constraints. Then, we focus on minimum time optimal control problems

for linear and nonlinear systems. Afterwards, we solve the robust case for linear systems

subject to bounded additive uncertainty. Finally, we present a data based policy to reduce

the computational burden of the LMPC at convergence.

In the concluding part of the thesis, we present a system identification strategy tailored

to iterative tasks and we demonstrate the applicability of the proposed approach through

autonomous racing experiments on the Berkeley Autonomous Race Car (BARC) platform.

Experimental results show that the LMPC safely learns to race a vehicle at the limits of

handling.