Adaptive Control and Reinforcement Learning, S11


16-899C ACRL:

Adaptive Control and Reinforcement Learning

Decision Making, Planning and Control

Time and Day: Spring, 2011, 4:30-5:50, Tuesday and Thursday, NSH 3002

Instructors: Drew Bagnell (

TA: Stephane Ross


Machine learning has escaped from the cage of perception. A growing number of state-of-the-art systems from field robotics, acrobatic autonomous helicopters, to the leading computer Go player and walking robots rely upon learning techniques to make decisions. This change represents a truly fundamental departure from traditional classification and regression methods as such learning systems must cope with

a) their own effects on the world,

b) sequential decision making and long control horizons, and

c) the exploration and exploitation trade-off. In the last 5 years, techniques and understanding of these have developed dramatically. One key to the advance of learning methods has been a tight integration with

optimization techniques, and as such our case studies will focus on this.

What? (Things we will cover)

Planning and Optimal Control Techniques

Reinforcement Learning and Adaptive Control

Imitation Learning and Inverse Optimal Control

Motion Planning, Trajectory Optimization, Operation Space Control

Exploration and Experiment Design

Policy Search Methods

Design for Learnability

Planning/Decision making under Uncertainty