Soft Actor-Critic Algorithms and Applications

This talk covers 2 similar papers on SAC by the same team at UC Berkeley, released 1 year apart. The first paper is a theoretical introduction to SAC, the second tweaks the algorithm slightly and showcases some robot use-cases.

Prerequisites: Chapters 1-5 of Sutton & Barto

References

[Haa18S]

Soft Actor-Critic Algorithms and Applications, Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, Sergey Levine.

Dec 2018

Model-free deep reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. However, these methods typically suffer from two major challenges: high sample complexity and brittleness to hyperparameters. Both of these challenges limit the applicability of such methods to real-world domains. In this paper, we describe …

[Haa18aS]

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, Sergey Levine.

Jan 2018

Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks. However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning. Both of these challenges severely limit the applicability of such methods to …