Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
TensorFlow Reinforcement Learning Quick Start Guide

You're reading from   TensorFlow Reinforcement Learning Quick Start Guide Get up and running with training and deploying intelligent, self-learning agents using Python

Arrow left icon
Product type Paperback
Published in Mar 2019
Publisher Packt
ISBN-13 9781789533583
Length 184 pages
Edition 1st Edition
Languages
Arrow right icon
Author (1):
Arrow left icon
Kaushik Balakrishnan Kaushik Balakrishnan
Author Profile Icon Kaushik Balakrishnan
Kaushik Balakrishnan
Arrow right icon
View More author details
Toc

Table of Contents (11) Chapters Close

Preface 1. Up and Running with Reinforcement Learning FREE CHAPTER 2. Temporal Difference, SARSA, and Q-Learning 3. Deep Q-Network 4. Double DQN, Dueling Architectures, and Rainbow 5. Deep Deterministic Policy Gradient 6. Asynchronous Methods - A3C and A2C 7. Trust Region Policy Optimization and Proximal Policy Optimization 8. Deep RL Applied to Autonomous Driving 9. Assessment 10. Other Books You May Enjoy

The A3C algorithm

As we mentioned earlier, we have parallel workers in A3C, and each worker will compute the policy gradients and pass them on to the central (or master) processor. The A3C paper also uses the advantage function to reduce variance in the policy gradients. The loss functions consist of three losses, which are weighted and added; they include the value loss, the policy loss, and an entropy regularization term. The value loss, Lv, is an L2 loss of the state value and the target value, with the latter computed as a discounted sum of the rewards. The policy loss, Lp, is the product of the logarithm of the policy distribution and the advantage function, A. The entropy regularization, Le, is the Shannon entropy, which is computed as the product of the policy distribution and its logarithm, with a minus sign included. The entropy regularization term is like a bonus for...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image