You're reading from The Reinforcement Learning Workshop Learn how to apply cutting-edge reinforcement learning algorithms to a wide range of control problems

Product type Paperback

Published in Aug 2020

Publisher Packt

ISBN-13 9781800200456

Length 822 pages

Edition 1st Edition

Languages

Python

Tools

OpenAI Gym

Concepts

Neural Networks

Authors (9):

Dr. Alexandra Galina Petre

Anand N.S.

Quan Nguyen

Anthony So

Mayur Kulkarni

Aritra Sen

Alessandro Palmas

Emanuele Ghelfi

Saikat Basak

+5 more

View More author details

Table of Contents (14) Chapters

Preface

1. Introduction to Reinforcement Learning

2. Markov Decision Processes and Bellman Equations FREE CHAPTER

3. Deep Learning in Practice with TensorFlow 2

4. Getting Started with OpenAI and TensorFlow for Reinforcement Learning

5. Dynamic Programming

6. Monte Carlo Methods

7. Temporal Difference Learning

8. The Multi-Armed Bandit Problem

9. What Is Deep Q-Learning?

10. Playing an Atari Game with Deep Recurrent Q-Networks

11. Policy-Based Methods for Reinforcement Learning

12. Evolutionary Strategies for RL

Appendix

The UCB algorithm

The term upper confidence bound denotes the fact that instead of considering the average of past rewards returned from each arm like Greedy, the algorithm computes an upper bound for its estimates of the expected reward for each arm.

This concept of a confidence bound is quite common in probability and statistics, where the distribution of a quantity that we care about (in this case, the reward from each arm) cannot be represented well using simply the average of past observations. Instead, a confidence bound is a numerical range that aims to estimate and narrow down where most of the values in the distribution in question will lie. For example, this idea is widely used in Bayesian analyses and Bayesian optimization.

In the following section, we will discuss how UCB establishes its use of a confidence bound.

Optimism in the Face of Uncertainty

Consider the middle of the process of a bandit with only two arms. We have already pulled the first arm 100 times...

The rest of the chapter is locked

Authors (9)

Alessandro Palmas

Alessandro Palmas is an aerospace engineer with more than 7 years of proven expertise in software development for advanced scientific applications and complex software systems. As the R&D head in an aerospace & defense Italian SME, he coordinates projects in contexts ranging from space flight dynamics to machine learning-based autonomous systems. His main ML focus is on computer vision, 3D models, volumetric networks, and deep reinforcement learning. He also founded innovative initiatives, his last being Artificial Twin, which provides advanced technologies for machine learning, physical modeling, and computational geometry applications. Two key areas in which current Artificial Twin deep RL work is focused on are video games entertainment, and guidance, navigation & control systems.

See other products by Alessandro Palmas

Aritra Sen

Aritra Sen currently works as a data scientist in Ericsson. His current role includes building and deploying large scale machine learning solutions for the telecom industry. He has around 10 years of experience in data science and business intelligence. He previously worked in Cognizant, KPMG, IBM, and TCS. Aritra also has a keen interest in blogging and he regularly writes about machine learning, deep learning, etc. He also filed a patent related to the telecom industry.

See other products by Aritra Sen

Mayur Kulkarni

Mayur Kulkarni works in the Machine Learning research team at Microsoft and has previously been at IIT Bombay, and IIM Lucknow. He has also been an instructor for the postgraduate programs in Artificial Intelligence and Machine Learning at UpGrad and IIIT Bangalore, covering topics in Deep Reinforcement Learning. He is one of the contributors to DVC, torch, and scikit-learn, which are some of the most popular open-source machine learning libraries in Python.

See other products by Mayur Kulkarni

Dr. Alexandra Galina Petre

Dr. Alexandra Galina Petre is a machine learning and data science expert, currently leading and teaching various engineering modules in Coventry, United Kingdom. Her leadership and management experience is linked to her work in quality management for the Airbus A380 and her IET membership. She received her Ph.D. in user feedback-based reinforcement learning for vehicle comfort control with a focus on revolutionary heating ventilation and air conditioning SARSA-based control systems that can learn from the drivers preferential changes to the UI. Her research is focusing on how thermal comfort depends on the occupants inclination to manual control as outlined in the SAE paper published in 2019, and the development of a novel Java-based user model (UBL) integrated within a car cabin environment. She is working on deep RL implementations in Python and R-based statistical developments within various automation and control projects.

See other products by Dr. Alexandra Galina Petre

Emanuele Ghelfi

Emanuele Ghelfi is a computer science and machine learning engineer. He received an M.Sc. degree in computer science and engineering at Politecnico di Milano in December 2018. In his thesis, he proposed a new RL algorithm for an MDP extension. The paper from the thesis got accepted at ICML 2019. Hes an organizer of the community data science and artificial intelligence in Parma. Emanuele presented tutorials about generative adversarial networks at conferences like PyCon X (Florence) and EuroSciPy (Bilbao). He is also a developer of the machine learning package AshPy, available on GitHub and PyPi.

See other products by Emanuele Ghelfi

Anthony So

Anthony So is a renowned leader in data science. He has extensive experience in solving complex business problems using advanced analytics and AI in different industries including financial services, media, and telecommunications. He is currently the chief data officer of one of the most innovative fintech start-ups. He is also the author of several best-selling books on data science, machine learning, and deep learning. He has won multiple prizes at several hackathon competitions, such as Unearthed, GovHack, and Pepper Money. Anthony holds two master's degrees, one in computer science and the other in data science and innovation.

See other products by Anthony So

Anand N.S.

Anand N.S. has more than two decades of technology experience working, with a strong hands-on track record of application of artificial intelligence, machine learning, and data science to create measurable business outcomes. He has been granted several US patents in the areas of data science, machine learning, and artificial Intelligence. Anand has a B.Tech in Electrical Engineering from IIT Madras and an MBA with a Gold Medal from IIM Kozhikode.

See other products by Anand N.S.

Quan Nguyen

Quan Nguyen, the author of the first edition of this book, is a Python programmer with a strong passion for machine learning. He holds a dual degree in mathematics and computer science, with a minor in philosophy, earned from DePauw University. Quan is deeply involved in the Python community and has authored multiple Python books, contributing to the Python Software Foundation and regularly sharing insights on DataScience portal. He is currently pursuing a Ph.D. in computer science at Washington University in St. Louis.

See other products by Quan Nguyen

Saikat Basak

Saikat Basak is a data scientist and a passionate programmer. Having worked with multiple industry leaders, he has a good understanding of problem areas that can potentially be solved using data. Apart from being a data guy, he is also a science geek and loves to explore new ideas in the frontiers of science and technology.

See other products by Saikat Basak