Completed The Fast Deep RL Course? Then you know how to solve simple problems using OpenAI Gym and Ray-RLlib. You also have a good basis in Deep RL.

The next obvious step is to apply this knowledge to real-world problems. At this juncture, it is common to face the following hurdles.

  • No premade Gym environment exists for the problem you want to attack. You must define and create your own custom environment.
  • After creating a custom environment, you apply an appropriate Deep RL algorithm with default framework settings. But the agent doesn't seem to learn anything or displays poor learning.

In this course, you will gain the skills needed to overcome these hurdles and become effective in real-world applications.

You will learn the main ideas behind designing and implementing custom Gym environments for real-world problems.

You will also learn how to apply several performance enhancing tricks and Ray Tune experiments to easily identify the most promising tricks. Here is a list of tricks we will cover.

  • Redefining the observations and actions to cast the environment as close as possible to a Markov Decision Process
  • Scaling observations, actions, and rewards to standard ranges preferred by Deep Neural Nets
  • Shaping reward functions to help the agent better distinguish between good and bad actions
  • Preprocessing the inputs to reduce complexity while preserving critical information
  • Tuning the network size and hyperparameters of the Ray-RLlib algorithms to improve performance

Remember the shop inventory management problem from the The Fast Deep RL Course? That's an example of a real world problem.

By the end of the course, you will make a custom Gym environment for inventory management, shape rewards, normalize observations and actions, tune hyperparameters, and much more. By running Ray Tune experiments, you will find the best learning settings and create a Deep RL agent that performs better than classical inventory management techniques.

We will also solve another problem where the agent learns directly from images. This will show you how to use preprocessing effectively.

After doing these examples, you will be able to use a step-by-step method to solve real-world Deep RL problems that you encounter in your industry.

If you liked The Fast Deep RL Course, I think you will like this course too. After all, real world application is the next natural and exciting step. This course is open for pre-enrollment and the launch is planned in 6 to 8 months.

By enrolling now,

  • you support the creation of this course.
  • Early supporters will get access to lessons as soon as I make them; no waiting around till launch.
  • The planned launch price is $90. As an early supporter, you will get a 50% discount on that price.

Thank you for supporting this course!

Prerequisites



What will you learn?

Chapter 1


In Chapter 1, you will learn how to create custom Gym environments.

  • You will be able to design observations and actions such that the environment is close to a Markov Decision Process. This gives Deep RL algorithms the best chance for learning.
  • You will be able to implement custom Gym environments by inheriting the base environment class and defining the required methods.

Chapter 2


In Chapter 2, you will learn how to scale observations and actions, and shape rewards using Gym Wrappers.

  • You will be able to modify your custom environments further by using Gym Wrappers.
  • You will write wrappers for scaling observations and actions to ranges preferred by Deep Neural Nets.
  • You will write several wrappers to shape the reward function.


Chapter 3


In Chapter 3, you will learn how to find the best learning settings by running Ray Tune experiments.

  • You will be able to tune hyperparameters and network size for Ray RLlib algorithms.
  • You will be able to run automated experiments with different environment and hyperparameter settings.
  • You will be able to allocate CPU and GPU resources efficiently in your local Ray server for the fastest experiment execution.
  • You will visualize experiment results using Tensorboard and pick the best learning settings.


Atari DQN Preprocessing

Chapter 4


In Chapter 4, we will solve a problem where the agent learns directly from images.

  • You will be able to preprocess images to reduce input complexity while preserving critical information.




Planned features


Easy to digest


Bite sized video lessons with no fluff (on an average 6 mins long and rarely over 10 mins).

The whole course can be completed in 4 hours (including exercises).

Easy to follow


All videos will have closed captions.

Learn by doing


Video lessons and demonstrations will be followed by coding exercises whenever possible.

Project based


The exercises will be part of an overarching project, where you will teach one agent to manage shop inventory and another agent that learns directly from images.


Hi, I am Dibya, the instructor of this course 👋





"Dibya is one of the most fluent Python instructors in the community"


Anton Caceres - Python Software Foundation Fellow

"Dibya cares deeply about students and makes complex concepts easily accessible"


Hadrien Lacroix - Data Science Curriculum Manager, Datacamp


"No matter how difficult the task, Dibya made sure everyone left with a smile"


Olga Kupriyanova, PyLadies Organizer


Pre-enroll to support this course🌱


If you want to support this course, please pre-enroll. As an early supporter, you will get some nice perks: early access to lessons and a 50% discount on the launch price of $90.

To ensure you don't have a bad experience, your enrollment is covered by an unconditional money-back guarantee until launch + an additional 14-days after the launch.

Thank you in advance for supporting this course!