Webinar: How to train a policy for controlling a machine (using simulation and deep reinforcement learning)

The How to train a policy for controlling a machine webinar demonstrated the use of a simulation environment in deep reinforcement learning. The recording is available below, along with the supplementary materials – so you can try it out for yourself and explore some of the possibilities.

The webinar video provides a step-by-step guide to:

• building a statechart model as the training environment
• exporting the model as a standalone application and importing it to IntelliJ
• training a policy to control the machine
• and then finally importing the trained policy back to the simulation model for evaluation

This webinar’s focus is not on teaching deep reinforcement learning, but on how AnyLogic can be used to build simulated training environments and testbeds for use in artificial intelligence.

Webinar video structure:

1. Brief introduction to simulation modeling and AnyLogic software.
2. Introduction to Statecharts (mini-training).
3. Model definition and overview.
4. Step-by-step instruction of building a statechart model as the training environment. The model makes use of a statechart, an extended version of state diagrams and a visual construct that enables modeling of event and time‐driven behaviors.
5. Exporting the model and adding it to a workflow inside an IDE (IntelliJ).
6. Utilizing the Reinforcement Learning for Java (RL4J) library to make the agent learn a policy that takes a necessary sequence of actions to reach a desired state.
7. Importing the trained policy into the AnyLogic model as a testbed. The statechart model introduced in the first half of the webinar is then used as an environment to teach a learning agent how to take proper actions in order to reach a desired state.
8. Q&A.

AI Webinar Materials Download

Download contents:

• The statechart model created in AnyLogic (including the learned policy in the model folder).
• The full contents of the exported statechart model (you need this if you do not have AnyLogic Professional).
• The project created in IntelliJ; including the code, necessary libraries, and learned policy.
• RL4J library used in the training and the testbed (model). This is a subsection of the full RL4J library, consisting of the DQN algorithm and dependencies to run on any OS.
  ❗Please note that if the RL4J library was removed from the AL model or the IntelliJ project, you will need to add it back as a dependency. This can be done by following the instructions in the webinar video recording for AnyLogic (54:40 in webinar) or IntelliJ (47:10 in webinar).
• A document that goes over the code used in the training and provides information about the variables/functions used.

The materials above in combination with the video recording of the webinar cover everything that is needed to reproduce the entire training setup.

Many thanks to all who attended this exploration of training a machine control policy using deep reinforcement learning in a simulation modeling environment.

Visit our Artificial Intelligence page for more content, including other example RL models with source files and documentation.