A personal deep reinforcement learning project in Python using TensorFlow and Keras. Includes a custom DQN library and 4 examples demonstrating Q-Learning and DQN in various Gymnasium environments.

The dqn package is a custom implementation of a Deep Q-Network (DQN) agent with utilities for:

• Experience replay (uniform and prioritized)
• Exploration policies (epsilon-greedy and boltzman)
• Atari frame preprocessing and staking
• Optimized DQN operations using TensorFlow compiled function: experiences are fetched from replay buffer in RAM, while processing, mini-batch formation and training is perform on GPU. Requires GPU with CUDA support and proper NVIDIA drivers. Follow this guide for installing TensorFlow with CUDA: https://www.tensorflow.org/install/pip

Classic tabular Q-Learning for Taxi-v3 using NumPy. View example

DQN for CartPole-v1 using TensorFlow and Keras. View example

Key features:

• Uses a simple feedforward neural network to approximate Q-values.
• Utilizes DQNAgent class with an esilon-greedy policy.
• Implements experience replay through ReplayBuffer class for stable learning.
• Contains raw step-by-step DQN training loop.

DQN agent for Atari, based on the original Google DeepMind implementation. View example

Key features:

• Employs Double-DQN for more stable learning.
• Process game frames using convolutional neural networks (CNNs).
• Utilizes frame stacking of 4 frames to capture movement.

Rainbow DQN agent for Atari that combines multiple enhancements to improve learning performance. View example

Key features:

• Dueling Q-Networks to separately estimate state values and advantages.
• Prioritized Experience Replay (PER) for more efficient learning.
• N-step learning to propagate rewards faster.
• Vectorized environments to accelerate training.

Clone the repository and install the dqn package and its dependencies:

git clone https://github.com/pabloramesc/dqn-lab.git
cd dqn-lab
pip install -e .
pip install -r requirements.txt

Run any example script from the project root:

python examples/taxi-qlearning.py

Make sure dqn package and all dependencies in requirements.txt are installed.

Note: To avoid using pre-trained models, delete or move the corresponding model files in the model folders.

This project is licensed under the MIT License. See the LICENSE file for details.