Pytorch implementation of Mastering Diverse Domains through World Models. DreamerV3 is a scalable algorithm that outperforms previous approaches across various domains with fixed hyperparameters.

Instructions

Get dependencies:

pip install -r requirements.txt

Run training on DMC Vision:

python3 dreamer.py --configs dmc_vision --task dmc_walker_walk --logdir ./logdir/dmc_walker_walk

Monitor results:

tensorboard --logdir ./logdir

Benchmarks

So far, this repository allows testing the following benchmarks.

Environment	Observation	Action	Description
DMC Proprio	State	Continuous	This benchmark contains 18 continuous control tasks with low-dimensional inputs and a budget of 500K environment steps.
DMC Vision	Image	Continuous	This benchmark consists of 20 continuous control tasks where the agent receives only high-dimensional images as inputs and a budget of 1M environment steps.
Atari 100k	Image	Discrete	This benchmark includes 26 Atari games and a budget of only 400K environment steps, amounting to 100K steps after action repeat or 2 hours of real time.
Crafter	Image	Discrete	This survival environment evaluates diverse agent abilities, including exploration, reasoning, credit assignment, and generalization.
Memory Maze	Image	Discrete	Memory Maze is a 3D benchmark with randomized mazes to evaluate RL agents' long-term memory.

Results

DMC Vision

Atari 100k

DMC Proprio

Acknowledgments

This code is heavily inspired by the following works:

danijar's Dreamer-v3 jax implementation: https://github.com/danijar/dreamerv3
danijar's Dreamer-v2 tensorflow implementation: https://github.com/danijar/dreamerv2
jsikyoon's Dreamer-v2 pytorch implementation: https://github.com/jsikyoon/dreamer-torch
RajGhugare19's Dreamer-v2 pytorch implementation: https://github.com/RajGhugare19/dreamerv2
denisyarats's DrQ-v2 original implementation: https://github.com/facebookresearch/drqv2