Skip to content

Update to Gymnasium v1.0 #516

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 37 commits into
base: master
Choose a base branch
from
Open

Update to Gymnasium v1.0 #516

wants to merge 37 commits into from

Conversation

pseudo-rnd-thoughts
Copy link
Collaborator

@pseudo-rnd-thoughts pseudo-rnd-thoughts commented Jul 5, 2025 •
edited
Loading

Description

Updates CleanRL to Gymnasium v1.1, changing wrappers, increasing ale-py version, and python versions (3.9 to 3.13)

The pettingzoo implementation is broken as supersuit needs to be updated

Types of changes

  • Bug fix
  • New feature
  • New algorithm
  • Documentation

Checklist:

  • I've read the CONTRIBUTION guide (required).
  • I have ensured pre-commit run --all-files passes (required).
  • I have updated the tests accordingly (if applicable).
  • I have updated the documentation and previewed the changes via mkdocs serve.
    • I have explained note-worthy implementation details.
    • I have explained the logged metrics.
    • I have added links to the original paper and related papers.

If you need to run benchmark experiments for a performance-impacting changes:

  • I have contacted @vwxyzjn to obtain access to the openrlbenchmark W&B team.
  • I have used the benchmark utility to submit the tracked experiments to the openrlbenchmark/cleanrl W&B project, optionally with --capture_video.
  • I have performed RLops with python -m openrlbenchmark.rlops.
    • For new feature or bug fix:
      • I have used the RLops utility to understand the performance impact of the changes and confirmed there is no regression.
    • For new algorithm:
      • I have created a table comparing my results against those from reputable sources (i.e., the original paper or other reference implementation).
    • I have added the learning curves generated by the python -m openrlbenchmark.rlops utility to the documentation.
    • I have added links to the tracked experiments in W&B, generated by python -m openrlbenchmark.rlops ....your_args... --report, to the documentation.

@pseudo-rnd-thoughts
Remove SB3 as a necessary module
d0d0cff
@pseudo-rnd-thoughts
Remove additional sb3 checks
7865501
@pseudo-rnd-thoughts
Update poetry.lock and requirements + add `from __future__ import ann…
bbc117f 
…otations`
@pseudo-rnd-thoughts
Add get_obs_shape function
6587ee6
@pseudo-rnd-thoughts
Remove stable-baselines3 from test installs
d3da8ff
@pseudo-rnd-thoughts
Add copyright notices
11f70c5
@pseudo-rnd-thoughts
Update pre-commit
61739bd
@pseudo-rnd-thoughts
Update tests
7356872
@pseudo-rnd-thoughts
Update docs and tests
11782c3
@pseudo-rnd-thoughts
Merge branch 'master' into replace-poetry-with-uv
e03aac1
@pseudo-rnd-thoughts
Fix pre-commit and workflow
cd90426
@pseudo-rnd-thoughts
Update workflow action versions
4346403
@pseudo-rnd-thoughts
Add typing_extensions>=4.6.0 for optuna
a88614c
@pseudo-rnd-thoughts
Increase mujoco bound
e2680d8
@pseudo-rnd-thoughts
Merge branch 'replace-poetry-with-uv' into remove-sb3
26b95c4 
# Conflicts:
#	.github/workflows/tests.yaml
#	cleanrl/c51.py
#	cleanrl/c51_atari.py
#	cleanrl/c51_atari_jax.py
#	cleanrl/c51_jax.py
#	cleanrl/ddpg_continuous_action.py
#	cleanrl/ddpg_continuous_action_jax.py
#	cleanrl/dqn.py
#	cleanrl/dqn_atari.py
#	cleanrl/dqn_atari_jax.py
#	cleanrl/dqn_jax.py
#	cleanrl/qdagger_dqn_atari_impalacnn.py
#	cleanrl/qdagger_dqn_atari_jax_impalacnn.py
#	cleanrl/sac_atari.py
#	cleanrl/sac_continuous_action.py
#	cleanrl/td3_continuous_action.py
#	cleanrl/td3_continuous_action_jax.py
#	cleanrl_utils/buffers.py
#	docs/get-started/basic-usage.md
#	poetry.lock
#	pyproject.toml
#	requirements/requirements-atari.txt
#	requirements/requirements-cloud.txt
#	requirements/requirements-dm_control.txt
#	requirements/requirements-docs.txt
#	requirements/requirements-envpool.txt
#	requirements/requirements-jax.txt
#	requirements/requirements-mujoco.txt
#	requirements/requirements-optuna.txt
#	requirements/requirements-pettingzoo.txt
#	requirements/requirements-procgen.txt
#	requirements/requirements.txt
@pseudo-rnd-thoughts
Remove sb3 from requirements
7ca4c96
@pseudo-rnd-thoughts
Update to Gymnasium v1.0
cf6554d
@pseudo-rnd-thoughts
Revert mujoco<=2.3.3
6e5693d
@pseudo-rnd-thoughts
Add export MUJOCO_GL=egl as environment variable
bf811e0
@pseudo-rnd-thoughts
Use MUJOCO_GL egl in mujoco tests
7d84668
@pseudo-rnd-thoughts
Split up the test_mujoco.py file
2d490b9
@pseudo-rnd-thoughts
Specify the DISPLAY: :0
6aa2982
@pseudo-rnd-thoughts
register ale-py envs
9026018
@pseudo-rnd-thoughts
Add xvfb screen
7350f6e
@pseudo-rnd-thoughts
Merge branch 'replace-poetry-with-uv' into remove-sb3
c2f7c6d 
# Conflicts:
#	uv.lock
@pseudo-rnd-thoughts
Update lock
800be9b
@pseudo-rnd-thoughts
Update pytorch version to 2.4.1 as 2.5.1 doesn't have py38 wheels
4339766
@pseudo-rnd-thoughts
Add optax and chex to requirements
ba42e1c
@pseudo-rnd-thoughts
Merge branch 'remove-sb3' into gymnasium-v1.0
b08a2e7 
# Conflicts:
#	pyproject.toml
#	requirements/requirements-atari.txt
#	requirements/requirements-cloud.txt
#	requirements/requirements-dm_control.txt
#	requirements/requirements-docs.txt
#	requirements/requirements-envpool.txt
#	requirements/requirements-jax.txt
#	requirements/requirements-mujoco.txt
#	requirements/requirements-optuna.txt
#	requirements/requirements-pettingzoo.txt
#	requirements/requirements-procgen.txt
#	requirements/requirements.txt
#	uv.lock
@pseudo-rnd-thoughts
Update requirements
274f9d8
@vercel Vercel
Copy link

vercel bot commented Jul 5, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
cleanrl ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 8, 2025 2:36pm

@pseudo-rnd-thoughts pseudo-rnd-thoughts changed the title Gymnasium v1.0 Update to Gymnasium v1.0 Jul 5, 2025
@pseudo-rnd-thoughts
Copy link
Collaborator Author

pseudo-rnd-thoughts commented Jul 6, 2025 •
edited
Loading

Supersuit is broken and needs to be updated as the vector environment returns the wrong info type

@pseudo-rnd-thoughts pseudo-rnd-thoughts mentioned this pull request Jul 7, 2025
Closed
57 tasks
@pseudo-rnd-thoughts
Merge branch 'master-upstream' into gymnasium-v1.0
21e6c7b 
# Conflicts:
#	.github/workflows/tests.yaml
#	cleanrl/c51_atari.py
#	cleanrl/c51_atari_jax.py
#	cleanrl/dqn_atari.py
#	cleanrl/dqn_atari_jax.py
#	cleanrl/ppo_atari.py
#	cleanrl/ppo_atari_lstm.py
#	cleanrl/ppo_atari_multigpu.py
#	cleanrl/rainbow_atari.py
#	cleanrl/sac_atari.py
#	pyproject.toml
#	requirements/requirements-atari.txt
#	requirements/requirements-cloud.txt
#	requirements/requirements-dm_control.txt
#	requirements/requirements-docs.txt
#	requirements/requirements-envpool.txt
#	requirements/requirements-jax.txt
#	requirements/requirements-mujoco.txt
#	requirements/requirements-optuna.txt
#	requirements/requirements-pettingzoo.txt
#	requirements/requirements-procgen.txt
#	requirements/requirements.txt
#	uv.lock
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@pseudo-rnd-thoughts