A research platform for continual RL that allows for a progression of increasingly sophisticated behaviour.

Please find the documentation here.

AgarCL is based on the game Agar.io. It's a non-episodic, high-dimensional problem featuring stochastic, ever-evolving dynamics, continuous actions, and partial observability.

The recommended way to use AgarCL is within a Docker container running a Linux OS. This ensures there are no conflicts with other installed packages or platforms. This installation script will allow you to interact with AgarCL in a headless mode.

Follow these steps to set up the container:

1. Download the Dockerfile

   • Download the Dockerfile.txt.
   • You can jump to step 4 for instructions to use a pre-built image that we share.

2. Navigate to the Directory Containing the Dockerfile

   • Open your terminal and navigate to the folder where the Dockerfile.txt is located:

     cd /path/to/Dockerfile/directory

3. Build the Docker Image

   • Build the Docker image by specifying the custom Dockerfile using the -f flag:

     docker build -f Dockerfile.txt -t agarclimage .

   • Skip to step 5, now that your image is built.

4. Directly use the pre-built image

   • docker pull agarcl/agarclimage

5. Run the Docker Container

   • Once the image has been built, run the container:

     docker run --gpus all -it --name agarclcontainer agarclimage

   • This command will start the container with the name agarclcontainer. The --gpus all flag tells Docker to use all available GPUs on your host system for the container.

Now, let's install the platform on your system (agarclcontainer container):

1. Clone the AgarCL Repository

   • Clone the repository with the --recursive flag to ensure all submodules are included:

     git clone --recursive [email protected]:machado-research/AgarCL.git

2. Install the Platform

   • Change into the AgarCL directory:

     cd AgarCL

   • Run the installation command to set up the platform:

     python setup.py install --user

   • This will install the platform in your local user environment.

1. Clone the AgarCL-benchmark Repository

   • Clone the repository:

     git clone [email protected]/AgarCL/AgarCL-benchmark.git

2. Navigate to the AgarCL-Benchmark Directory

   • Change into the AgarCL-benchmark directory:

     cd AgarCL-benchmark

3. Clone the AgarCLgit Repository

   • Clone the AgarCL repository with the --recursive flag to ensure all submodules are included:

     git clone --recursive [email protected]:AgarCL/AgarCL.git

4. Navigate to the AgarCL Directory

   • Change into the AgarCL directory:

     cd AgarCL

5. Install the Platform

   • Run the installation command to set up the platform:

     python setup.py install --user

Ensure the project is compiled with clang++, not g++

💡 Before starting:

Follow the instructions here and make sure you have homebrew correctly installed and updated.

Make sure Command Line Tools are installed properly, follow the documentation.

Note: The installer will automatically install CMake 3.22 (a compatible version for this project)

Then follow these steps to set up the AgarCL environment on macOS:

1. Clone the repository:

   git clone --recursive [email protected]:AgarCL/AgarCL.git

2. Change into the project directory:

   cd AgarCL

3. Create a Python virtual environment:

   python -m venv agarclenv

4. Activate the virtual environment:

   source agarclenv/bin/activate

5. Run the installer script:

   ./install.sh

6. Install Python dependencies:

   pip install -r requirements.txt

7. Build & install the Python package:

   python setup.py install

1. Clone the repository:

   git clone --recursive [email protected]:AgarCL/AgarCL.git

2. Change into the project directory:

   cd AgarCL

3. Create a Python virtual environment:

   python -m venv agarclenv

4. Activate the virtual environment:

   source agarclenv/bin/activate

5. Make the install script executable:

   chmod +x install.sh

6. Run the installer script (may require sudo):

   sudo ./install.sh

7. Install Python dependencies:

   pip install -r requirements.txt

8. Build and install the Python package:

   python setup.py install

To run the Go Bigger example, execute the following line:

python project_path/bench/go_bigger_example.py

To run the Screen Observations example, execute the following line:

python project_path/bench/screen_obs_example.py

import gymnasium as gym

# Initialise the environment
env = gym.make("agario-screen-v0", render_mode="human")

# Reset the environment to generate the first observation
observation = env.reset()
for _ in range(1000):
    # this is where you would insert your policy
    c_target_space = gym.spaces.Box(low=-1, high=1, shape=(2,))
    d_target_space = gym.spaces.Discrete(3)
    action = [(c_target_space.sample(), d_target_space.sample())]

    # step (transition) through the environment with the action
    # receiving the next observation, reward and if the episode has terminated or truncated
    observation, reward, terminated, truncated, info = env.step(action)

    # If the episode has ended then we can reset to start a new episode
    if terminated or truncated:
        observation = env.reset()

env.close()

In order to play the game yourself or enable rendering in the gym environment, you will need to build the game client yourself on a system where OpenGL has been installed. Issue the following commands:

git submodule update --init --recursive
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j 2 client agario

This will output an executable named client in the directory agario

agario/client

Use your cursor to control the agent.

AgarCL allows you to save and load snapshots of the environment's state. This feature is useful for debugging, benchmarking, or resuming training from a specific point.

To save the current state of the environment, use the save_env_state method:

env.save_env_state('path_to_save_snapshot.json')

This will save the environment's state to a JSON file at the specified path.

To load a previously saved snapshot, use the load_env_state method:

env.load_env_state('path_to_snapshot.json')

Before loading a snapshot, ensure that the load_env_state option is enabled in the environment configuration. This will allow the environment to restore its state from the specified JSON file.

Here is an example of how to use these methods in a script:

import gymnasium as gym

# Initialize the environment
env = gym.make("agario-screen-v0", render_mode="human")

# Load a snapshot if available
env.load_env_state('snapshot.json')

# Reset the environment
env.reset()

# Perform some steps
for _ in range(100):
   action = [(env.action_space.sample(), env.action_space.sample())]
   observation, reward, terminated, truncated, info = env.step(action)
   if terminated or truncated:
      break

# Save the environment's state
env.save_env_state('snapshot.json')

env.close()

This functionality ensures reproducibility and allows for efficient experimentation with different configurations.

AgarCL provides functionality to record and save videos of the environment's execution. This is useful for visualizing agent behavior or debugging.

To enable video recording, set the record_video parameter to True in the environment configuration. You can also enable video recording programmatically:

env.enable_video_recorder()

To save the recorded video, use the generate_video method:

env.generate_video('path_to_save_video', 'video_name.avi')

This will save the video to the specified path with the given file name.

To stop recording, use the disable_video_recorder method:

env.disable_video_recorder()

Here is an example of how to record and save a video:

import gymnasium as gym

# Initialize the environment
env = gym.make("agario-screen-v0", render_mode="rgb_array")

# Enable video recording
env.enable_video_recorder()

# Reset the environment
env.reset()

# Perform some steps
for _ in range(100):
   action = [(env.action_space.sample(), env.action_space.sample())]
   observation, reward, terminated, truncated, info = env.step(action)
   if terminated or truncated:
      break

# Save the video
env.generate_video('videos', 'example_run.avi')

# Disable video recording
env.disable_video_recorder()

env.close()

Display the environment in a live GUI window for debugging, demos, and visually tracking your agent’s decisions as they happen.

An example of how to invoke the window:

import gymnasium as gym

# Initialize the environment
env = gym.make("agario-screen-v0", render_mode="human")

# Reset the environment
env.reset()

# Perform some steps
for _ in range(100):
   action = [(env.action_space.sample(), env.action_space.sample())]
   observation, reward, terminated, truncated, info = env.step(action)

   # Update the on-screen display
   env.render()
   if terminated or truncated:
      break

env.close()

This functionality allows you to capture and analyze the agent's performance visually.

If you use our environment, cite the following reference:

@article{aymanagarcl,
  title={The Cell Must Go On: Agar.io for Continual Reinforcement Learning},
  author={Mohamed A. Mohamed and Kateryna Nekhomiazh and Vedant Vyas,
Marcos M. Jos\'e and Andrew Patterson and Marlos C. Machado},
  journal={https://arxiv.org/abs/2505.18347},
  year={2025}
}

This implementation is built upon the AgarLE repository.