This repository contains the models and scripts for reproducing the ARCH-COMP Moonlight benchmarks.

The benchmarks have been implemented via python 3.11 and Poetry. Moreover, since the benchmarks are based on the classical ARCH-COMP problems, Matlab must be available in the running environment (the suite has been tested with Matlab R2024a).

poetry install

poetry run experiments

The results of the benchmarks are stored in the output directory. They can also be visualized by looking at the paper_table.ipynb Jupyter notebook.

The project is organized in the following way:

• arch_comp_moonlight contains the core implementation of the benchmarking suite.
• experiments contains the code for running the specific benchmark problems.
• models contains the models of the benchmark problems, as provided by the ARCH-Comp team.
• output contains the results of the benchmarks.

• Refresh the changelog by running poetry run git-changelog
• Then commit following the Conventional Commits standard.
• Add a tag to the commit, named as the last item of the changelog (e.g. for v0.3.0: git tag -a v0.3.0 -m "v0.3.0").
• Finally, push the changes to the repository.

To add the new benchmark, create a new directory for it in the experiments directory. In order to have it ready, one needs to:

• Implement the Simulator abstract class for the specific benchmark.
• Implement the Runner class for the specific benchmark.

These two classes are responsible for, respectively, passing the data to the simulator, and passing the results back to the monitor and to the optimizer.

To add a new monitor, create a new file (or directory) for it in the arch_comp_moonlight/monitors directory. It is sufficient that it implements the Monitor abstract class, to make it usable by the benchmarks.

To add a new optimizer, create a new file (or directory) for it in the arch_comp_moonlight/optimizers directory. It is sufficient that it implements the Optimizer abstract class, to make it usable by the benchmarks.

This project is licensed under the MIT License - see the LICENSE file for details.