Is this codebase ready to use?

[Hodge931]

I got the following error:

Traceback (most recent call last):
  File "/home/harry/raftpp_0420/verl/trainer/main_ppo.py", line 59, in main
    run_ppo(config)
  File "/home/harry/raftpp_0420/verl/trainer/main_ppo.py", line 77, in run_ppo
    ray.get(runner.run.remote(config))
  File "/home/harry/miniconda3/envs/raftpp/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
    return fn(*args, **kwargs)
  File "/home/harry/miniconda3/envs/raftpp/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/home/harry/miniconda3/envs/raftpp/lib/python3.10/site-packages/ray/_private/worker.py", line 2771, in get
    values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
  File "/home/harry/miniconda3/envs/raftpp/lib/python3.10/site-packages/ray/_private/worker.py", line 919, in get_objects
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(NotImplementedError): ray::TaskRunner.run() (pid=1073013, ip=10.164.0.11, actor_id=0123973c436b13f1d7479b4201000000, repr=<main_ppo.TaskRunner object at 0x7f516f0130a0>)
  File "/home/harry/raftpp/verl/workers/reward_manager/naive.py", line 70, in __call__
    score = self.compute_score(
  File "/home/harry/raftpp/verl/utils/reward_score/__init__.py", line 47, in _default_compute_score
    raise NotImplementedError(f"Reward function is not implemented for {data_source=}")
NotImplementedError: Reward function is not implemented for data_source='numina_math'

Maybe the codebase is still not ready?

[hendrydong]

This should be easily fixed by modifying the reward design, such as: https://github.com/RLHFlow/Minimal-RL/blob/main/verl/utils/reward_score/__init__.py .

[Hodge931]

Could I get your reward design? Just to avoid any mismatch. Thanks a lot!