Trying to use PPO with EleutherAI/gpt-neo-1.3B gives error #121
Description
🐛 Describe the bug
Trying a hello world PPO with gpt-neo-1.3b.
def reward_fn_simple(samples):
total = []
for sample in samples:
if 'God' in sample:
total.append(100.0)
elif 'Pope' in sample:
total.append(-1000.0)
else:
total.append(0.0)
return total
config_file = yaml.safe_load(open("configs/ppo_gpt_neo.yml"))
config = TRLConfig.update(config_file, {})
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = transformers.AutoTokenizer.from_pretrained(config.model.tokenizer_path)
model = trlx.train(""EleutherAI/gpt-neo-1.3B", prompts=["Q: Who rules the world? A:"] * 100, reward_fn=reward_fn_simple, config=config
Fails with:
File "/root/trlxenv/lib/python3.8/site-packages/transformers/configuration_utils.py", line 254, in __getattribute__
return super().__getattribute__(key)
AttributeError: 'GPTNeoConfig' object has no attribute 'n_layer'
This is likely because gptj6b has n_layer
but it is called num_layer in gpt1.3b
Which trlX version are you using?
0.3.0
Additional system and package information
Linux, python 3.8.13, Linux