To be implemented! Coming soon!

To load models in 4bits with transformers and bitsandbytes, you have to install accelerate and transformers from source and make sure you have the latest version of the bitsandbytes library. After installing PyTorch, you can achieve the above with the following command:

pip install -U -r requirements.txt

1. Change the bnb.py in your peft path(python path/peft/tuners/lora/bnb.py) with the new one.

2. Change the layer.py in your peft path(python path/peft/tuners/lora/layer.py) with the new one.

Use the merge.py to merge the base_model and the adapter

Our code is based on QLoRA