Colab notebook for demo

[markwsac]

Thanks a ton team for the release of model. Any notebook for demo? Some steps about mentioning model paths are a bit confusing. thanks

[shuentang]

me too

[eran-barkai]

+1

[merrymercy]

contributions are welcome

[markwsac]

I found working code here along with Colab notebook - Vicuna

[EnesGumuskaynak]

Hey @merrymercy If you assignee to me. I can do a notebook

[merrymercy]

@EnesGumuskaynak Please go ahead

[merrymercy]

@markwsac We cannot distribute merged weights because of the LLaMA model license. We encourage you to use our recommended way to run Vicuna.

[GeorvityLabs]

@EnesGumuskaynak any updates?

[EnesGumuskaynak]

I've just started

[GeorvityLabs]

@EnesGumuskaynak hope you make one using vicuna-7b , 8bit? with chat interface

[EnesGumuskaynak]

I'm working on it but colab free not offer much ram and dont allow to create swap file. 3 step need more than colab offer.

1. LLama weights to hugging face version convert ()
2. Hugging face version to vicuna version applying delta
3. Start vicuna model "Loading checkpoint " part

The common aspect of all 3 steps is to first load the model into ram and then start the process.

[EnesGumuskaynak]

since we cannot share the model weights, people will need to upload the original llama weights or the huggingface version into Colab. This will further complicate the process of running the demo on Google Colab. It seems more reasonable to create notebooks with documentation for local use. I am open to other suggestions as well.

[GeorvityLabs]

since we cannot share the model weights, people will need to upload the original llama weights or the huggingface version into Colab. This will further complicate the process of running the demo on Google Colab. It seems more reasonable to create notebooks with documentation for local use. I am open to other suggestions as well.

yea, you can use vicuna models already available on huggingface model hub , people have converted the delta weights to usable weights.