LLaVA-Next v1.6

[raminguyen]

Hello everyone,

I am currently working with this model: LLaVA-v1.6 Mistral 7B. I have my own image dataset, but the images are stored in array format. I would appreciate some guidance on how to convert these images into a suitable inputs for the model. Below is the code I am using:

prompt = ""

max_output_token = 500

prompt = f"[INST] \n{prompt} [/INST]"

inputs = processor(prompt, image, return_tensors="pt").to("cuda:0")

output = model.generate(**inputs, max_new_tokens=max_output_token)

response = processor.decode(output[0], skip_special_tokens=True)

pprint(response)

Thanks very much.