Crash on MacOS with the example

[pminev]

Hi,

I tried the example that is attached in the README. Running on MacBook Pro M3 Max

huggingface-cli download ChenMnZ/Llama-3-8b-instruct-EfficientQAT-w2g128-GPTQ --local-dir ${model_dir}
python tools/run_pipeline.py -o ${model_dir} -m llama-3-8b-2bit -q int_n

and all steps are completed fine until the last which is the actual inference where I received a bus error:

.zsh: bus error  3rdparty/llama.cpp/build/bin/llama-cli -m "./models/models.INT_N.gguf" -p ...

The command was the default one which is called from the pipeline script:

3rdparty/llama.cpp/build/bin/llama-cli -m /Users/pacominev/repos/tmp/T-MAC/models/models.INT_N.gguf -n 128 -t 4 -p "Microsoft Corporation is an American multinational corporation and technology company headquartered in Redmond, Washington." -ngl 0 -c 2048

Any ideas?