main: failed to quantize model from './models/7B/ggml-model-f16.bin'

[huntharo]

It looks like this assumes numpy is installed but does not install it if it is not. I don't notice a mention of needing numpy to be installed before running in the README. Maybe a doc update and check / exit if not installed would be sufficient.

This does try to install numpy and torch but if that fails for some reason the build continues and then fails at the end.

The server also starts without error in this case... but has no response to any prompts (error or output).

1st Error - This should probably stop the build when it fails

pip install torch torchvision torchaudio sentencepiece numpy
exit

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
bash-3.2$ pip install torch torchvision torchaudio sentencepiece numpy
ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch

2nd Error

bash-3.2$ python3 convert-pth-to-ggml.py models/7B/ 1
Traceback (most recent call last):
  File "/Users/huntharo/dalai/convert-pth-to-ggml.py", line 23, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'

[ekp1k80]

try using #16 with virtualenv

[ekp1k80]

you can move the models you already download to llama.cpp folder in the #16 repo

[carlosvillu]

Hi,

I am getting this error maybe there is a relation with the other error.

{◂} ~/dalai ./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2
Illegal instruction: 4

[spullara]

I am getting the "Illegal instruction: 4" error as well. It looks like it is compiling it for x86_64 rather than ARM.

Makefile:24: Your arch is announced as x86_64, but it seems to actually be ARM64. Not fixing that can lead to bad performance. For more info see: https://github.com/ggerganov/whisper.cpp/issues/66#issuecomment-1282546789
sysctl: unknown oid 'machdep.cpu.leaf7_features'

[patrickstorm]

I am getting the same error as spullara. I am on an M1, and curiously, when I run the make command directly from the dalai folder in my root dir, the uname outputs are correct. It's just when running via npx dalai llama

Edit: I got it working, but this might be very unique to my computer. These are the steps I took:

• I removed the ~/dalai and ~/llama.cpp folders
• Made sure the dalai package was up to date by clearing my .npm cache
• I ran npx dalai install and quit right after it spit out my incorrect arch details, before it starts downloading the model
• Went to ~/llama.cpp and ran make
• Then re-ran npx dalai install

I'm not 100% sure why this worked, but I did it twice just to make sure.

[theRichu]

ggerganov/llama.cpp#41 (comment)

this comment works!