Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to install Metal wheel on Mac M1 "ggml-common.h" file not found #1332

Closed
abetlen opened this issue Apr 5, 2024 · 0 comments
Closed

Failed to install Metal wheel on Mac M1 "ggml-common.h" file not found #1332

abetlen opened this issue Apr 5, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@abetlen
Copy link
Owner

abetlen commented Apr 5, 2024 •
edited
Loading

image

Steps

  • Install Metal Wheel via pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/metal
  • Run model with n_gpu_layers > 0
@abetlen abetlen added the bug Something isn't working label Apr 5, 2024
@abetlen abetlen closed this as completed in b3bfea6 Apr 6, 2024
xhedit pushed a commit to xhedit/llama-cpp-conv that referenced this issue Apr 6, 2024
@abetlen @xhedit
fix: Always embed metal library. Closes abetlen#1332
1bcb07e
xhedit added a commit to xhedit/llama-cpp-conv that referenced this issue Apr 6, 2024
@xhedit @Limour-dev
@abetlen @lawfordp2017 @ymikhailov @CISC
merge test (#2)
bf766bd 
* feat: add support for KV cache quantization options (abetlen#1307)

* add KV cache quantization options

abetlen#1220
abetlen#1305

* Add ggml_type

* Use ggml_type instead of string for quantization

* Add server support

---------

Co-authored-by: Andrei Betlen <abetlen@gmail.com>

* fix: Changed local API doc references to hosted (abetlen#1317)

* chore: Bump version

* fix: last tokens passing to sample_repetition_penalties function (abetlen#1295)

Co-authored-by: ymikhaylov <ymikhaylov@x5.ru>
Co-authored-by: Andrei <abetlen@gmail.com>

* feat: Update llama.cpp

* fix: segfault when logits_all=False. Closes abetlen#1319

* feat: Binary wheels for CPU, CUDA (12.1 - 12.3), Metal (abetlen#1247)

* Generate binary wheel index on release

* Add total release downloads badge

* Update download label

* Use official cibuildwheel action

* Add workflows to build CUDA and Metal wheels

* Update generate index workflow

* Update workflow name

* feat: Update llama.cpp

* chore: Bump version

* fix(ci): use correct script name

* docs: LLAMA_CUBLAS -> LLAMA_CUDA

* docs: Add docs explaining how to install pre-built wheels.

* docs: Rename cuBLAS section to CUDA

* fix(docs): incorrect tool_choice example (abetlen#1330)

* feat: Update llama.cpp

* fix: missing logprobs in response, incorrect response type for functionary, minor type issues. Closes abetlen#1328 abetlen#1314

* fix: missing logprobs in response, incorrect response type for functionary, minor type issues. Closes abetlen#1328 Closes abetlen#1314

* feat: Update llama.cpp

* fix: Always embed metal library. Closes abetlen#1332

* feat: Update llama.cpp

* chore: Bump version

---------

Co-authored-by: Limour <93720049+Limour-dev@users.noreply.github.com>
Co-authored-by: Andrei Betlen <abetlen@gmail.com>
Co-authored-by: lawfordp2017 <lawfordp@gmail.com>
Co-authored-by: Yuri Mikhailov <bitsharp@gmail.com>
Co-authored-by: ymikhaylov <ymikhaylov@x5.ru>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@abetlen