Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Information on new important updates in llama.cpp #627

Closed
zsogitbe opened this issue Mar 27, 2024 · 1 comment
Closed

Information on new important updates in llama.cpp #627

zsogitbe opened this issue Mar 27, 2024 · 1 comment
Labels
Upstream Tracking an issue in llama.cpp
Milestone

Comments

@zsogitbe
Copy link
Contributor

zsogitbe commented Mar 27, 2024

There have been 3 important updates to the llama.cpp recently (a few days ago). One of them is the addition of pipeline parallelism (multi-threading) (ggerganov/llama.cpp#6017). The other important update was the correction of the embeddings bug. And one more update to release all GPU memory (this was also a bug - memory was not released). Thank you slaren!

These are very important updates. Looking forward to the addition of the latest llama.cpp code to LLamaSharp.

@AsakusaRinne AsakusaRinne added the Upstream Tracking an issue in llama.cpp label Mar 27, 2024
@AsakusaRinne AsakusaRinne added this to the v0.12.0 milestone Apr 3, 2024
@martindevans
Copy link
Member

See #662, that update should include these things.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Upstream Tracking an issue in llama.cpp
Projects
None yet
Development

No branches or pull requests

3 participants