Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Q4 cleanup #1061

Merged
merged 2 commits into from
Apr 19, 2023
Merged

Q4 cleanup #1061

merged 2 commits into from
Apr 19, 2023

Conversation

sw
Copy link
Contributor

@sw sw commented Apr 19, 2023

Cleanup following #951 and #1046:

  • remove unused ggml_vec_dot_q4_0
  • warn for unused C functions (I didn't touch C++)
  • use ggml_is_quantized for the work buffer calculation

@sw sw marked this pull request as ready for review April 19, 2023 14:19
@sw sw requested review from dfyz and ggerganov April 19, 2023 14:23
@dfyz
Copy link
Collaborator

dfyz commented Apr 19, 2023

I think that everything inside this #ifdef should be removed as well. More precisely, bytes_from_q4_0_twoblocks_avx512() and dot_q4_0_twoblocks_avx512() should be removed, since they are only used it ggml_vec_dot_q4_0().

I'm guessing the CI didn't catch it with -Wno-unused-function because we only test AVX-512 under MSVC? Might be a good idea to include an AVX-512 build for Linux as well, instead of ACCELERATE (which is a no-op on Linux).

I just realized that CI wouldn't fail in any case because -Wno-unused-function is only a warning, not an error.

@sw
Copy link
Contributor Author

sw commented Apr 19, 2023

You are right. Parts of this clever code may be useful for other quantization types, but that's what the git history is for.

We might want to add -Werror if CI should catch warnings.

Copy link
Collaborator

@dfyz dfyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if I can/should approve this, but this PR looks pretty uncontroversial to me.

@ggerganov ggerganov merged commit f3d4edf into ggerganov:master Apr 19, 2023
@sw sw deleted the q4-cleanup branch April 19, 2023 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants