will infinity use all available cpus? or would I rather just give it one cpu and run multiple instances? #191
-
|
just trying to deploy an embedding server for sentence transformers models without reinventing the wheel. |
Beta Was this translation helpful? Give feedback.
Answered by
michaelfeil
Apr 3, 2024
Replies: 1 comment
-
|
Yes - why not try it out?
|
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
nikcaryo-super
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Yes - why not try it out?
optimum(onnx) / ct2 / torch are using all cores. Don't be shocked, the default setting in these libs will be all physical cores (aka w/o hyperthreading), which should be a good choice.