You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently bucket URLs (i.e. s3://, gcs://) are only supported in conjunction with .spec.cacheProfile. It would be nice to be able to load from those locations without needed to setup a cache profile.
Option A: Wait for vllm-project/vllm#10192 to land.
Option B: Load to disk/mem on the node, then point the serving engine at that mount path.
The text was updated successfully, but these errors were encountered:
Option B might be useful for engines other than vLLM. It is probably worthwhile checking to see if those engines already support loading directly from bucket URLs at this point.
Currently bucket URLs (i.e.
s3://
,gcs://
) are only supported in conjunction with.spec.cacheProfile
. It would be nice to be able to load from those locations without needed to setup a cache profile.Option A: Wait for vllm-project/vllm#10192 to land.
Option B: Load to disk/mem on the node, then point the serving engine at that mount path.
The text was updated successfully, but these errors were encountered: