ms_wrapper.py have wrong, device_map had weights offloaded to the disk

[toniedeng]

class SunsimiaoTextGeneration(TorchModel):
def init(self, model_dir=None, *args, **kwargs):
super().init(model_dir, *args, **kwargs)
self.logger = get_logger()
# loading tokenizer
self.tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
self.model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True)
self.model = self.model.eval()

Message=The current device_map had weights offloaded to the disk. Please provide an offload_folder for them. Alternatively, make sure you have safetensors installed if the model you are using offers the weights in this format.
Source=C:\Users\Administrator.cache\modelscope\modelscope_modules\Sunsimiao\ms_wrapper.py
StackTrace:
File "C:\Users\Administrator.cache\modelscope\modelscope_modules\Sunsimiao\ms_wrapper.py", line 42, in init
self.model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True)
File "C:\Users\Administrator.cache\modelscope\modelscope_modules\Sunsimiao\ms_wrapper.py", line 20, in init (Current frame)
model = SunsimiaoTextGeneration(model) if isinstance(model, str) else model
File "C:\Users\Administrator\source\repos\Sunsimiao\scripts\inference_ms.py", line 4, in
pipe = pipeline(task=Tasks.text_generation,

[jingnant]

可能是由于显存不足，尝试在model给offload_folder参数
self.model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True,offload_folder="offload_folder")