[Safetensors] Fix mmap for Windows system

[DrownFish19]

PR types

Bug fixes

PR changes

APIs

Description

To address the disparities in mmap APIs across Windows and Unix, this PR includes checks for operating system differences.

The mmap comment is copied as follows:

Windows: mmap(fileno, length[, tagname[, access[, offset]]])

Maps length bytes from the file specified by the file handle fileno, and returns a mmap object. If length is larger than the current size of the file, the file is extended to contain length bytes. If length is 0, the maximum length of the map is the current size of the file, except that if the file is empty Windows raises an exception (you cannot create an empty mapping on Windows).

Unix: mmap(fileno, length[, flags[, prot[, access[, offset]]]])

Maps length bytes from the file specified by the file descriptor fileno, and returns a mmap object. If length is 0, the maximum length of the map will be the current size of the file when mmap is called. flags specifies the nature of the mapping. MAP_PRIVATE creates a private copy-on-write mapping, so changes to the contents of the mmap object will be private to this process, and MAP_SHARED creates a mapping that's shared with all other processes mapping the same areas of the file. The default value is MAP_SHARED.

To map anonymous memory, pass -1 as the fileno (both versions).

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 55.70%. Comparing base (e681019) to head (0dde9a2).
Report is 213 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/trainer/plugins/unified_checkpoint.py	75.00%	1 Missing ⚠️
paddlenlp/transformers/model_utils.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8734      +/-   ##
===========================================
- Coverage    55.73%   55.70%   -0.04%     
===========================================
  Files          623      623              
  Lines        97464    97405      -59     
===========================================
- Hits         54324    54259      -65     
- Misses       43140    43146       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[VictoryDong]

我认为280行这里可以改为： self.file_mmap = mmap.mmap(self.file.fileno(), 0, access=mmap.ACCESS_COPY )