Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try to understand the file mapping #7

Closed
mhtvsSFrpHdE opened this issue Jul 19, 2022 · 1 comment
Closed

Try to understand the file mapping #7

mhtvsSFrpHdE opened this issue Jul 19, 2022 · 1 comment

Comments

@mhtvsSFrpHdE
Copy link
Owner

mhtvsSFrpHdE commented Jul 19, 2022

By read these link:
https://docs.microsoft.com/en-us/windows/win32/memory/file-mapping
https://stackoverflow.com/questions/31349539/how-file-mapping-works

Here is the guess about why kernel cache will not override mapped file (virtual machine disk file):
Virtual machine programs sure not to load the entire 120 GB into system RAM,
they use address via file mapping, and when they do I/O,
Windows handle the disk access and RAM cache.

When a process does CreateFileMappingW or something, they say:
"Hey, I'm important."
During the entire system does I/O, the one created file mapping has higher priority.
Unless explicit clear standby list, they won't shrink their size in RAM until closed.

A test can give more details (doing the test, not complete yet):

  • Open RAMMap
  • Open several virtual machines to fill the standby list
  • Open several big video files to see the standby list change
  • Close the virtual machine and video player
  • Now the standby list is full and no file mapping was created. Run prefetch
  • Check performance, will either slow or fast
    • If slow, then the standby list is first come, first served, no override
    • If fast, then there is an override between cache
@mhtvsSFrpHdE
Copy link
Owner Author

The test result shows:

  • After virtual machine close, prefetch can shrink vmdk from 10,576,564 to 6,350,312
    New files are cached, and some old caches are freed.
  • It's not a first come, first served, no override list
    There are indeed exist override rules.
  • File mapping will cause some files to have priority in the standby list
    When file mapping is open or active, it won't shrink (mostly look like), until closed.

This can explain why sometimes prefetch lost its efficacy.
When physical RAM usage is relative heavy (or at least Windows think it is),
anything that can still remain inside the standby list will have priority.

Unless to get rid of them, our prefetch can't put the file into the standby list without explicitly open,
and explicitly open will cause the file can't be deleted.

In my opinion, a virtual machine does not have to take that vast amount of disk cache.
Inside the VM, there's another layer of guest operating system kernel cache,
they can do well for the frequent access file.

So when vmware-vmx.exe exist && standby list is full,
clear the standby list before prefetch can solve the problem:
shrink too big vmdk and put files into standby list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant