Save 10 min of plot time and minimize the 5s Lookup WARNING on plotter/harvester machine #6648
rmica
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Maybe someone already suggested this solution.
For those who manually script their plotting on Linux.
Leave the plot on the NVME with the tmp2 == dest option.
Ex: -t /opt/nvme0 -2 /opt/nvme0/_THREAD_A -d /opt/nvme0/_THREAD_A
Create a custom cgroup/blkio/chiaslowcopy, and read-limit the NVME device to about 80-100 MB/s.
Write a script looking for .plot files in the _THREAD dirs, copying them one by one to destination HDD,
(suffix the destination file with .tmp until the end of the copy, and then rename it).
Add the copy PID to the chiaslowcopy's tasks.
At the end of the copy remove the plot from its _THREAD dir.
This will reduce the impact of the copy on the harvester's lookup time.
The copy time is about 15-30 mins (depending on read-limit and parallel I/O load). The new plot started on the NVME will not claim full space until phase 2, so you have plenty of time to slow-copy the file without additional NVME space requirement.
Hope this helps.
Beta Was this translation helpful? Give feedback.
All reactions