Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance manage numcalc #101

Merged
merged 8 commits into from
Jun 8, 2023
Merged

Enhance manage numcalc #101

merged 8 commits into from
Jun 8, 2023

Conversation

f-brinkmann
Copy link
Contributor

@f-brinkmann f-brinkmann commented May 17, 2023

  • More simple and robust RAM management
  • Improved verbosity in command line output

Here are the changes in the RAM management 79d07c0

And this is an example of the new output


Starting manage_numcalc with the following arguments [May 17 2023, 21:04:01]
----------------------------------------------------------------------------
project_path: /home/bruel/Downloads/projects_empty
numcalc_path: NumCalc
max_ram_load: 30.00 GB (31.05 GB detected)
ram_safety_factor: 1.05
max_cpu_load: 90 %
max_instances: 8 (8 cores detected)
wait_time: 15 seconds
starting_order: alternate
confirm_errors: False

NumCalc executable: NumCalc

Obtaining RAM estimates for /home/bruel/Downloads/projects_empty/right_ear/NumCalc/source_1
Obtaining RAM estimates for /home/bruel/Downloads/projects_empty/left_ear/NumCalc/source_1

Per project summary of instances that will be run
-------------------------------------------------
Detected 2 Mesh2HRTF projects in
/home/bruel/Downloads/projects_empty

128/128 frequency steps to run in 'right_ear'
128/128 frequency steps to run in 'left_ear'


Started 'right_ear' project (1/2, May 17 2023, 21:04:10)
--------------------------------------------------------
Running 128/128 unfinished frequency steps in the project

1/128 starting instance from 'right_ear' (May 17 2023, 21:04:10)
source 1, step 128, 22050.0 Hz
estimated 7.99 GB RAM of available 26.36 GB required

2/128 starting instance from 'right_ear' (May 17 2023, 21:04:25)
source 1, step 30, 5167.97 Hz
estimated 4.53 GB RAM of available 18.87 GB required

3/128 starting instance from 'right_ear' (May 17 2023, 21:04:40)
source 1, step 127, 21877.7 Hz
estimated 7.99 GB RAM of available 14.61 GB required

4/128 starting instance from 'right_ear' (May 17 2023, 21:04:55)
source 1, step 33, 5684.77 Hz
estimated 4.53 GB RAM of available 7.08 GB required

... waiting for resources and checking every second (May 17 2023, 21:05:10)
4 NumCalc instances running at 51.20% CPU load
2.88 GB RAM available (27.12 GB used), 4.76 GB required

@chris-hld, this should be pretty much identical to #100 but does not use psutil.psutil.virtual_memory().used because the documentation mentions that this is for informal purposes only. It seems to work well on my machine. Can you check it on the cluster the next days?

@f-brinkmann f-brinkmann marked this pull request as draft May 17, 2023 18:59
@chris-hld
Copy link

Seems to work fine!
I would also output ram.available once in the beginning to double check the configuration, maybe after the ram detected print.

@f-brinkmann
Copy link
Contributor Author

@chris-hld: No including in echoing the parameters max_ram_load: 30.00 GB (31.05 GB detected, 27.68 GB available)

I will wait with the patch release of this until we figured #104

@f-brinkmann f-brinkmann marked this pull request as ready for review May 24, 2023 17:20
@f-brinkmann f-brinkmann requested a review from SDX-LV May 24, 2023 17:20
@chris-hld
Copy link

I can report that the RAM resource management now works as expected. I will highlight that at least on my system 30s between call and measuring RAM again was not enough time to fully register/load. With 60s in between this seems resolved.
Last commit produced this output

-----------------------------------------------------
Running 13/256 unfinished frequency steps in the project

1/13 starting instance from 'FABIAN' (May 29 2023, 10:43:37)
source 2, step 128, 22050.0 Hz
estimated 81.55 GB RAM of available 380.00 GB required

2/13 starting instance from 'FABIAN' (May 29 2023, 10:44:37)
source 2, step 106, 18260.2 Hz
estimated 56.11 GB RAM of available 302.74 GB required

3/13 starting instance from 'FABIAN' (May 29 2023, 10:45:37)
source 1, step 128, 22050.0 Hz
estimated 81.55 GB RAM of available 249.50 GB required

4/13 starting instance from 'FABIAN' (May 29 2023, 10:46:37)
source 2, step 108, 18604.7 Hz
estimated 59.77 GB RAM of available 172.23 GB required

5/13 starting instance from 'FABIAN' (May 29 2023, 10:47:38)
source 1, step 127, 21877.7 Hz
estimated 81.55 GB RAM of available 115.53 GB required

... waiting for resources and checking every second (May 29 2023, 10:48:38)
5 NumCalc instances running at 25.70% CPU load
41.32 GB RAM available (401.78 GB used), 62.76 GB required

6/13 starting instance from 'FABIAN' (May 29 2023, 12:15:36)
source 2, step 109, 18776.9 Hz
estimated 59.77 GB RAM of available 63.78 GB required

... waiting for resources and checking every second (May 29 2023, 12:16:36)
6 NumCalc instances running at 32.70% CPU load
7.2 GB RAM available (435.89 GB used), 66.73 GB required

7/13 starting instance from 'FABIAN' (May 29 2023, 12:50:50)
source 2, step 126, 21705.5 Hz
estimated 78.95 GB RAM of available 79.71 GB required

... waiting for resources and checking every second (May 29 2023, 12:51:51)
6 NumCalc instances running at 13.90% CPU load
25.52 GB RAM available (417.58 GB used), 66.73 GB required

8/13 starting instance from 'FABIAN' (May 29 2023, 13:00:02)
source 1, step 112, 19293.8 Hz
estimated 63.55 GB RAM of available 100.36 GB required

... waiting for resources and checking every second (May 29 2023, 13:01:02)
7 NumCalc instances running at 14.10% CPU load
40.16 GB RAM available (402.93 GB used), 72.47 GB required

9/13 starting instance from 'FABIAN' (May 29 2023, 13:20:59)
source 2, step 122, 21016.4 Hz
estimated 73.07 GB RAM of available 73.85 GB required

... waiting for resources and checking every second (May 29 2023, 13:21:59)
6 NumCalc instances running at 14.20% CPU load
24.96 GB RAM available (418.14 GB used), 72.47 GB required

@SDX-LV
Copy link
Contributor

SDX-LV commented May 29, 2023

Hi, I have re-installed my system and currently do not even have Python installed yet. + going on a long vacation soon, So I will not be of much help during the summer.

@f-brinkmann f-brinkmann merged commit f003394 into develop Jun 8, 2023
@f-brinkmann f-brinkmann deleted the enhance/manage_numcalc branch June 8, 2023 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants