PSE to provide analysis of resources needed by each SBG Task

[LucaCinquini]

Description: In order to be able to scale the SBG end-to-end workflow, SPS needs to know what resources (memory and CPUs) need to be allocated for executing each task. Upper limits are needed.

[mike-gangl]

i can't give an upper limit (because the more the better), but the test runs i've done usually are limited by ISO fit runs.

For ISO Fit, we used a memory optimized instance 4 vcpu/32 GB ram which worked pretty well for all other tasks, so i'd start with that as the upper bound for all.

[mike-gangl]

For what its worth, the actual sister worked used c5.9xlarge (36 cpu, 72GB ram) instances for ISOFit, so maybe after enough memory, it's CPU constrained (there are certainly multi-threading capabilities in the ISOFIT code).

[mike-gangl]

lastly (in my chain of thoughts), this is where we actually need the SPS to provide telemetry for a run, so that a user can tune the system based on actual executions in the system (and eventually give you an 'estimate' on the costs

[LucaCinquini]

That is a big machine....
We will start with the limits you specified: 4 vcpu/32 GB ram

[mike-gangl]

Agreed, we'll only move up to a large machine if we want to try and run an operational scenario to benchmark against.

[LucaCinquini]

Closing ticket since Mike provided the best possible estimate at this point. We still need to demonstrate scaling.