Tracking Metrics of Interest

[mat-adamec]

It seems like trying to get a good list of metrics together is a recurring topic. It seemed like a good idea to make a centralized place for suggestions/ideas here. Maybe we can compile a list and then discuss it next meeting?

Some of the metrics brought up by Brian at the meeting today include:

user metrics:
- how many total users?
- how many users in specific time periods?
- how many users running actual analyses?

metrics outside the facility:
- for emails lists
- for people who look at coffea-casa but don't sign in

and for Dask itself:
- how many workers on average per user?
- did anybody only use 1 worker?
- proof that users are using coffea-casa for actual analyses (or more than just toy runs)
- maybe we can get more information about CPU/RAM/etc. from Condor

If I recorded something wrong or missed an important suggestion, feel free to tag on (and also add in more!)

(@bbockelm @oshadura @durbar2003)

[alexander-held]

Maybe something like an average time it takes from a worker being requested to it starting to run (related to how busy the batch system is)?

[mat-adamec]

Not for Durbar, but still tangentially relevant: it was also mentioned today that @HollyWingren should look into metrics related to strong vs. weak scaling, as defined below:

• Strong scaling is for a fixed number of events (how does it change as you add cores or more throughput to the system?) This must run out because finite work can't be divided among an infinite number of resources.

• Weak scaling doubles both problem size and throughput.

I remembered we talked about this over a year ago now, and this was linked by Oksana as a useful theory resource pertaining to the subject: https://www.kth.se/blogs/pdc/2018/11/scalability-strong-and-weak-scaling/