Add more arithmetic performance counters

[hkaiser]

Add more arithmetic performance counters exposing statistic properties calculated from the values gathered by querying arbitrary underlying counters. These new counters would work along the lines of the existing arithmetics counters (like /arithmetics/add) which can be invoked with a list of other counters to query in order to produce the required result.

For instance:

hello_world \
    --hpx:threads=4 \
    --hpx:print-counter=/threads/count/cumulative
    --hpx:print-counter=/arithmetics/average@/threads{locality#0/worker-thread#*}/count/cumulative

could produce something like:

hello world from OS-thread 1 on locality 0
hello world from OS-thread 0 on locality 0
hello world from OS-thread 3 on locality 0
hello world from OS-thread 2 on locality 0
/threads{locality#0/worker-thread#0}/count/cumulative,1,0.0025214,[s],27
/threads{locality#0/worker-thread#1}/count/cumulative,1,0.0025453,[s],33
/threads{locality#0/worker-thread#2}/count/cumulative,1,0.0025683,[s],29
/threads{locality#0/worker-thread#3}/count/cumulative,1,0.0025904,[s],33
/arithmetics/average@/threads{locality#0/worker-thread#*}/count/cumulative,1,0.0026002,[s],30.5

etc.

[aserio]

We should probably make this a GSoC project

[diehlpk]

@hkaiser https://github.com/STEllAR-GROUP/hpx/wiki/GSoC-2017-Project-Ideas#add-more-arithmetic-performance-counters

[Sumit2318]

Sir, would you please guide me How to start contributing to this project?

[hkaiser]

Sir, would you please guide me How to start contributing to this project?

This is a nice and simple task, just perfect for GSoC. To start off, you should build HPX on your machine and make sure the examples are running properly. The next step would be to familiarize yourself with the concepts of performance counters in HPX. I think the docs should give you a good head start to this (see http://stellar-group.github.io/hpx/docs/html/hpx/manual/performance_counters.html).

Once you understand them and how to use them, I'd suggest trying to find the existing simple arithmetic performance counter implementations (i.e. /arithmetic/add and friends) and to try to analyze how things actually work (in the code). This gives you a good platform to start thinking about a) what additional counters might be useful, and b) how to implement them.

Generally, I'd suggest that you join the #ste||ar IRC channel on freenode, there you will always find somebody to talk to. Alternatively subscribe to hpx-users@stellar.cct.lsu.edu and ask your questions there.

[hkaiser]

This was solved by merging #2745

[victor-ludorum]

Hello @hkaiser , If you don't mind Can you open this issue . So, that I can discuss some relative stuff related to arithmetic counters here .

[victor-ludorum]

Sir, I have researched on this topic and found some statistical stuff which I have mentioned in my discussion at user group like

• count
• covariance
• density
• kurtosis
• moment

and some more important ones . Kindly suggest some more important stuff related to this.

And please suggest one more thing that do I have to implement numeric functions too.
like

• LOG
• EXP

As in this issue there is a line " arithmetic performance counters exposing statistic properties " therefore, I have thought that it is all related to statistical stuffs

[hkaiser]

@victor-ludorum FWIW, table 36 on this page lists the existing arithmetic counters, and table 35 lists the existing statistical counters.

Also I'm not sure what LOG and EXP are supposed to do - could you elaborate, please?

[victor-ludorum]

Thanks @hkaiser Sir!! I have gone through the links specified by you .

Also I'm not sure what LOG and 'EXP` are supposed to do - could you elaborate, please?

Actually, there are arithmetic counters like addition, subtraction .. Therefore, Is the implementation of logarithms and exponentiation is also required . Although these are not related to any statistical analysis.
But they are related to arithmetics therefore I have mentioned them . So, Kindly suggest some arithmetic counters which I should work on . I have got an idea about statistical counters that should be worked on but I can't get any info about arithmetic counters which should be extended in the list . Is the implemented of GCD ,LCM are required in arithmetic counters . Please suggest .

[msimberg]

Just for the record, I suggested the log and exp functions, but was just brainstorming and they might not be useful. And they're indeed not statistical operations. In the end (IMO) the non-statistical arithmetic performance counters are a matter of convenience (nothing is strictly required). It's always possible to do transformations on your data afterwards in the analysis software of your choice. Sorry for any possible confusion I've caused.

[victor-ludorum]

@msimberg Thanks sir,:blush: for the reply . You have cleared my confusion . I would be pleased if you guide me further on this .

[victor-ludorum]

@hkaiser , @msimberg Sir, I have figured out steps that should be taken to add new arithmetic counters . If its possible then Can I implement anyone new counter like count or any else just to figure out more clearly all the work needed to add new counter . I know its a GSOC project and I can't work on them unless I have been selected or being assigned . So, just asking otherwise no problem 😊 .

[hkaiser]

@victor-ludorum In open source there is no rule saying that you can't work on something you're interested in. You don't have to ask for permission to try something, really.

[victor-ludorum]

Thanks @hkaiser I will work and submit a PR . So, that you can guide me how to improve that.

[victor-ludorum]

Hello @hkaiser @msimberg !!
As there are two types of Performance Counter which is General Statistics and Arithmetic Performance Counter as therein Existing Performance Counter . So, I think as the new counter is also related to statistics in arithmetic performance counter, we should add new performance counter to General Statistics Performance Counter too. There are many performance counters which are same in both like Max, Min, Median, Variance etc. Hence I think we should add performance counter in both of them.

I have asked this question in mail-group but I didn't get any reply. Therefore, I am asking here again.
Please suggest !!

Thanks !!

[victor-ludorum]

Hello, @hkaiser I think my previous comment is bit unclear. So, I am clearing it further here.
Sir, what I meant is that there are some counters which are same in both statistics performance counter and Arithmetic Performance Counter like Max, Min, Median and Variance. So, what I am thinking is that I can add same counters in both arithmetic and statistics performance counter, which have significance in both types of counters.
Please Suggest !!

[hkaiser]

Sure, please feel free to make things consistent.

[victor-ludorum]

Thanks, @hkaiser Sir, I will work on this and make things consistent 😊

[victor-ludorum]

Hello @hkaiser Sir!! I need one of your advice. In this arithmetic performance counter, we normally take the values from other performance counters, then perform the arithmetic operation on the sampled data collected over. We generally perform some statistical operation on it . So, what I am thinking is to implement rolling windows performance counter like rolling_mean, rolling_variance , rolling_sum and some other as per requirement in arithmetic_performance_counter as there are several rolling windows function already present in statistical performance counter too like rolling_min and rolling_max.
Again as they have same properties so I can add its implementation too in statistics performance counter too.
Please suggest !!
Thanks!!

[hkaiser]

@victor-ludorum The rolling-window statistical operations make perfect sense for the statistics counters as there we don't know upfront how many values we will have to do calculations on. OTOH, we do know how many values we have to look at for the arithmetic counters. For this reason, I think creating rolling-window counterparts for those might not be necessary. What was your rationale to propose those? Just consistency?

[victor-ludorum]

@hkaiser Sir, Actually what I have got knowledge about these rolling operation is that they may operate over the last n samples of data. For example
rolling_mean is used to calculate mean over the last n sample of data.
rolling_sum is used to calculate sum over the last n samples of data.
rolling_variance is used to calculate variance over the last sample of data.
So, I have thought that in arithmetic performance counter too we can use it to have some statistical operation over the last samples of data. I am going to add this to statistical performance counters.
But just I am assuring that whether it is useful for arithmetics performance counter or not.
Thanks!!

[hkaiser]

We know how many values we will have to evaluate for the arithmetic counters. So the rolling-window functionalities are not needed, I think.

[victor-ludorum]

Okay @hkaiser !! Now I have understood the things. Thanks 😊 !!