CreateCounter can create corruptable counter

[davhdavh]

Description

👍 This will correctly register 1 metrics with 2 different tag values for type:

Meter meter = new Meter("HatCo.Store");
Counter<int> blueHatsSold = meter.CreateCounter<int>("hatco.store.hats_sold");
Counter<int> redHatsSold = meter.CreateCounter<int>("hatco.store.hats_sold");

Console.WriteLine("Press any key to exit");
while (!Console.KeyAvailable)
{
   // Pretend our store has a transaction each second that sells 4 hats
   Thread.Sleep(1000);
   redHatsSold.Add(4, tag: new("type", "red"));
   blueHatsSold.Add(5, tag: new("type", "blue"));
}

👎 This creates 2 counter with same name, but tags specified at creation:

Meter meter = new Meter("HatCo.Store");
Counter<int> blueHatsSold = meter.CreateCounter<int>("hatco.store.hats_sold", null, null, [new("type", "blue")]);
Counter<int> redHatsSold = meter.CreateCounter<int>("hatco.store.hats_sold", null, null, [new("type", "red")]);

Console.WriteLine("Press any key to exit");
while (!Console.KeyAvailable)
{
   // Pretend our store has a transaction each second that sells 4 hats
   Thread.Sleep(1000);
   redHatsSold.Add(4);
   blueHatsSold.Add(5);
}

dotnet-counters monitor -n ConsoleApp10 --counters HatCo.Store --showDeltas shows just 1 metric label for type=blue, that consistently shows a value that matches the blue; but a delta value that is the difference between blue and red.
So somehow the same metrics is reported on top of itself with wrong tags.

Reproduction Steps

See above code examples

Expected behavior

Both code examples work the same

Actual behavior

invalid metrics reported

Regression?

No response

Known Workarounds

The first example obviously works

Configuration

8.0

Other information

No response

[KalleOlaviNiemitalo]

For .NET Runtime, I think this is a duplicate of dotnet/runtime#93767, which was fixed in dotnet/runtime#104993 before v9.0.0-preview.6.24327.7.

For dotnet-counters, TraceEventExtensions will have to be changed to use the new instrumentId payload field. AFAICT, there's no pull request for that yet. I guess it'll be tracked in the issue #4564, even though that one talks about multiple meters rather than multiple instruments in the same meter.

[ericstj]

@tarekgh can you confirm @KalleOlaviNiemitalo's analysis

[tarekgh]

Yes @KalleOlaviNiemitalo is correct. We added a support to emit a unique Id for every instrument in the runtime. Tools like dotnet-counter and dotnet-monitor or any similar tools need to benefit from that and ensure distinguishing between instruments created with the same name. We need to transfer this issue to the https://github.com/dotnet/diagnostics repo.

@noahfalk @davmason @tommcdon @brianrob I don't have write permission on the diagnostics repo to transfer the issue there, can anyone of you help transferring it there?

[noahfalk]

I guess it'll be tracked in the issue #4564, even though that one talks about multiple meters rather than multiple instruments in the same meter.

I'm happy to leave both this issue and #4564 open for tracking multiple instruments and multiple meters respectively. They technically could be fixed independently even though probably one PR is going to resolve both at the same time.