Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reimplement expvar metrics to be tolerant of duplicates #40

Merged
merged 4 commits into from
Mar 5, 2018

Conversation

yurishkuro
Copy link
Member

@yurishkuro yurishkuro commented Mar 4, 2018

Required for jaegertracing/jaeger#716

Add a new module metrics/adapters that provide caching of the created metrics objects and the common functionality of managing namespaces and flattening tags into metric id string.

Yuri Shkuro added 2 commits March 4, 2018 15:28
Signed-off-by: Yuri Shkuro <ys@uber.com>
Signed-off-by: Yuri Shkuro <ys@uber.com>
@yurishkuro yurishkuro requested a review from black-adder March 4, 2018 20:37
@codecov
Copy link

codecov bot commented Mar 4, 2018

Codecov Report

Merging #40 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@          Coverage Diff           @@
##           master    #40    +/-   ##
======================================
  Coverage     100%   100%            
======================================
  Files          21     25     +4     
  Lines         595    710   +115     
======================================
+ Hits          595    710   +115
Impacted Files Coverage Δ
metrics/prometheus/cache.go 100% <ø> (ø) ⬆️
metrics/go-kit/prometheus/factory.go 100% <ø> (ø) ⬆️
metrics/go-kit/expvar/factory.go 100% <ø> (ø) ⬆️
metrics/adapters/tagless.go 100% <100%> (ø)
metrics/adapters/cache.go 100% <100%> (ø)
metrics/expvar/factory.go 100% <100%> (ø)
metrics/adapters/factory.go 100% <100%> (ø)
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 34d9cc2...c210388. Read the comment docs.

r.lock.Lock()
defer r.lock.Unlock()
r.timers[name] = t
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's times like these that make me miss overloading functions and generics/templates. I'm assuming there is no way Go would support a single get/set.

Copy link
Contributor

@isaachier isaachier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


import "github.com/uber/jaeger-lib/metrics"

// FactoryWithoutTags creates metrics with fully qualified name and tags.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment inconsistent


func (f *factory) Counter(name string, tags map[string]string) metrics.Counter {
fullName, fullTags, key := f.getKey(name, tags)
c, ok := f.cache.getCounter(key)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There maybe a race condition here. If 2 threads get here at the same time, they both will see nil counter and then both proceed to create a counter and only one is saved in the cache.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, good catch. We could do a getOrSet() passing a lambda to create if not found. Certainly for metrics related to service names we could easily see a race condition.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not do what Isaac recommended in other PRs and make the cache a lock free data structure and have the factories, etc. do the locking? This isn't that performance intensive anyway.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the cache is lock-free, then the locking has to be in the factory, but factory gets recreated every time you call factory.Namespace(), and they all share the same cache. I think making cache thread-safe gives better encapsulation in this case.

Yuri Shkuro added 2 commits March 5, 2018 16:44
Signed-off-by: Yuri Shkuro <ys@uber.com>
Signed-off-by: Yuri Shkuro <ys@uber.com>
@yurishkuro
Copy link
Member Author

@black-adder comments addressed

@yurishkuro yurishkuro merged commit 771be3f into master Mar 5, 2018
@yurishkuro yurishkuro deleted the duplicate-tolerant-expvar-metrics branch March 5, 2018 22:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants