Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SAMPLING_STORAGE_TYPE environment variable #3573

Merged
merged 4 commits into from
Mar 10, 2022

Conversation

joe-elliott
Copy link
Member

Which problem is this PR solving?

  • Adds support for a new environment variable SAMPLING_STORAGE_TYPE. This allows the user to independently configure
    their SPAN_STORAGE_TYPE and SAMPLING_STORAGE_TYPE. e.g. the user could choose to store their spans in elasticsearch while putting their adaptive sampling data in cassandra. This also paves the way for simpler to operate sampling storage types like redis or memcached.

Short description of the changes

  • Add support for SAMPLING_STORAGE_TYPE to plugin/storage/factory_config.go
  • Add support to plugin/storage/factory.go to return the correct factory or fall back to previous behavior if unspecified.
  • Add relevant testing.

Signed-off-by: Joe Elliott <number101010@gmail.com>
Signed-off-by: Joe Elliott <number101010@gmail.com>
@codecov
Copy link

codecov bot commented Mar 9, 2022

Codecov Report

Merging #3573 (5fda43c) into main (f55a1a2) will increase coverage by 0.04%.
The diff coverage is 83.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3573      +/-   ##
==========================================
+ Coverage   96.46%   96.51%   +0.04%     
==========================================
  Files         264      264              
  Lines       15429    15447      +18     
==========================================
+ Hits        14884    14909      +25     
+ Misses        461      453       -8     
- Partials       84       85       +1     
Impacted Files Coverage Δ
plugin/storage/factory.go 97.19% <80.00%> (+2.71%) ⬆️
plugin/sampling/strategystore/adaptive/factory.go 100.00% <100.00%> (ø)
plugin/storage/factory_config.go 100.00% <100.00%> (ø)
...lugin/sampling/strategystore/adaptive/processor.go 100.00% <0.00%> (+0.61%) ⬆️
cmd/collector/app/server/zipkin.go 70.73% <0.00%> (+2.43%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f55a1a2...5fda43c. Read the comment docs.

@joe-elliott joe-elliott marked this pull request as ready for review March 9, 2022 19:55
@joe-elliott joe-elliott requested a review from a team as a code owner March 9, 2022 19:55
@joe-elliott joe-elliott requested a review from albertteoh March 9, 2022 19:55
@yurishkuro
Copy link
Member

Just an observation - this will be subject to the existing limitation that a given kind of storage (e.g. cassandra) can only be configured once, i.e. it's not possible to have span and sampling storage use different instances of cassandra (or even different namespaces).

I don't think it should block this PR, but it may be worth thinking how that can be improved. For example, we have the mechanism for configuring different "named" storage, namely "archive", via flags like --es.archive.***. We could extend that mechanism to storage types, e.g. --es.dependencies.*** or --es.sampling.***.

Of course, if the new solution prevents sharing the same instance as we can today that would be a regression.

yurishkuro
yurishkuro previously approved these changes Mar 9, 2022
albertteoh
albertteoh previously approved these changes Mar 10, 2022
Copy link
Contributor

@albertteoh albertteoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Does this mean we need to add support for SAMPLING_STORAGE_TYPE in jaeger-operator similar to jaegertracing/jaeger-operator#1700?

@@ -37,15 +38,18 @@ func TestFactoryConfigFromEnv(t *testing.T) {
assert.Equal(t, cassandraStorageType, f.SpanWriterTypes[0])
assert.Equal(t, cassandraStorageType, f.SpanReaderType)
assert.Equal(t, cassandraStorageType, f.DependenciesStorageType)
assert.Equal(t, "", f.SamplingStorageType)
Copy link
Contributor

@albertteoh albertteoh Mar 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could use the more declarative assert.Empty(t, f.SamplingStorageType)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott joe-elliott dismissed stale reviews from albertteoh and yurishkuro via d73d7a8 March 10, 2022 14:24
@joe-elliott
Copy link
Member Author

LGTM. Does this mean we need to add support for SAMPLING_STORAGE_TYPE in jaeger-operator similar to jaegertracing/jaeger-operator#1700?

Oof, I don't know much about the operator. I had intended to PR the docs, but can't say I feel confident in my ability to make a well thought out operator change.

this will be subject to the existing limitation that a given kind of storage (e.g. cassandra) can only be configured once, i.e. it's not possible to have span and sampling storage use different instances of cassandra (or even different namespaces).

This is a good point. I will include a note about this in my docs PR.

@joe-elliott joe-elliott enabled auto-merge (squash) March 10, 2022 15:59
@joe-elliott
Copy link
Member Author

I'm unable to merge b/c of the Codecov failure. Can you merge? or should i try to catch the last line or so :)?

@yurishkuro yurishkuro disabled auto-merge March 10, 2022 16:38
@yurishkuro yurishkuro merged commit 4f55a70 into jaegertracing:main Mar 10, 2022
albertteoh pushed a commit to albertteoh/jaeger that referenced this pull request Jul 13, 2022
* Added SAMPLING_STORAGE_TYPE support in factory config

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Added factory support and tests

Signed-off-by: Joe Elliott <number101010@gmail.com>

* review

Signed-off-by: Joe Elliott <number101010@gmail.com>
Signed-off-by: Albert Teoh <see.kwang.teoh@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants