-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move Collector architecture documentation and diagrams from Github to opentelemetry.io #4029
Move Collector architecture documentation and diagrams from Github to opentelemetry.io #4029
Conversation
|
For details of config file format see | ||
[this document](https://docs.google.com/document/d/1NeheFG7DmcUYo_h2vLtNRlia9x5wOJMlV4QKEK05FhQ/edit#). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this in a google doc and not in a markdown file in the collector repo? I guess this has historical reasons, maybe we can migrate it as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure about the document's origin, but I'm happy to convert it to Markdown and move it to the Collector repo if that's the consensus.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question where this should live, this is up to the @open-telemetry/collector-approvers
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd remove it altogether. It's so old that a few things are already wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed in 2740396. Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a really great start, we might need to do some passes to make this content up-to-date.
@tiffany76 no need to keep this as a draft from my point of view! This will also trigger some review requests with @open-telemetry/collector-approvers who should definitely provide an early review. |
Co-authored-by: Severin Neumann <severin.neumann@altmuehlnet.de>
traces: # a pipeline of “traces” type | ||
receivers: [otlp] | ||
processors: [memory_limiter, batch] | ||
exporters: [jaeger] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I count 11 more mentions of jaeger in the document, including several in diagrams (two of which I'm still working on).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do not need to remove all of them, only the instances where the jaeger exporter is used since it has been deprecated a while ago already, the receivers are still fine, but we still could consider using OTLP there. Jaeger can remain if named as a potential backend (which now supports OTLP)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed mentions of jaeger exporter to otlp in 8e12f26.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work, thank you! Just some suggestions and nits.
Co-authored-by: Fabrizio Ferri-Benedetti <fferribenedetti@splunk.com>
/fix:format |
You triggered fix:format action run at https://github.com/open-telemetry/opentelemetry.io/actions/runs/8086355257 |
Is there a preview on how this will look on the opentelemetry.io website? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't finish reviewing yet, but here are a few early comments.
For details of config file format see | ||
[this document](https://docs.google.com/document/d/1NeheFG7DmcUYo_h2vLtNRlia9x5wOJMlV4QKEK05FhQ/edit#). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd remove it altogether. It's so old that a few things are already wrong.
|
||
> The configuration uses composite key names in the form of `type[/name]` as | ||
> defined in | ||
> [this document](https://docs.google.com/document/d/1NeheFG7DmcUYo_h2vLtNRlia9x5wOJMlV4QKEK05FhQ/edit#)). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, I'd remove mentions to this file. While it has parts that clarify this particular trait, other parts of the doc are outdated and might confuse users.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've removed mention of the file (2740396) but left the config explanation.
flowchart LR | ||
R1("`#quot;opentelemetry-collector#quot; Receiver`") --> FO((fan-out)) | ||
FO -->|Pipeline 'traces'| P1["`#quot;memory_limiter#quot; Processor`"] | ||
FO -->|Pipeline 'traces/2'| P2["`#quot;tags#quot; Processor`"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's this tags processor?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The 'tags' processor is mentioned in two diagrams from the original document: the receiver diagram and the exporter diagram.
I've recreated them as Mermaid diagrams, so we can make changes easily if the 'tags' processor is no longer valid or relevant.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't remember about that processor, and I would expect this diagram to be related to the config that was shared in the previous example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can say 'transform' instead of tags here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The collector config coming right before this would need to be updated as well. I believe this diagram here is a representation of that config.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated configs and diagrams to replace tags
with transform
: ff7579f.
Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de>
I'm missing "Connectors" here, and I think @djaglowski's talk at KubeCon Amsterdam was great in that area, where he not only mentioned how connectors work, but how they differ from the collector on seemingly identical features (same receiver on two pipelines vs. forward connector, I believe). It might be good to incorporate that, or at least link the presentation there. |
I wonder if we should do this in a follow up PR?, so step 1 is to bring over that content from the collector repo to docs, step2 is removing that file from the collector repo, step3 is some additional scrubbing of this file (incl. adding some words about connectors) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree with Severin that we can move forward with this PR and leave connectors to a follow up PR.
flowchart LR | ||
R1("`#quot;opentelemetry-collector#quot; Receiver`") --> FO((fan-out)) | ||
FO -->|Pipeline 'traces'| P1["`#quot;memory_limiter#quot; Processor`"] | ||
FO -->|Pipeline 'traces/2'| P2["`#quot;tags#quot; Processor`"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can say 'transform' instead of tags here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am happy with this as is, we will need to do some follow up work on it, but I prefer merging this in and then incrementally improve.
Fixes #3547.
This PR creates a new docs page (Architecture) in the Collector documentation on opentelemetry.io. The content is being moved from Github (
docs/design.md
) to the OTel website to supplement existing documentation.The file has been copyedited and modified to fit website heading conventions.
The diagram image files have been replaced in favor of Mermaid diagrams.
(Note: Work to remove outdated content and merge what remains with existing opentelemetry.io pages will be handled in a future issue/PR.)