-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' of github.com:jaronoff97/blog
- Loading branch information
Showing
4 changed files
with
85 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
{"post":5,"update":18,"python":6,"code":17,"first":3,"lists":1,"for":1,"dictionary":1,"final":1,"swift":6,"closures":1,"struct":1,"class":1,"extensions":1,"optional":1,"protocol":1,"java":18,"url":1,"r":35,"second":1,"third":1,"fourth":1,"fifth":1,"ood":17,"notes":32,"ds":29,"sixth":1,"review":23,"hackny":10,"summer":9,"ar":1,"arkit":1,"firebase":1,"blog":11,"marvel":1,"transit":1,"argo":1,"kubernetes":2} | ||
{"update":19,"tails":1,"opentelemetry":1,"kubernetes":2,"hackny":10,"argo":1,"blog":11,"transit":1,"marvel":1,"swift":6,"code":17,"firebase":1,"review":23,"summer":9,"ds":29,"r":35,"ood":17,"java":18,"notes":32,"sixth":1,"fifth":1,"fourth":1,"third":1,"second":1,"first":3,"post":5,"url":1,"extensions":1,"optional":1,"protocol":1,"closures":1,"struct":1,"class":1,"ar":1,"arkit":1,"python":6,"final":1,"dictionary":1,"lists":1,"for":1} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
--- | ||
title: Announcing Tails | ||
date: 2024-04-17 | ||
description: 'A whole new post' | ||
tags: [update, tails, opentelemetry] | ||
draft: false | ||
project: tails | ||
--- | ||
|
||
# What's the problem? | ||
|
||
Say you're looking at some metrics in Grafana but the labels look all wrong. You expected to see something like `pod` but instead you're seeing `kube_pod`. Maybe the k8sattribute processor you wanted to use to enrich all of your traces isn't working. It could be a log whose body isn't being extracting correctly. | ||
|
||
These are the biggest problems I've had when using the otel collector: | ||
|
||
- resource attribute enrichment working | ||
- attribute naming conventions | ||
- is data being processed at all? | ||
|
||
# What can we do about it? | ||
|
||
The otel collector ecosystem is vast and full of possibilities. Thanks to the work of the [OpAMP-SIG](https://github.com/open-telemetry/opamp-spec/blob/main/specification.md) and [Antoine Toulme](https://github.com/atoulme), we're able to solve all of these problems! By utilizing the [Remote Tap Processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/remotetapprocessor) and the [OpAMP Extension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/opampextension) we can hook into a live collector and see what's going on in there. | ||
|
||
I call the tool "tails" because the goal is to tail a collector and find exactly what you're looking for. The UI allows you to view the live configuration of the collector (if you have the opamp extension configured). You can also connect to the collector's remote tap processor to view the live telemetry. It's run as a docker container which is released [here](https://github.com/jaronoff97/tails/pkgs/container/tails). You can run this like so in a Kubernetes cluster: | ||
|
||
Collector config: | ||
|
||
```yaml | ||
--- | ||
extensions: | ||
opamp: | ||
server: | ||
ws: | ||
endpoint: ws://127.0.0.1:4000/v1/opamp | ||
tls: | ||
insecure: true | ||
--- | ||
processors: | ||
remotetap: | ||
endpoint: localhost:12001 | ||
--- | ||
service: | ||
pipelines: | ||
traces: | ||
receivers: [otlp] | ||
processors: [remotetap, batch] | ||
exporters: [debug] | ||
|
||
metrics: | ||
receivers: [otlp] | ||
processors: [remotetap, batch] | ||
exporters: [debug] | ||
|
||
logs: | ||
receivers: [otlp] | ||
processors: [remotetap, batch] | ||
exporters: [debug] | ||
|
||
extensions: [opamp] | ||
``` | ||
Add this in either a collector CRD of helm chart: | ||
```yaml | ||
additionalContainers: | ||
- name: tails | ||
image: ghcr.io/jaronoff97/tails:v0.0.6 | ||
ports: | ||
- containerPort: 4000 | ||
env: | ||
- name: SECRET_KEY_BASE | ||
value: <add a 64 byte length secret> | ||
``` | ||
and apply the change! | ||
<Image alt="deployed kubernetes pods" src="/static/images/kube-screen.png" width={1440} height={1840} /> | ||
You'll see your pod has a new container that you can port-forward to: | ||
`k port-forward pod/<pod-name> -p 4000:4000`. Once you're port-forwarding, open up http://localhost:4000 and you'll be greeted with the tails UI. Start your remote tap, and navigate to your desired telemetry type. From here, you can filter attributes, resource attributes, and view raw data. You can view a quick demo [here](https://drive.google.com/file/d/1uiN5S4KcZqX11rIoMQp7HBZTrse3mOAS/preview): | ||
|
||
<iframe src="https://drive.google.com/file/d/1uiN5S4KcZqX11rIoMQp7HBZTrse3mOAS/preview" width="640" height="480" allow="autoplay"></iframe> | ||
|
||
Please give it a try and let me know your thoughts! Feel free to open an issue [here](https://github.com/jaronoff97/tails/issues) or find me in the CNCF Slack! |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.