Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Updated Fluent Bit docs to use official plugin + Sandbox #14004

Open
wants to merge 34 commits into
base: main
Choose a base branch
from

Conversation

Jayclifford345
Copy link
Contributor

@Jayclifford345 Jayclifford345 commented Aug 29, 2024

We now recommend using the official Fluent Bit plugin since the Fluent Bit community actively maintains it and is now more feature-rich than the community plugin. The update:

  • Moved community plugin to its own page for users still using the community plugin
  • Updated _index to give examples using the official fluent bit plugin
  • Added interactive sandbox

Sandbox preview for testing: https://killercoda.com/grafana-dev-testing/course/loki/fluentbit-loki-tutorial

Note link to the sandbox will fail in docs will fail with a 404 since we have not pushed the sandbox to production

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

@Jayclifford345 Jayclifford345 requested a review from a team as a code owner August 29, 2024 14:43
@github-actions github-actions bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Aug 29, 2024
Copy link
Contributor

@JStickler JStickler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[docs team] I'm only partway through this review, but I'm already hopelessly confused as to which plugin we're recommending. The naming conventions don't seem to be consistent between the files. I'm unclear on the difference between "official" and "community" or where you obtain either plugin. And if we're recommending one plugin over the other, why are we documenting both?
If there is some reason to document both, then we should turn the `_index" page into a landing page that briefly explains the difference between the two, and have the two plugin pages and the tutorial as child pages.

docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/community-plugin.md Outdated Show resolved Hide resolved
@Jayclifford345
Copy link
Contributor Author

Hi @JStickler, I think the PR is ready for another review. I added the index page we discussed in our meeting and did a bit of rewording around how we position each of the plugins. Let me know what you think!

Copy link
Contributor

@JStickler JStickler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[docs team]

docs/sources/send-data/fluentbit/community-plugin.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/community-plugin.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/community-plugin.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/community-plugin.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/community-plugin.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/fluent-bit-plugin.md Outdated Show resolved Hide resolved
Jayclifford345 and others added 8 commits October 1, 2024 16:01
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
@Jayclifford345
Copy link
Contributor Author

Hi @JStickler that is the changes in and the sandbox updated: https://killercoda.com/grafana-dev-testing/course/loki/fluentbit-loki-tutorial

Thanks again for reviewing

Copy link
Contributor

@JStickler JStickler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[docs team] Let me know when this is ready for another review.

docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/community-plugin.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/community-plugin.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/community-plugin.md Outdated Show resolved Hide resolved
docs/sources/send-data/fluentbit/fluent-bit-plugin.md Outdated Show resolved Hide resolved
Jayclifford345 and others added 10 commits October 7, 2024 09:46
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
@Jayclifford345
Copy link
Contributor Author

Hi @JStickler! Thanks for the catch on the config. That should be the config completely blank and the changes made: https://killercoda.com/grafana-dev-testing/course/loki/fluentbit-loki-tutorial

@@ -0,0 +1,281 @@
---
title: Fluent Bit Community Plugin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
title: Fluent Bit Community Plugin
title: Fluent Bit community plugin

- ../clients/fluentbit/
weight: 500
---
# Fluent Bit Community Plugin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Fluent Bit Community Plugin
# Fluent Bit community plugin


{{< /admonition >}}

The Fluent Bit Community plugin by Grafana Labs (`grafana-loki`) provided an alternative way to send logs to Loki. Although very similar to the [official plugin](https://grafana.com/docs/loki/<LOKI_VERSION>/send-data/fluentbit/fluent-bit-plugin/) there are some differences in the configuration options. This page provides instructions for how to install, configure, and use the Fluent Bit Community plugin to send logs to Loki. Although the plugin is no longer actively maintained, this documentation is still available for reference.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The Fluent Bit Community plugin by Grafana Labs (`grafana-loki`) provided an alternative way to send logs to Loki. Although very similar to the [official plugin](https://grafana.com/docs/loki/<LOKI_VERSION>/send-data/fluentbit/fluent-bit-plugin/) there are some differences in the configuration options. This page provides instructions for how to install, configure, and use the Fluent Bit Community plugin to send logs to Loki. Although the plugin is no longer actively maintained, this documentation is still available for reference.
The Fluent Bit community plugin by Grafana Labs (`grafana-loki`) provided an alternative way to send logs to Loki. Although very similar to the [official plugin](https://grafana.com/docs/loki/<LOKI_VERSION>/send-data/fluentbit/fluent-bit-plugin/) there are some differences in the configuration options. This page provides instructions for how to install, configure, and use the Fluent Bit community plugin to send logs to Loki. Although the plugin is no longer actively maintained, this documentation is still available for reference.


### Labels

Labels are used to [query logs](https://grafana.com/docs/loki/<LOKI_VERSION>/query/) `{container_name="nginx", cluster="us-west1"}`, they are usually metadata about the workload producing the log stream (`instance`, `container_name`, `region`, `cluster`, `level`). In Loki labels are indexed, so you should be cautious when choosing them. High cardinality label values can have drastic impact on performance.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Labels are used to [query logs](https://grafana.com/docs/loki/<LOKI_VERSION>/query/) `{container_name="nginx", cluster="us-west1"}`, they are usually metadata about the workload producing the log stream (`instance`, `container_name`, `region`, `cluster`, `level`). In Loki labels are indexed, so you should be cautious when choosing them. High cardinality label values can have drastic impact on performance.
Labels, for example `{container_name="nginx", cluster="us-west1"}`, are used to [query logs](https://grafana.com/docs/loki/<LOKI_VERSION>/query/). Labels are usually metadata about the workload producing the log stream (`instance`, `container_name`, `region`, `cluster`, `level`). In Loki labels are indexed, so you should be cautious when choosing them. High cardinality label values can have drastic impact on query performance.

}
```

and a LabelMap file as follow :
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
and a LabelMap file as follow :
and a LabelMap file as follows :

Comment on lines +121 to +123
```bash
helm repo add fluent https://fluent.github.io/helm-charts
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
```bash
helm repo add fluent https://fluent.github.io/helm-charts
```
```bash
helm repo add fluent https://fluent.github.io/helm-charts

Comment on lines +126 to +139
```yaml
config:
outputs: |
[OUTPUT]
Name loki
Match *
Host YourHost.Company.net
port 443
tls on
tls.verify on
http_user XXX
http_passwd XXX
Labels agent=fluend-bit
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
```yaml
config:
outputs: |
[OUTPUT]
Name loki
Match *
Host YourHost.Company.net
port 443
tls on
tls.verify on
http_user XXX
http_passwd XXX
Labels agent=fluend-bit
```
```yaml
config:
outputs: |
[OUTPUT]
Name loki
Match *
Host YourHost.Company.net
port 443
tls on
tls.verify on
http_user XXX
http_passwd XXX
Labels agent=fluend-bit

Labels agent=fluend-bit
```

Note we are only updating the `outputs` section of the Fluent Bit configuration. This is to replace the default output plugin with the Loki output plugin. If you need to update other parts of the Fluent Bit configuration refer to the [Fluent Bit values file reference](https://github.com/fluent/helm-charts/blob/main/charts/fluent-bit/values.yaml).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Note we are only updating the `outputs` section of the Fluent Bit configuration. This is to replace the default output plugin with the Loki output plugin. If you need to update other parts of the Fluent Bit configuration refer to the [Fluent Bit values file reference](https://github.com/fluent/helm-charts/blob/main/charts/fluent-bit/values.yaml).
Note we are only updating the `outputs` section of the Fluent Bit configuration. This is to replace the default output plugin with the Loki output plugin. If you need to update other parts of the Fluent Bit configuration refer to the [Fluent Bit values file reference](https://github.com/fluent/helm-charts/blob/main/charts/fluent-bit/values.yaml).

Comment on lines +145 to +147
```bash
helm install fluent-bit fluent/fluent-bit -f values.yaml
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
```bash
helm install fluent-bit fluent/fluent-bit -f values.yaml
```
```bash
helm install fluent-bit fluent/fluent-bit -f values.yaml

helm install fluent-bit fluent/fluent-bit -f values.yaml
```

## Where next?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Where next?
## Next steps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/XL type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants