Skip to content

Latest commit

 

History

History
122 lines (96 loc) · 5.79 KB

downstream-prometheus.md

File metadata and controls

122 lines (96 loc) · 5.79 KB
Raw

grafana/prometheus Maintenance

Grafana Labs includes the Agent as part of their internal monitoring, running it alongside Prometheus. This gives an opportunity to utilize the Agent to proof-of-concept additions to Prometheus before they get moved upstream. A grafana/prometheus repository maintained by Grafana Labs holds non-trivial and experimental changes. Having this repository allows for experimental features to be vendored into the Agent and enables faster development iteration. Ideally, this experimental testing can help serve as evidence towards usefulness and correctness when the feature becomes proposed upstream.

We are committing ourselves to doing the following:

  1. Keep changes mergeable upstream: we want to continue to be good OSS citizens, and we intend that all features we add to our Prometheus repository will become an upstream PR. We will maintain our repository in a way that supports doing this.
  2. Always vendor a branch from grafana/prometheus based off of a recent Prometheus stable release; we want the Agent's Prometheus roots to be stable.
  3. Reduce code drift: The code the Agent uses on top of Prometheus will be layered on top of a Prometheus release rather than sandwiched in between.
  4. Keep the number of experimental changes not merged upstream to a minimum. We're not trying to fork Prometheus.

Maintenance of the grafana/prometheus repository revolves around feature branches (named feat-SOME-FEATURE) and release branches (named release-vX.Y.Z-grafana). The release branches will always use the same release version as the prometheus/prometheus release it is based off of.

By adding features to the grafana/prometheus repository first, we are committing ourselves to extra maintenance of features that have not yet been merged upstream. Feature authors will have to babysit their features to coordinate with the Prometheus release schedule to always be compatible. Maintenance burden becomes lightened once each feature is upstreamed as breaking changes will no longer happen out of sync with upstream changes for the respective upstreamed feature.

We are purposefully carrying this extra burden because we intend to ultimately make Prometheus better and contribute all of our enhancements upstream. We want to strive to benefit the Prometheus ecosystem at large.

Creating a New Feature

Grafana Labs developers should try to get all features upstreamed first. If it's clear the feature is experimental or more unproven than the upstream team is comfortable with, developers should then create a downstream grafana/prometheus feature branch.

For grafana/prometheus maintainers to create a new feature, they will do the following:

  1. Create a feature branch in grafana/prometheus based on the latest release tag that grafana/prometheus currently has a release branch for. The feature branch should follow the naming convention feat-<feature name>.
  2. Implement the feature and open a PR to merge the feature branch into the associated grafana/prometheus release branch.
  3. After updating the release branch, open a PR to update grafana/agent to use the latest release branch SHA.

Updating an Existing Feature

If a feature branch that was already merged to a release branch needs to be updated for any reason:

  1. Push directly to the feature branch or open a PR to merge changes into that feature branch.
  2. Open a PR to merge the new changes from the feature branch into the associated release branch.
  3. After updating the release branch, open a PR to update grafana/agent by vendoring the changes using the latest release branch SHA.

Handling New Upstream Release

When a new upstream prometheus/prometheus release is available, we must go through the following process:

  1. Create a new grafana/prometheus release branch named release-X.Y.Z-grafana.
  2. For all feature branches still not merged upstream, rebase them on top of the newly created branch. Force push them to update the grafana/prometheus feature branch.
  3. Create one or more PRs to introduce the features into the newly created release branch.

Once a new release branch has been created, the previous release branch in grafana/prometheus is considered stale and will no longer receive updates.

Updating the Agent's vendor

The easiest way to do this is the following:

  1. Edit go.mod and change the replace directive to the release branch name.
  2. Update README.md in the Agent to change which version of Prometheus the Agent is vendoring.
  3. Run go mod tidy -compat=1.17 && go mod vendor.
  4. Commit and open a PR.

Gotchas

If the grafana/prometheus feature is incompatible with the upstream prometheus/prometheus master branch, merge conflicts would prevent an upstream PR from being merged. There are a few ways this can be handled at the feature author's discretion:

When this happens, downstream feature branch maintainers should wait until a new prometheus/prometheus release is available and rebase their feature branch on top of the latest release. This will make the upstream PR compatible with the master branch, though the window of compatibility is unpredictable and may change at any time.

If it proves unfeasible to get a feature branch merged upstream within the "window of upstream compatibility," feature branch maintainers should create a fork of their branch that is based off of master and use that master-compatible branch for the upstream PR. Note that this means any changes made to the feature branch will now have to be mirrored to the master-compatible branch.

Open Questions

If two feature branches depend on one another, a combined feature branch (like an "epic" branch) should be created where development of interrelated features go. All features within this category go directly to the combined "epic" branch rather than individual branches.