Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make Protobuf optional #1190

Merged
merged 19 commits into from
Nov 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ jobs:
PROTO_GENERATION: true
REQUIRE_PROTO_UP_TO_DATE: true
run: |
echo "Java version: $(java -version) in $(which java), Maven version: $(mvn -v)"
echo "JAVA_HOME: $JAVA_HOME"
mvn -v
./mvnw clean install
./mvnw javadoc:javadoc -P javadoc # just to check if javadoc is generated
21 changes: 21 additions & 0 deletions .github/workflows/native-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: GraalVM Native Tests

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

jobs:
native-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up JDK
uses: actions/setup-java@v4
with:
java-version: '21'
distribution: graalvm
cache: 'maven'
- name: Run the Maven verify phase
run: ./scripts/run-native-tests.sh
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
cache: 'maven'

- name: Build with Maven
run: mvn -B package -P release -Dmaven.test.skip=true
run: ./scripts/build-release.sh ${{ github.ref_name }}

- name: Set up Apache Maven Central
uses: actions/setup-java@v4
Expand Down
10 changes: 0 additions & 10 deletions RELEASING.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,3 @@
## Update Version

In a new PR, update the version in `pom.xml` using

```shell
mvn versions:set -DnewVersion=<VERSION>
```

Commit the changes and open a PR.

## Create a Release

1. Go to https://github.com/prometheus/client_java/releases
Expand Down
27 changes: 23 additions & 4 deletions docs/content/exporters/formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,35 @@ All exporters the following exposition formats:

Moreover, gzip encoding is supported for each of these formats.

Scraping with a Prometheus server
---------------------------------
## Scraping with a Prometheus server

The Prometheus server sends an `Accept` header to specify which format is requested. By default, the Prometheus server will scrape OpenMetrics text format with gzip encoding. If the Prometheus server is started with `--enable-feature=native-histograms`, it will scrape Prometheus protobuf format instead.

Viewing with a Web Browser
--------------------------
## Viewing with a Web Browser

If you view the `/metrics` endpoint with your Web browser you will see Prometheus text format. For quick debugging of the other formats, exporters provide a `debug` URL parameter:

* `/metrics?debug=openmetrics`: View OpenMetrics text format.
* `/metrics?debug=text`: View Prometheus text format.
* `/metrics?debug=prometheus-protobuf`: View a text representation of the Prometheus protobuf format.

## Exclude protobuf exposition format

You can exclude the protobuf exposition format by including the
`prometheus-metrics-exposition-textformats` module and excluding the
`prometheus-metrics-exposition-formats` module in your build file.

For example, in Maven:

```xml
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exporter-httpserver</artifactId>
<exclusions>
<exclusion>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exposition-formats</artifactId>
</exclusion>
</exclusions>
</dependency>
```
8 changes: 8 additions & 0 deletions docs/content/getting-started/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,14 @@ implementation 'io.prometheus:prometheus-metrics-exporter-httpserver:$version'

There are alternative exporters as well, for example if you are using a Servlet container like Tomcat or Undertow you might want to use `prometheus-exporter-servlet-jakarta` rather than a standalone HTTP server.

{{< hint type=note >}}

If you do not use the protobuf exposition format, you can
[exclude](../../exporters/formats#exclude-protobuf-exposition-format)
it from the dependencies.

{{< /hint >}}

# Dependency management

A Bill of Material
Expand Down
10 changes: 4 additions & 6 deletions docs/content/internals/model.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,25 +7,23 @@ The illustration below shows the internal architecture of the Prometheus Java cl

![Internal architecture of the Prometheus Java client library](/client_java/images/model.png)

prometheus-metrics-core
-----------------------
## prometheus-metrics-core

This is the user facing metrics library, implementing the core metric types, like [Counter](/client_java/api/io/prometheus/metrics/core/metrics/Counter.html), [Gauge](/client_java/api/io/prometheus/metrics/core/metrics/Gauge.html) [Histogram](/client_java/api/io/prometheus/metrics/core/metrics/Histogram.html), and so on.

All metric types implement the [Collector](/client_java/api/io/prometheus/metrics/model/registry/Collector.html) interface, i.e. they provide a [collect()](/client_java/api/io/prometheus/metrics/model/registry/Collector.html#collect()) method to produce snapshots.

prometheus-metrics-model
------------------------
## prometheus-metrics-model

The model is an internal library, implementing read-only immutable snapshots. These snapshots are returned by the [Collector.collect()](/client_java/api/io/prometheus/metrics/model/registry/Collector.html#collect()) method.

There is no need for users to use `prometheus-metrics-model` directly. Users should use the API provided by `prometheus-metrics-core`, which includes the core metrics as well as callback metrics.

However, maintainers of 3rd party metrics libraries might want to use `prometheus-metrics-model` if they want to add Prometheus exposition formats to their metrics library.

exporters and exposition formats
--------------------------------
## Exporters and exposition formats

The `prometheus-metrics-exposition-formats` module converts snapshots to Prometheus exposition formats, like text format, OpenMetrics text format, or Prometheus protobuf format.

The exporters like `prometheus-metrics-exporter-httpserver` or `prometheus-metrics-exporter-servlet-jakarta` use this to convert snapshots into the right format depending on the `Accept` header in the scrape request.

12 changes: 10 additions & 2 deletions integration-tests/it-common/pom.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
Expand All @@ -11,11 +12,18 @@
<artifactId>it-common</artifactId>

<name>Integration Tests - Common Utilities</name>
<url>http://github.com/prometheus/client_java</url>
<description>
Common utilities for integration tests
</description>

<dependencies>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exposition-formats</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>

<build>
<testResources>
<testResource>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
package io.prometheus.client.it.common;

import static java.nio.charset.StandardCharsets.UTF_8;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.fail;

import io.prometheus.metrics.expositionformats.generated.com_google_protobuf_4_28_3.Metrics;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URISyntaxException;
import java.net.URL;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import java.util.zip.GZIPInputStream;
import org.apache.commons.io.IOUtils;
import org.junit.jupiter.api.AfterEach;
import org.testcontainers.containers.BindMode;
import org.testcontainers.containers.GenericContainer;

public abstract class ExporterTest {
private final GenericContainer<?> sampleAppContainer;
private final Volume sampleAppVolume;
protected final String sampleApp;

public ExporterTest(String sampleApp) throws IOException, URISyntaxException {
this.sampleApp = sampleApp;
this.sampleAppVolume =
Volume.create("it-exporter")
.copy("../../it-" + sampleApp + "/target/" + sampleApp + ".jar");
this.sampleAppContainer =
new GenericContainer<>("openjdk:17")
.withFileSystemBind(sampleAppVolume.getHostPath(), "/app", BindMode.READ_ONLY)
.withWorkingDirectory("/app")
.withLogConsumer(LogConsumer.withPrefix(sampleApp))
.withExposedPorts(9400);
}

// @BeforeEach?
protected void start() {
start("success");
}

protected void start(String outcome) {
sampleAppContainer
.withCommand("java", "-jar", "/app/" + sampleApp + ".jar", "9400", outcome)
.start();
}

@AfterEach
public void tearDown() throws IOException {
sampleAppContainer.stop();
sampleAppVolume.remove();
}

public static void assertContentType(String expected, String actual) {
if (!expected.replace(" ", "").equals(actual)) {
assertThat(actual).isEqualTo(expected);
}
}

protected Response scrape(String method, String queryString, String... requestHeaders)
throws IOException {
return scrape(
method,
new URL(
"http://localhost:"
+ sampleAppContainer.getMappedPort(9400)
+ "/metrics?"
+ queryString),
requestHeaders);
}

public static Response scrape(String method, URL url, String... requestHeaders)
throws IOException {
long timeoutMillis = TimeUnit.SECONDS.toMillis(5);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod(method);
for (int i = 0; i < requestHeaders.length; i += 2) {
con.setRequestProperty(requestHeaders[i], requestHeaders[i + 1]);
}
long start = System.currentTimeMillis();
Exception exception = null;
while (System.currentTimeMillis() - start < timeoutMillis) {
try {
if (con.getResponseCode() == 200) {
return new Response(
con.getResponseCode(),
con.getHeaderFields(),
IOUtils.toByteArray(con.getInputStream()));
} else {
return new Response(
con.getResponseCode(),
con.getHeaderFields(),
IOUtils.toByteArray(con.getErrorStream()));
}
} catch (Exception e) {
exception = e;
try {
Thread.sleep(100);
} catch (InterruptedException ignored) {
// ignore
}
}
}
if (exception != null) {
exception.printStackTrace();
}
fail("timeout while getting metrics from " + url);
return null; // will not happen
}

public static class Response {
public final int status;
private final Map<String, String> headers;
public final byte[] body;

private Response(int status, Map<String, List<String>> headers, byte[] body) {
this.status = status;
this.headers = new HashMap<>(headers.size());
this.body = body;
for (Map.Entry<String, List<String>> entry : headers.entrySet()) {
if (entry.getKey()
!= null) { // HttpUrlConnection uses pseudo key "null" for the status line
this.headers.put(entry.getKey().toLowerCase(Locale.ROOT), entry.getValue().get(0));
}
}
}

public String getHeader(String name) {
// HTTP headers are case-insensitive
return headers.get(name.toLowerCase(Locale.ROOT));
}

public String stringBody() {
return new String(body, UTF_8);
}

public String gzipBody() throws IOException {
return new String(
IOUtils.toByteArray(new GZIPInputStream(new ByteArrayInputStream(body))), UTF_8);
}

public List<Metrics.MetricFamily> protoBody() throws IOException {
List<Metrics.MetricFamily> metrics = new ArrayList<>();
InputStream in = new ByteArrayInputStream(body);
while (in.available() > 0) {
metrics.add(Metrics.MetricFamily.parseDelimitedFrom(in));
}
return metrics;
}
}
}
69 changes: 69 additions & 0 deletions integration-tests/it-exporter/it-exporter-no-protobuf/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.prometheus</groupId>
<artifactId>it-exporter</artifactId>
<version>1.3.3</version>
</parent>

<artifactId>it-exporter-no-protobuf</artifactId>

<name>Integration Tests - HTTPServer Exporter Sample - no protobuf</name>
<description>
HTTPServer Sample for the Exporter Integration Test without protobuf
</description>

<dependencies>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exporter-httpserver</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exposition-formats</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-core</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<groupId>io.prometheus</groupId>
<artifactId>prometheus-metrics-exposition-formats</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>

<build>
<finalName>exporter-no-protobuf</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>io.prometheus.metrics.it.exporter.httpserver.HTTPServerSample</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Loading