Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Registry manifest and Schema diff #400

Draft
wants to merge 35 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
edab834
feat(semconv): create registry manifest
lquerel Oct 3, 2024
fe7d7b5
feat(schema): Parse OTEL schema 1.1
lquerel Oct 4, 2024
76db2aa
feat(cli): Add registry diff command (WIP)
lquerel Oct 4, 2024
ad77910
Merge branch 'main' into generate-otel-schema
lquerel Oct 4, 2024
6de07ee
feat(schema): Support new deprecated field format
lquerel Oct 17, 2024
5efe6b1
Merge remote-tracking branch 'upstream/main' into generate-otel-schema
lquerel Oct 23, 2024
6cdfbfa
feat(diff): Add registry diff command.
lquerel Oct 23, 2024
880649f
feat(diff): Add diff for metrics
lquerel Oct 24, 2024
a87807b
feat(diff): Add diff for events, spans, and resources
lquerel Oct 24, 2024
8f8bdaf
feat(diff): Add template rendering capability
lquerel Oct 25, 2024
a818f4e
feat(diff): Add diff-format for ansi, ansi-stats, json, and yaml
lquerel Oct 28, 2024
05256ea
feat(diff): Add diff-format for markdown
lquerel Oct 29, 2024
c584f30
chore(diff): Fix unit tests
lquerel Oct 29, 2024
374def2
feat(cli): Prepare registry update-schema command
lquerel Oct 30, 2024
1b7c9c2
Merge branch 'main' into generate-otel-schema
lquerel Oct 30, 2024
2abd024
feat(cli): Prepare registry update-schema command
lquerel Oct 30, 2024
83f45eb
feat(cli): Prepare registry update-schema command
lquerel Oct 31, 2024
6312e42
chore(typo): fix typo in comment
lquerel Dec 2, 2024
6532bf3
Merge remote-tracking branch 'upstream/main' into generate-otel-schema
lquerel Dec 2, 2024
f5efacc
chore(build): Resolve merging with main upstream
lquerel Dec 2, 2024
9bda8cb
chore(build): Clean up code to prepare PR
lquerel Dec 2, 2024
edf85b0
chore(build): Clean up code to prepare PR
lquerel Dec 2, 2024
3b41c58
Merge branch 'main' into generate-otel-schema
lquerel Dec 6, 2024
bde3eef
chore(build): Prepare resolve_telemetry_schema to return WResult
lquerel Dec 6, 2024
904e856
Merge branch 'generate-otel-schema' of https://github.com/lquerel/ote…
lquerel Dec 6, 2024
3c4b6dd
chore(build): Fix `registry diff` to return a diff even when there ar…
lquerel Dec 9, 2024
c0609b5
Merge branch 'open-telemetry:main' into generate-otel-schema
lquerel Dec 17, 2024
6ba427c
chore(merge): Merge with main branch
lquerel Dec 17, 2024
ebcd15f
feat(diff): Test schema diff feature
lquerel Dec 19, 2024
8dcd311
feat(diff): Remove complex logic infering the type of deprecation bas…
lquerel Dec 19, 2024
b58d200
feat(diff): Rename the field Deprecated::Renamed::new_name into Depre…
lquerel Dec 19, 2024
a1ab93a
chore(doc): Fix documentation of the ResolvedTelemetrySchema::groups …
lquerel Dec 19, 2024
396450f
chore(doc): Fix documentation based on @jsuereth feedback
lquerel Dec 19, 2024
12c79c2
Merge remote-tracking branch 'upstream/main' into generate-otel-schema
lquerel Dec 19, 2024
0a63cd8
chore(merge): Merge with upstream main branch
lquerel Dec 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions crates/weaver_semconv/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,14 @@
use crate::Error::CompoundError;
use miette::Diagnostic;
use serde::Serialize;
use std::path::PathBuf;
use weaver_common::diagnostic::{DiagnosticMessage, DiagnosticMessages};
use weaver_common::error::{format_errors, WeaverError};

pub mod any_value;
pub mod attribute;
pub mod group;
mod manifest;
pub mod metric;
pub mod registry;
pub mod semconv;
Expand Down Expand Up @@ -143,6 +145,24 @@ pub enum Error {
error: String,
},

/// This error is raised when a registry manifest is not found.
#[error("The registry manifest at {path:?} is not found.")]
#[diagnostic(severity(Error))]
RegistryManifestNotFound {
/// The path to the registry manifest file.
path: PathBuf,
},

/// This error is raised when a registry manifest is invalid.
#[error("The registry manifest at {path:?} is invalid. {error}")]
#[diagnostic(severity(Error))]
InvalidRegistryManifest {
/// The path to the registry manifest file.
path: PathBuf,
/// The error that occurred.
error: String,
},

/// A container for multiple errors.
#[error("{:?}", format_errors(.0))]
CompoundError(#[related] Vec<Error>),
Expand Down
160 changes: 160 additions & 0 deletions crates/weaver_semconv/src/manifest.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
// SPDX-License-Identifier: Apache-2.0

//! Contains the definitions for the semantic conventions registry manifest.
//!
//! This struct is used to specify the registry, including its name, version,
//! description, and few other details.
//!
//! In the future, this struct may be extended to include additional information
//! such as the registry's owner, maintainers, and dependencies.

use crate::Error;
use crate::Error::{InvalidRegistryManifest, RegistryManifestNotFound};
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
use weaver_common::error::handle_errors;

/// Represents the information of a semantic convention registry manifest.
///
/// This information defines the registry's name, version, description, and schema
/// base url.
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct RegistryManifest {
/// The name of the registry. This name is used to define the package name.
pub name: String,

/// An optional description of the registry.
///
/// This field can be used to provide additional context or information about the registry's
/// purpose and contents.
/// The format of the description is markdown.
pub description: Option<String>,

/// The version of the registry which will be used to define the semconv package version.
pub semconv_version: String,

/// The base URL where the registry's schema files are hosted.
pub schema_base_url: String,
}

impl RegistryManifest {
Fixed Show fixed Hide fixed
Fixed Show fixed Hide fixed
/// Attempts to load a registry manifest from a file.
///
/// The expected file format is YAML.
pub fn try_from_file<P: AsRef<std::path::Path>>(path: P) -> Result<Self, Error> {
Fixed Show fixed Hide fixed
Fixed Show fixed Hide fixed
let manifest_path_buf = path.as_ref().to_path_buf();

if !manifest_path_buf.exists() {
return Err(RegistryManifestNotFound {
path: manifest_path_buf.clone(),
});
}

let file = std::fs::File::open(path).map_err(|e| InvalidRegistryManifest {
path: manifest_path_buf.clone(),
error: e.to_string(),
})?;
let reader = std::io::BufReader::new(file);
let manifest: RegistryManifest =
serde_yaml::from_reader(reader).map_err(|e| InvalidRegistryManifest {
path: manifest_path_buf.clone(),
error: e.to_string(),
})?;

manifest.validate(manifest_path_buf.clone())?;

Ok(manifest)
}

fn validate(&self, path: PathBuf) -> Result<(), Error> {
Fixed Show fixed Hide fixed
Fixed Show fixed Hide fixed
let mut errors = vec![];

if self.name.is_empty() {
errors.push(InvalidRegistryManifest {
path: path.clone(),
error: "The registry name is required.".to_owned(),
});
}

if self.semconv_version.is_empty() {
errors.push(InvalidRegistryManifest {
path: path.clone(),
error: "The registry version is required.".to_owned(),
});
}

if self.schema_base_url.is_empty() {
errors.push(InvalidRegistryManifest {
path: path.clone(),
error: "The registry schema base URL is required.".to_owned(),
});
}

handle_errors(errors)?;

Ok(())
}
}

#[cfg(test)]
mod tests {
use super::*;
use crate::Error::CompoundError;

#[test]
fn test_not_found_registry_info() {
let result = RegistryManifest::try_from_file("tests/test_data/missing_registry.yaml");
assert!(
matches!(result, Err(RegistryManifestNotFound { path, .. }) if path.ends_with("missing_registry.yaml"))
);
}

#[test]
fn test_incomplete_registry_info() {
let result = RegistryManifest::try_from_file(
"tests/test_data/incomplete_semconv_registry_manifest.yaml",
);
assert!(
matches!(result, Err(InvalidRegistryManifest { path, .. }) if path.ends_with("incomplete_semconv_registry_manifest.yaml"))
);
}

#[test]
fn test_valid_registry_info() {
let config =
RegistryManifest::try_from_file("tests/test_data/valid_semconv_registry_manifest.yaml")
.expect("Failed to load the registry configuration file.");
assert_eq!(config.name, "vendor_acme");
assert_eq!(config.semconv_version, "0.1.0");
assert_eq!(config.schema_base_url, "https://acme.com/schemas/");
}

#[test]
fn test_invalid_registry_info() {
let result = RegistryManifest::try_from_file(
"tests/test_data/invalid_semconv_registry_manifest.yaml",
);
let path = PathBuf::from("tests/test_data/invalid_semconv_registry_manifest.yaml");

let expected_errs = CompoundError(vec![
InvalidRegistryManifest {
path: path.clone(),
error: "The registry name is required.".to_owned(),
},
InvalidRegistryManifest {
path: path.clone(),
error: "The registry version is required.".to_owned(),
},
InvalidRegistryManifest {
path: path.clone(),
error: "The registry schema base URL is required.".to_owned(),
},
]);

if let Err(observed_errs) = result {
assert_eq!(observed_errs, expected_errs);
} else {
panic!("Expected an error, but got a result.");
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# This an invalid semconv registry manifest file because the version and schema base url fields are missing.
name: vendor_acme
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
name: ""
description: This registry contains the semantic conventions for the Acme vendor.
semconv_version: ""
schema_base_url: ""
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
name: vendor_acme
description: This registry contains the semantic conventions for the Acme vendor.
semconv_version: 0.1.0
schema_base_url: https://acme.com/schemas/
Loading