Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(config, docs): Replace 'vector.toml' with 'vector.yaml' #18388

Merged
merged 11 commits into from
Sep 18, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,10 @@ debug = true
name = "vector"
section = "admin"
maintainer-scripts = "distribution/debian/scripts/"
conf-files = ["/etc/vector/vector.yaml", "/etc/default/vector"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll want to avoid adding this line as it triggers a cargo-deb bug and is unnecessary for files it /etc. See #18455

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, removed this line.

assets = [
["target/release/vector", "/usr/bin/", "755"],
["config/vector.toml", "/etc/vector/vector.toml", "644"],
["config/vector.yaml", "/etc/vector/vector.yaml", "644"],
["config/examples/*", "/etc/vector/examples/", "644"],
["distribution/systemd/vector.service", "/lib/systemd/system/vector.service", "644"],
["distribution/systemd/vector.default", "/etc/default/vector", "600"],
Expand Down
47 changes: 47 additions & 0 deletions config/vector.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# __ __ __
# \ \ / / / /
# \ V / / /
# \_/ \/
#
# V E C T O R
# Configuration
#
# ------------------------------------------------------------------------------
# Website: https://vector.dev
# Docs: https://vector.dev/docs
# Chat: https://chat.vector.dev
# ------------------------------------------------------------------------------

# Change this to use a non-default directory for Vector data storage:
# data_dir = "/var/lib/vector"

# Random Syslog-formatted logs
sources:
dummy_logs:
type: "demo_logs"
format: "syslog"
interval: 1

# Parse Syslog logs
# See the Vector Remap Language reference for more info: https://vrl.dev
transforms:
parse_logs:
type: "remap"
inputs: ["dummy_logs"]
source: |
. = parse_syslog!(string!(.message))

# Print parsed logs to stdout
sinks:
print:
type: "console"
inputs: ["parse_logs"]
encoding:
codec: "json"

# Vector's GraphQL API (disabled by default)
# Uncomment to try it out with the `vector top` command or
# in your browser at http://localhost:8686
# api:
# enabled: true
# address: "127.0.0.1:8686"
4 changes: 2 additions & 2 deletions distribution/docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ observability data with Vector.
As shown above, you can pass a custom
[Vector configuration file][docs.setup.configuration] via the `-c` flag. You'll want
to do this since the
[default `/etc/vector/vector.toml` configuration file][urls.default_configuration]
[default `/etc/vector/vector.yaml` configuration file][urls.default_configuration]
doesn't do anything.

## Deploying
Expand Down Expand Up @@ -128,7 +128,7 @@ Vector's Docker source files are located
[docs.transforms]: https://vector.dev/docs/reference/transforms/
[pages.index#correctness]: https://vector.dev/#correctness
[pages.index#performance]: https://vector.dev/#performance
[urls.default_configuration]: https://github.com/vectordotdev/vector/blob/master/config/vector.toml
[urls.default_configuration]: https://github.com/vectordotdev/vector/blob/master/config/vector.yaml
[urls.docker_alpine]: https://hub.docker.com/_/alpine
[urls.docker_debian]: https://hub.docker.com/_/debian
[urls.rust]: https://www.rust-lang.org/
Expand Down
2 changes: 1 addition & 1 deletion distribution/docker/alpine/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ FROM docker.io/alpine:3.18
RUN apk --no-cache add ca-certificates tzdata

COPY --from=builder /vector/bin/* /usr/local/bin/
COPY --from=builder /vector/config/vector.toml /etc/vector/vector.toml
COPY --from=builder /vector/config/vector.yaml /etc/vector/vector.yaml
COPY --from=builder /var/lib/vector /var/lib/vector

# Smoke test
Expand Down
2 changes: 1 addition & 1 deletion distribution/docker/distroless-static/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ RUN mkdir -p /var/lib/vector
FROM gcr.io/distroless/static:latest

COPY --from=builder /vector/bin/* /usr/local/bin/
COPY --from=builder /vector/config/vector.toml /etc/vector/vector.toml
COPY --from=builder /vector/config/vector.yaml /etc/vector/vector.yaml
COPY --from=builder /var/lib/vector /var/lib/vector

# Smoke test
Expand Down
2 changes: 1 addition & 1 deletion distribution/init.d/vector
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
#
# description: A lightweight and ultra-fast tool for building observability pipelines
# processname: vector
# config: /etc/vector/vector.toml
# config: /etc/vector/vector.yaml
# pidfile: /var/run/vector.pid

### BEGIN INIT INFO
Expand Down
2 changes: 1 addition & 1 deletion distribution/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ install_from_archive() {
printf "%s Install succeeded! 🚀\n" "$_prompt"
printf "%s To start Vector:\n" "$_prompt"
printf "\n"
printf "%s vector --config $prefix/config/vector.toml\n" "$_indent"
printf "%s vector --config $prefix/config/vector.yaml\n" "$_indent"
printf "\n"
printf "%s More information at https://vector.dev/docs/\n" "$_prompt"

Expand Down
2 changes: 1 addition & 1 deletion distribution/rpm/vector.spec
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ mkdir -p %{buildroot}%{_datadir}/%{_name}
mkdir -p %{buildroot}%{_unitdir}

cp -a %{_builddir}/bin/vector %{buildroot}%{_bindir}
cp -a %{_builddir}/config/vector.toml %{buildroot}%{_sysconfdir}/%{_name}/vector.toml
cp -a %{_builddir}/config/vector.yaml %{buildroot}%{_sysconfdir}/%{_name}/vector.yaml
cp -a %{_builddir}/config/examples/. %{buildroot}%{_sysconfdir}/%{_name}/examples
cp -a %{_builddir}/systemd/vector.service %{buildroot}%{_unitdir}/vector.service
cp -a %{_builddir}/systemd/vector.default %{buildroot}%{_sysconfdir}/default/vector
Expand Down
4 changes: 3 additions & 1 deletion src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,10 @@ impl Opts {
pub struct RootOpts {
/// Read configuration from one or more files. Wildcard paths are supported.
/// File format is detected from the file name.
/// If zero files are specified the default config path
/// If zero files are specified the deprecated default config path
/// `/etc/vector/vector.toml` will be targeted.
/// And if the aforementioned file does not exist,
/// then `/etc/vector/vector.yaml` will be used.
#[arg(
id = "config",
short,
Expand Down
4 changes: 3 additions & 1 deletion src/config/cmd.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ pub struct Opts {

/// Read configuration from one or more files. Wildcard paths are supported.
/// File format is detected from the file name.
/// If zero files are specified the default config path
/// If zero files are specified the deprecated default config path
/// `/etc/vector/vector.toml` will be targeted.
/// And if the aforementioned file does not exist,
/// then `/etc/vector/vector.yaml` will be used.
#[arg(
id = "config",
short,
Expand Down
40 changes: 29 additions & 11 deletions src/config/loading/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -305,22 +305,40 @@ where
}

#[cfg(not(windows))]
fn default_config_paths() -> Vec<ConfigPath> {
vec![ConfigPath::File(
"/etc/vector/vector.toml".into(),
Some(Format::Toml),
)]
fn default_path() -> PathBuf {
"/etc/vector/vector.toml".into()
}

#[cfg(windows)]
fn default_config_paths() -> Vec<ConfigPath> {
fn default_path() -> PathBuf {
let program_files =
std::env::var("ProgramFiles").expect("%ProgramFiles% environment variable must be defined");
let config_path = format!("{}\\Vector\\config\\vector.toml", program_files);
vec![ConfigPath::File(
PathBuf::from(config_path),
Some(Format::Toml),
)]
format!("{}\\Vector\\config\\vector.toml", program_files).into()
}

fn default_config_paths() -> Vec<ConfigPath> {
#[cfg(not(windows))]
let default_path = default_path();

#[cfg(windows)]
let default_path = default_path();

let yaml_path = default_path.with_extension("yaml");
if default_path.exists() {
warn!("DEPRECATED Using the deprecated {:?} config path as the default config location. Vector is migrating \
to YAML as the default config format. Future Vector versions will use {:?} as the default config location.",
default_path,
yaml_path);

vec![ConfigPath::File(default_path, Some(Format::Toml))]
} else {
warn!(
"The {:?} config path does not exist. Vector will attempt to use new default {:?} instead.",
default_path, yaml_path
);

vec![ConfigPath::File(yaml_path, Some(Format::Yaml))]
}
}

#[cfg(all(
Expand Down
2 changes: 1 addition & 1 deletion src/graph.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ pub struct Opts {
/// Read configuration from one or more files. Wildcard paths are supported.
/// File format is detected from the file name.
/// If zero files are specified the default config path
/// `/etc/vector/vector.toml` will be targeted.
/// `/etc/vector/vector.yaml` will be targeted.
#[arg(
id = "config",
short,
Expand Down
2 changes: 1 addition & 1 deletion src/unit_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ pub struct Opts {
paths_yaml: Vec<PathBuf>,

/// Any number of Vector config files to test. If none are specified the
/// default config path `/etc/vector/vector.toml` will be targeted.
/// default config path `/etc/vector/vector.yaml` will be targeted.
#[arg(value_delimiter(','))]
paths: Vec<PathBuf>,

Expand Down
3 changes: 2 additions & 1 deletion src/validate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ pub struct Opts {
/// Any number of Vector config files to validate.
/// Format is detected from the file name.
/// If none are specified the default config path `/etc/vector/vector.toml`
/// will be targeted.
/// will be targeted. And if the aforementioned file does not exist,
// then `/etc/vector/vector.yaml` will be used.
#[arg(env = "VECTOR_CONFIG", value_delimiter(','))]
pub paths: Vec<PathBuf>,

Expand Down
2 changes: 1 addition & 1 deletion website/content/en/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ platform:
configure:
title: "Easy to configure"
description: "A simple, composable format enables you to build flexible pipelines"
filename: "/etc/vector/vector.toml"
filename: "/etc/vector/vector.yaml"
below: "Configuration examples are in [TOML](https://toml.io) but Vector also supports [YAML](https://yaml.org) and [JSON](https://json.org)"
# Example configs are specified in cue/examples.cue

Expand Down
8 changes: 4 additions & 4 deletions website/content/en/docs/administration/management.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ To manage the Vector executable directly, without a process manager:
{{< tab title="Start" >}}

```bash
vector --config /etc/vector/vector.toml
vector --config /etc/vector/vector.yaml

# Or supply a JSON or YAML config file
```
Expand Down Expand Up @@ -129,9 +129,9 @@ If you're running Vector on Windows (perhaps you installed it using [MSI]), you

```powershell
C:\Program Files\Vector\bin\vector \
--config C:\Program Files\Vector\config\vector.toml
--config C:\Program Files\Vector\config\vector.yaml

# Or supply a JSON or YAML config file
# Or supply a TOML or JSON config file
```

{{< /tab >}}
Expand All @@ -147,7 +147,7 @@ If you're running Vector using [Docker], the command interface is the same acros
```bash
docker run \
-d \
-v ~/vector.toml:/etc/vector/vector.toml:ro \
-v ~/vector.yaml:/etc/vector/vector.yaml:ro \
-p 8686:8686 \
timberio/vector:{{< version >}}-alpine
```
Expand Down
4 changes: 2 additions & 2 deletions website/content/en/docs/administration/validating.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Vector provides a subcommand, [`validate`][validate], that checks the validity o
Here's an example:

```bash
vector validate /etc/vector/vector.toml
vector validate /etc/vector/vector.yaml
```

You can also check multiple files:
Expand Down Expand Up @@ -52,7 +52,7 @@ configured topology:
These environment checks can be disabled using the [`--no-environment`][no_environment] flag:

```bash
vector validate --no-environment /etc/vector/vector.toml
vector validate --no-environment /etc/vector/vector.yaml
```

[components]: /components
Expand Down
4 changes: 2 additions & 2 deletions website/content/en/docs/reference/configuration/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ vector --config /etc/vector/vector.yaml
{{< tab title="TOML" >}}

```shell
vector --config /etc/vector/vector.toml
vector --config /etc/vector/vector.yaml
```

{{< /tab >}}
Expand Down Expand Up @@ -295,7 +295,7 @@ data templating languages such as [ytt], [Jsonnet] and [Cue].

The location of your Vector configuration file depends on your installation
method. For most Linux-based systems, the file can be found at
`/etc/vector/vector.toml`.
`/etc/vector/vector.yaml`.

### Multiple files

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ You can execute tests within a [configuration](#configuring) file using Vector's
[`test`][vector_test] subcommand:

```bash
vector test /etc/vector/vector.toml
vector test /etc/vector/vector.yaml
```

You can also specify multiple configuration files to test:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ EOF
```shell
docker run \
-d \
-v $PWD/vector.toml:/etc/vector/vector.toml:ro \
-v $PWD/vector.toml:/etc/vector/vector.yaml:ro \
-p 8686:8686 \
timberio/vector:{{< version >}}-debian
```
Expand Down
2 changes: 1 addition & 1 deletion website/content/en/guides/level-up/transformation.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ supports JSON and YAML.
To start Vector using this topology:

```bash
vector --config-toml /etc/vector/vector.toml
vector --config-toml /etc/vector/vector.yaml
```

You should see lines like this emitted via stdout (formatted for readability
Expand Down
6 changes: 3 additions & 3 deletions website/content/en/guides/level-up/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ If you see an exception in Vector's logs then we've clearly found the issue.
Before you report a bug, please enable backtraces:

```bash
RUST_BACKTRACE=full vector --config=/etc/vector/vector.toml
RUST_BACKTRACE=full vector --config=/etc/vector/vector.yaml
```

Backtraces are _critical_ for debugging errors. Once you have the backtrace
Expand All @@ -95,14 +95,14 @@ Vector rate limits logs in the hot path. As a result, dropping to the
{{< tab title="Env Var" >}}

```shell
VECTOR_LOG=debug vector --config=/etc/vector/vector.toml
VECTOR_LOG=debug vector --config=/etc/vector/vector.yaml
```

{{< /tab >}}
{{< tab title="Flag" >}}

```bash
vector --verbose --config=/etc/vector/vector.toml
vector --verbose --config=/etc/vector/vector.yaml
```

{{< /tab >}}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
date: "2023-09-06"
title: "0.33 Upgrade Guide"
description: "An upgrade guide that addresses breaking changes in 0.33.0"
authors: ["pront"]
Fixed Show fixed Hide fixed
release: "0.33.0"
hide_on_release_notes: false
badges:
type: breaking change
---

Vector's 0.33.0 release includes **deprecations**:

1. [Default config location change](#default-config-location-change)

We cover them below to help you upgrade quickly:

## Upgrade guide

### Deprecations

#### Default config location change {#default-config-location-change}

The default config location `/etc/vector/vector.toml` which is used by Vector `0.32.0` is now deprecated. This location will still be used in `0.33.0`. The new default path is `/etc/vector/vector.yaml`, please migrate to this new default path or specify the config path explicitly.

Vector `0.33.0` will attempt to load `/etc/vector/vector.toml` first, and if it is not present, it will fallback to `/etc/vector/vector.yaml`. However, Vector release `0.34.0` will automatically load `/etc/vector/vector.yaml` only.
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ administration: interfaces: docker_cli: {
start: #"""
docker run \
-d \
-v \#(paths.config):/etc/vector/vector.toml:ro \
-v \#(paths.config):/etc/vector/vector.yaml:ro \
-p \#(_api_port):\#(_api_port) \{flags}
timberio/vector:{version}-{variant}
"""#
Expand Down
Loading