Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(docs): update a few more examples to YAML #19103

Merged
merged 1 commit into from
Nov 9, 2023
Merged

chore(docs): update a few more examples to YAML #19103

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion website/content/en/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ configure:
title: "Easy to configure"
description: "A simple, composable format enables you to build flexible pipelines"
filename: "/etc/vector/vector.yaml"
below: "Configuration examples are in [TOML](https://toml.io) but Vector also supports [YAML](https://yaml.org) and [JSON](https://json.org)"
below: "Configuration examples are in [YAML](https://yaml.org) but Vector also supports [TOML](https://toml.io) and [JSON](https://json.org)"
# Example configs are specified in cue/examples.cue

# Installation section
Expand Down
2 changes: 1 addition & 1 deletion website/content/en/docs/reference/configuration/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ vector --config /etc/vector/vector.yaml
{{< tab title="TOML" >}}

```shell
vector --config /etc/vector/vector.yaml
vector --config /etc/vector/vector.toml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is under the TOML tab.

```

{{< /tab >}}
Expand Down
10 changes: 5 additions & 5 deletions website/content/en/docs/setup/installation/manual/from-archives.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ source $HOME/.profile
Start Vector:

```shell
vector --config config/vector.toml
vector --config config/vector.yaml
```

### Linux (ARMv7)
Expand Down Expand Up @@ -79,7 +79,7 @@ source $HOME/.profile
Start Vector:

```shell
vector --config config/vector.toml
vector --config config/vector.yaml
```

### macoS (x86_64)
Expand Down Expand Up @@ -114,7 +114,7 @@ source $HOME/.profile
Start Vector:

```shell
vector --config config/vector.toml
vector --config config/vector.yaml
```

### Windows (x86_64)
Expand Down Expand Up @@ -180,7 +180,7 @@ source $HOME/.profile
Start Vector:

```shell
vector --config config/vector.toml
vector --config config/vector.yaml
```

## Next steps
Expand All @@ -190,7 +190,7 @@ vector --config config/vector.toml
The Vector configuration file is located at:

```shell
config/vector.toml
config/vector.yaml
```

Example configurations are located in `config/vector/examples/*`. You can learn more about configuring Vector in the [Configuration] documentation.
Expand Down
4 changes: 2 additions & 2 deletions website/content/en/docs/setup/installation/manual/from-source.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ When finished, the Vector binary is placed in `target/<target>/release/vector`.
Finally, you can start Vector:

```shell
target/<target>/release/vector --config config/vector.toml
target/<target>/release/vector --config config/vector.yaml
```

### Windows
Expand Down Expand Up @@ -146,7 +146,7 @@ The command above builds a Docker image with a Rust toolchain for a Linux target
The Vector configuration file is located at:

```shell
config/vector.toml
config/vector.yaml
```

Example configurations are located in `config/vector/examples/*`. You can learn more about configuring Vector in the [Configuration] documentation.
Expand Down
86 changes: 45 additions & 41 deletions website/cue/reference/configuration.cue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -583,13 +583,13 @@ configuration: {
Vector interpolates environment variables within your configuration file
with the following syntax:

```toml title="vector.toml"
[transforms.add_host]
inputs = ["apache_logs"]
type = "remap"
source = '''
.host = get_env_var!("HOSTNAME")
'''
```toml title="vector.yaml"
transforms:
add_host:
inputs: ["apache_logs"]
type: "remap"
source: |
.host = get_env_var!("HOSTNAME")
```
"""

Expand Down Expand Up @@ -639,21 +639,23 @@ configuration: {

The following example shows a simple configuration with two backends defined:

```toml title="vector.toml"
[secret.backend_1]
type = "exec"
command = ["/path/to/cmd1", "--some-option"]
[secret.backend_2]
type = "exec"
command = ["/path/to/cmd2"]

[sinks.dd_logs]
type = "datadog_logs"
default_api_key = "SECRET[backend_1.dd_api_key]"

[sinks.splunk]
type = "splunk_hec"
default_token = "SECRET[backend_2.splunk_token]"
```toml title="vector.yaml"
secret:
backend_1:
type: "exec"
command: ["/path/to/cmd1", "--some-option"]
backend_2:
type: "exec"
command: ["/path/to/cmd2"]

sinks:
dd_logs:
type: "datadog_logs"
default_api_key: "SECRET[backend_1.dd_api_key]"

splunk:
type: "splunk_hec"
default_token: "SECRET[backend_2.splunk_token]"
```

In that example Vector will retrieve the `dd_api_key` from `backen_1` and `splunk_token` from `backend_2`.
Expand Down Expand Up @@ -723,13 +725,13 @@ configuration: {
You can pass multiple configuration files when starting Vector:

```bash
vector --config vector1.toml --config vector2.toml
vector --config vector1.yaml --config vector2.yaml
```

Or use a [globbing syntax](\(urls.globbing)):

```bash
vector --config /etc/vector/*.toml
vector --config /etc/vector/*.yaml
```
"""
}
Expand All @@ -742,7 +744,7 @@ configuration: {
configure it as follows:

```toml
type = "sink_type"
type: "sink_type"
# here the sinks options
```

Expand All @@ -763,26 +765,28 @@ configuration: {

For example:

```toml
[sources.app1_logs]
type = "file"
includes = ["/var/log/app1.log"]
```yaml
sources:
app1_logs:
type: "file"
includes: ["/var/log/app1.log"]

[sources.app2_logs]
type = "file"
includes = ["/var/log/app.log"]
app2_logs:
type: "file"
includes: ["/var/log/app.log"]

[sources.system_logs]
type = "file"
includes = ["/var/log/system.log"]
system_logs:
type: "file"
includes: ["/var/log/system.log"]

[sinks.app_logs]
type = "datadog_logs"
inputs = ["app*"]
sinks:
app_logs:
type: "datadog_logs"
inputs: ["app*"]

[sinks.archive]
type = "aws_s3"
inputs = ["*_logs"]
archive:
type: "aws_s3"
inputs: ["*_logs"]
```
"""
}
Expand Down
111 changes: 60 additions & 51 deletions website/cue/reference/examples.cue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,78 +9,87 @@ config_examples: [#ConfigExample, ...#ConfigExample] & [
{
title: "Redacted Datadog Agent logs to Datadog"
example: #"""
[sources.datadog_agent]
type = "datadog_agent"
address = "0.0.0.0:80"
sources:
datadog_agent:
type: "datadog_agent"
address: "0.0.0.0:80"

[transforms.remove_sensitive_user_info]
type = "remap"
inputs = ["datadog_agent"]
source = '''
redact(., filters: ["us_social_security_number"])
'''
transforms:
remove_sensitive_user_info:
type: "remap"
inputs: ["datadog_agent"]
source: |
redact(., filters: ["us_social_security_number"])

[sinks.datadog_backend]
type = "datadog_logs"
inputs = ["remove_sensitive_user_info"]
default_api_key = "${DATADOG_API_KEY}"
sinks:
datadog_backend:
type: "datadog_logs"
inputs: ["remove_sensitive_user_info"]
default_api_key: "${DATADOG_API_KEY}"
"""#
},
{
title: "Kafka topic to Elasticsearch"
example: #"""
[sources.kafka_in]
type = "kafka"
bootstrap_servers = "10.14.22.123:9092,10.14.23.332:9092"
group_id = "vector-logs"
key_field = "message"
topics = ["logs-*"]
sources:
kafka_in:
type: "kafka"
bootstrap_servers: "10.14.22.123:9092,10.14.23.332:9092"
group_id: "vector-logs"
key_field: "message"
topics: ["logs-*"]

[transforms.json_parse]
type = "remap"
inputs = ["kafka_in"]
source = '''
parsed, err = parse_json(.message)
if err != null {
log(err, level: "error")
}
. |= object(parsed) ?? {}
'''
transforms:
json_parse:
type: "remap"
inputs: ["kafka_in"]
source: |
parsed, err = parse_json(.message)
if err != null {
log(err, level: "error")
}
. |= object(parsed) ?? {}

[sinks.elasticsearch_out]
type = "elasticsearch"
inputs = ["json_parse"]
endpoint = "http://10.24.32.122:9000"
index = "logs-via-kafka"
sinks:
elasticsearch_out:
type: "elasticsearch"
inputs: ["json_parse"]
endpoint: "http://10.24.32.122:9000"
index: "logs-via-kafka"
"""#
},
{
title: "Kubernetes logs to AWS S3"
example: #"""
[sources.k8s_in]
type = "kubernetes_logs"
sources:
k8s_in:
type: "kubernetes_logs"

[sinks.aws_s3_out]
type = "aws_s3"
inputs = ["k8s_in"]
bucket = "k8s-logs"
region = "us-east-1"
compression = "gzip"
encoding.codec = "json"
sinks:
aws_s3_out:
type: "aws_s3"
inputs: ["k8s_in"]
bucket: "k8s-logs"
region: "us-east-1"
compression: "gzip"
encoding:
codec: "json"
"""#
},
{
title: "Splunk HEC to Datadog"
example: #"""
[sources.splunk_hec_in]
type = "splunk_hec"
address = "0.0.0.0:8080"
token = "${SPLUNK_HEC_TOKEN}"
sources:
splunk_hec_in:
type: "splunk_hec"
address: "0.0.0.0:8080"
token: "${SPLUNK_HEC_TOKEN}"

[sinks.datadog_out]
type = "datadog_logs"
inputs = ["splunk_hec_in"]
default_api_key = "${DATADOG_API_KEY}"
sinks:
datadog_out:
type: "datadog_logs"
inputs: ["splunk_hec_in"]
default_api_key: "${DATADOG_API_KEY}"
"""#
},
]
2 changes: 1 addition & 1 deletion website/cue/reference/urls.cue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ urls: {
date: "https://man7.org/linux/man-pages/man1/date.1.html"
debian: "https://www.debian.org/"
debian_system_groups: "https://wiki.debian.org/SystemGroups"
default_configuration: "\(vector_repo)/blob/master/config/vector.toml"
default_configuration: "\(vector_repo)/blob/master/config/vector.yaml"
dnstap: "http://dnstap.info/"
docker: "https://www.docker.com/"
docker_alpine: "\(docker_hub)/_/alpine"
Expand Down
Loading