Skip to content

Commit

Permalink
chore(docs): update a few more examples to YAML (vectordotdev#19103)
Browse files Browse the repository at this point in the history
  • Loading branch information
@pront @AndrooTheChen
pront authored and AndrooTheChen committed Sep 23, 2024
1 parent 53ab02b commit 6244117
Show file tree
Hide file tree
Showing 7 changed files with 115 additions and 102 deletions.
2 changes: 1 addition & 1 deletion website/content/en/_index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ configure:
title: "Easy to configure"
description: "A simple, composable format enables you to build flexible pipelines"
filename: "/etc/vector/vector.yaml"
below: "Configuration examples are in [TOML](https://toml.io) but Vector also supports [YAML](https://yaml.org) and [JSON](https://json.org)"
below: "Configuration examples are in [YAML](https://yaml.org) but Vector also supports [TOML](https://toml.io) and [JSON](https://json.org)"
# Example configs are specified in cue/examples.cue

# Installation section
Expand Down
2 changes: 1 addition & 1 deletion website/content/en/docs/reference/configuration/_index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ vector --config /etc/vector/vector.yaml
{{< tab title="TOML" >}}

```shell
vector --config /etc/vector/vector.yaml
vector --config /etc/vector/vector.toml
```

{{< /tab >}}
Expand Down
10 changes: 5 additions & 5 deletions website/content/en/docs/setup/installation/manual/from-archives.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ source $HOME/.profile
Start Vector:

```shell
vector --config config/vector.toml
vector --config config/vector.yaml
```

### Linux (ARMv7)
Expand Down Expand Up @@ -79,7 +79,7 @@ source $HOME/.profile
Start Vector:

```shell
vector --config config/vector.toml
vector --config config/vector.yaml
```

### macoS (x86_64)
Expand Down Expand Up @@ -114,7 +114,7 @@ source $HOME/.profile
Start Vector:

```shell
vector --config config/vector.toml
vector --config config/vector.yaml
```

### Windows (x86_64)
Expand Down Expand Up @@ -180,7 +180,7 @@ source $HOME/.profile
Start Vector:

```shell
vector --config config/vector.toml
vector --config config/vector.yaml
```

## Next steps
Expand All @@ -190,7 +190,7 @@ vector --config config/vector.toml
The Vector configuration file is located at:

```shell
config/vector.toml
config/vector.yaml
```

Example configurations are located in `config/vector/examples/*`. You can learn more about configuring Vector in the [Configuration] documentation.
Expand Down
4 changes: 2 additions & 2 deletions website/content/en/docs/setup/installation/manual/from-source.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ When finished, the Vector binary is placed in `target/<target>/release/vector`.
Finally, you can start Vector:

```shell
target/<target>/release/vector --config config/vector.toml
target/<target>/release/vector --config config/vector.yaml
```

### Windows
Expand Down Expand Up @@ -146,7 +146,7 @@ The command above builds a Docker image with a Rust toolchain for a Linux target
The Vector configuration file is located at:

```shell
config/vector.toml
config/vector.yaml
```

Example configurations are located in `config/vector/examples/*`. You can learn more about configuring Vector in the [Configuration] documentation.
Expand Down
86 changes: 45 additions & 41 deletions website/cue/reference/configuration.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -583,13 +583,13 @@ configuration: {
Vector interpolates environment variables within your configuration file
with the following syntax:
```toml title="vector.toml"
[transforms.add_host]
inputs = ["apache_logs"]
type = "remap"
source = '''
.host = get_env_var!("HOSTNAME")
'''
```toml title="vector.yaml"
transforms:
add_host:
inputs: ["apache_logs"]
type: "remap"
source: |
.host = get_env_var!("HOSTNAME")
```
"""

Expand Down Expand Up @@ -639,21 +639,23 @@ configuration: {
The following example shows a simple configuration with two backends defined:
```toml title="vector.toml"
[secret.backend_1]
type = "exec"
command = ["/path/to/cmd1", "--some-option"]
[secret.backend_2]
type = "exec"
command = ["/path/to/cmd2"]
[sinks.dd_logs]
type = "datadog_logs"
default_api_key = "SECRET[backend_1.dd_api_key]"
[sinks.splunk]
type = "splunk_hec"
default_token = "SECRET[backend_2.splunk_token]"
```toml title="vector.yaml"
secret:
backend_1:
type: "exec"
command: ["/path/to/cmd1", "--some-option"]
backend_2:
type: "exec"
command: ["/path/to/cmd2"]
sinks:
dd_logs:
type: "datadog_logs"
default_api_key: "SECRET[backend_1.dd_api_key]"
splunk:
type: "splunk_hec"
default_token: "SECRET[backend_2.splunk_token]"
```
In that example Vector will retrieve the `dd_api_key` from `backen_1` and `splunk_token` from `backend_2`.
Expand Down Expand Up @@ -723,13 +725,13 @@ configuration: {
You can pass multiple configuration files when starting Vector:
```bash
vector --config vector1.toml --config vector2.toml
vector --config vector1.yaml --config vector2.yaml
```
Or use a [globbing syntax](\(urls.globbing)):
```bash
vector --config /etc/vector/*.toml
vector --config /etc/vector/*.yaml
```
"""
}
Expand All @@ -742,7 +744,7 @@ configuration: {
configure it as follows:
```toml
type = "sink_type"
type: "sink_type"
# here the sinks options
```
Expand All @@ -763,26 +765,28 @@ configuration: {
For example:
```toml
[sources.app1_logs]
type = "file"
includes = ["/var/log/app1.log"]
```yaml
sources:
app1_logs:
type: "file"
includes: ["/var/log/app1.log"]
[sources.app2_logs]
type = "file"
includes = ["/var/log/app.log"]
app2_logs:
type: "file"
includes: ["/var/log/app.log"]
[sources.system_logs]
type = "file"
includes = ["/var/log/system.log"]
system_logs:
type: "file"
includes: ["/var/log/system.log"]
[sinks.app_logs]
type = "datadog_logs"
inputs = ["app*"]
sinks:
app_logs:
type: "datadog_logs"
inputs: ["app*"]
[sinks.archive]
type = "aws_s3"
inputs = ["*_logs"]
archive:
type: "aws_s3"
inputs: ["*_logs"]
```
"""
}
Expand Down
111 changes: 60 additions & 51 deletions website/cue/reference/examples.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,78 +9,87 @@ config_examples: [#ConfigExample, ...#ConfigExample] & [
{
title: "Redacted Datadog Agent logs to Datadog"
example: #"""
[sources.datadog_agent]
type = "datadog_agent"
address = "0.0.0.0:80"
sources:
datadog_agent:
type: "datadog_agent"
address: "0.0.0.0:80"
[transforms.remove_sensitive_user_info]
type = "remap"
inputs = ["datadog_agent"]
source = '''
redact(., filters: ["us_social_security_number"])
'''
transforms:
remove_sensitive_user_info:
type: "remap"
inputs: ["datadog_agent"]
source: |
redact(., filters: ["us_social_security_number"])
[sinks.datadog_backend]
type = "datadog_logs"
inputs = ["remove_sensitive_user_info"]
default_api_key = "${DATADOG_API_KEY}"
sinks:
datadog_backend:
type: "datadog_logs"
inputs: ["remove_sensitive_user_info"]
default_api_key: "${DATADOG_API_KEY}"
"""#
},
{
title: "Kafka topic to Elasticsearch"
example: #"""
[sources.kafka_in]
type = "kafka"
bootstrap_servers = "10.14.22.123:9092,10.14.23.332:9092"
group_id = "vector-logs"
key_field = "message"
topics = ["logs-*"]
sources:
kafka_in:
type: "kafka"
bootstrap_servers: "10.14.22.123:9092,10.14.23.332:9092"
group_id: "vector-logs"
key_field: "message"
topics: ["logs-*"]
[transforms.json_parse]
type = "remap"
inputs = ["kafka_in"]
source = '''
parsed, err = parse_json(.message)
if err != null {
log(err, level: "error")
}
. |= object(parsed) ?? {}
'''
transforms:
json_parse:
type: "remap"
inputs: ["kafka_in"]
source: |
parsed, err = parse_json(.message)
if err != null {
log(err, level: "error")
}
. |= object(parsed) ?? {}
[sinks.elasticsearch_out]
type = "elasticsearch"
inputs = ["json_parse"]
endpoint = "http://10.24.32.122:9000"
index = "logs-via-kafka"
sinks:
elasticsearch_out:
type: "elasticsearch"
inputs: ["json_parse"]
endpoint: "http://10.24.32.122:9000"
index: "logs-via-kafka"
"""#
},
{
title: "Kubernetes logs to AWS S3"
example: #"""
[sources.k8s_in]
type = "kubernetes_logs"
sources:
k8s_in:
type: "kubernetes_logs"
[sinks.aws_s3_out]
type = "aws_s3"
inputs = ["k8s_in"]
bucket = "k8s-logs"
region = "us-east-1"
compression = "gzip"
encoding.codec = "json"
sinks:
aws_s3_out:
type: "aws_s3"
inputs: ["k8s_in"]
bucket: "k8s-logs"
region: "us-east-1"
compression: "gzip"
encoding:
codec: "json"
"""#
},
{
title: "Splunk HEC to Datadog"
example: #"""
[sources.splunk_hec_in]
type = "splunk_hec"
address = "0.0.0.0:8080"
token = "${SPLUNK_HEC_TOKEN}"
sources:
splunk_hec_in:
type: "splunk_hec"
address: "0.0.0.0:8080"
token: "${SPLUNK_HEC_TOKEN}"
[sinks.datadog_out]
type = "datadog_logs"
inputs = ["splunk_hec_in"]
default_api_key = "${DATADOG_API_KEY}"
sinks:
datadog_out:
type: "datadog_logs"
inputs: ["splunk_hec_in"]
default_api_key: "${DATADOG_API_KEY}"
"""#
},
]
2 changes: 1 addition & 1 deletion website/cue/reference/urls.cue
View file
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ urls: {
date: "https://man7.org/linux/man-pages/man1/date.1.html"
debian: "https://www.debian.org/"
debian_system_groups: "https://wiki.debian.org/SystemGroups"
default_configuration: "\(vector_repo)/blob/master/config/vector.toml"
default_configuration: "\(vector_repo)/blob/master/config/vector.yaml"
dnstap: "http://dnstap.info/"
docker: "https://www.docker.com/"
docker_alpine: "\(docker_hub)/_/alpine"
Expand Down

0 comments on commit 6244117

Please sign in to comment.