From db27e145db585c496665c5d9812c8f9c31baa86b Mon Sep 17 00:00:00 2001 From: Zike Yang Date: Mon, 11 Dec 2023 17:03:11 +0800 Subject: [PATCH] Refactor kafka sink connector creation doc --- connectors/kafka-sink/v2.10.5.9/kafka-sink.md | 47 +++++++++++++------ .../kafka-sink/v2.11.2.13/kafka-sink.md | 47 +++++++++++++------ connectors/kafka-sink/v3.0.2.1/kafka-sink.md | 47 +++++++++++++------ connectors/kafka-sink/v3.1.1.1/kafka-sink.md | 47 +++++++++++++------ 4 files changed, 132 insertions(+), 56 deletions(-) diff --git a/connectors/kafka-sink/v2.10.5.9/kafka-sink.md b/connectors/kafka-sink/v2.10.5.9/kafka-sink.md index ed23eb585..ee8f542d1 100644 --- a/connectors/kafka-sink/v2.10.5.9/kafka-sink.md +++ b/connectors/kafka-sink/v2.10.5.9/kafka-sink.md @@ -34,22 +34,41 @@ The prerequisites for connecting an Kafka sink connector to external systems inc Apache Kafka: Ensure you have a running Kafka instance. You can follow the official Kafka [Quickstart guide](https://kafka.apache.org/quickstart) to set up a Kafka instance if you don't have one already. ### 1. Create a connector -Depending on the environment, there are several ways to create an Kafka sink connector: - -- [Create a Connector on StreamNative Cloud](https://docs.streamnative.io/docs/connector-create). -- [Create a Connector with Function worker](https://pulsar.apache.org/docs/io-quickstart/). -Using this way requires you to download a **NAR** package to create a connector. You can download the version you need from the `download button` at the beginning of the article. -- [Create a Connector with Function mesh](https://functionmesh.io/docs/connectors/run-connector). -Using this way requires you to set the docker image. You can choose the version you want to launch from [here](https://hub.docker.com/r/streamnative/pulsar-io-kafka). - -No matter how you create an Kafka sink connector, the minimum connector configuration contains the following parameters: -```yaml -configs: - bootstrapServers: "localhost:9092" - topic: "kafka-topic-name" - ack: 1 + +The following command shows how to use [pulsarctl](https://github.com/streamnative/pulsarctl) to create a `builtin` connector. If you want to create a `non-builtin` connector, +you need to replace `--sink-type kafka` with `--archive /path/to/pulsar-io-kafka.nar`. You can find the button to download the `nar` package at the beginning of the document. + +{% callout title="For StreamNative Cloud User" type="note" %} +If you are a StreamNative Cloud user, you need [set up your environment](https://docs.streamnative.io/docs/connector-setup) first. +{% /callout %} + +```bash +pulsarctl sinks create \ + --sink-type kafka \ + --name kafka-sink \ + --tenant public \ + --namespace default \ + --inputs "Your topic name" \ + --parallelism 1 \ + --sink-config \ + '{ + "bootstrapServers": "localhost:9092", + "topic": "kafka-topic-name", + "ack": 1 + }' ``` +The `--sink-config` is the minimum necessary configuration for starting this connector, and it is a JSON string. You need to substitute the relevant parameters with your own. +If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference. + +{% callout title="Note" type="note" %} +You can also choose to use a variety of other tools to create a connector: +- [pulsar-admin](https://pulsar.apache.org/docs/3.1.x/io-use/): The command arguments for `pulsar-admin` are similar to those of `pulsarctl`. You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector ). +- [RestAPI](https://pulsar.apache.org/sink-rest-api/?version=3.1.1): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector). +- [Terraform](https://github.com/hashicorp/terraform): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector). +- [Function Mesh](https://functionmesh.io/docs/connectors/run-connector): The docker image can be found at the beginning of the document. +{% /callout %} + ### 2. Send messages to the topic {% callout title="Note" type="note" %} diff --git a/connectors/kafka-sink/v2.11.2.13/kafka-sink.md b/connectors/kafka-sink/v2.11.2.13/kafka-sink.md index df9fd8c4d..10305fc50 100644 --- a/connectors/kafka-sink/v2.11.2.13/kafka-sink.md +++ b/connectors/kafka-sink/v2.11.2.13/kafka-sink.md @@ -34,22 +34,41 @@ The prerequisites for connecting an Kafka sink connector to external systems inc Apache Kafka: Ensure you have a running Kafka instance. You can follow the official Kafka [Quickstart guide](https://kafka.apache.org/quickstart) to set up a Kafka instance if you don't have one already. ### 1. Create a connector -Depending on the environment, there are several ways to create an Kafka sink connector: - -- [Create a Connector on StreamNative Cloud](https://docs.streamnative.io/docs/connector-create). -- [Create a Connector with Function worker](https://pulsar.apache.org/docs/io-quickstart/). -Using this way requires you to download a **NAR** package to create a connector. You can download the version you need from the `download button` at the beginning of the article. -- [Create a Connector with Function mesh](https://functionmesh.io/docs/connectors/run-connector). -Using this way requires you to set the docker image. You can choose the version you want to launch from [here](https://hub.docker.com/r/streamnative/pulsar-io-kafka). - -No matter how you create an Kafka sink connector, the minimum connector configuration contains the following parameters: -```yaml -configs: - bootstrapServers: "localhost:9092" - topic: "kafka-topic-name" - ack: 1 + +The following command shows how to use [pulsarctl](https://github.com/streamnative/pulsarctl) to create a `builtin` connector. If you want to create a `non-builtin` connector, +you need to replace `--sink-type kafka` with `--archive /path/to/pulsar-io-kafka.nar`. You can find the button to download the `nar` package at the beginning of the document. + +{% callout title="For StreamNative Cloud User" type="note" %} +If you are a StreamNative Cloud user, you need [set up your environment](https://docs.streamnative.io/docs/connector-setup) first. +{% /callout %} + +```bash +pulsarctl sinks create \ + --sink-type kafka \ + --name kafka-sink \ + --tenant public \ + --namespace default \ + --inputs "Your topic name" \ + --parallelism 1 \ + --sink-config \ + '{ + "bootstrapServers": "localhost:9092", + "topic": "kafka-topic-name", + "ack": 1 + }' ``` +The `--sink-config` is the minimum necessary configuration for starting this connector, and it is a JSON string. You need to substitute the relevant parameters with your own. +If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference. + +{% callout title="Note" type="note" %} +You can also choose to use a variety of other tools to create a connector: +- [pulsar-admin](https://pulsar.apache.org/docs/3.1.x/io-use/): The command arguments for `pulsar-admin` are similar to those of `pulsarctl`. You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector ). +- [RestAPI](https://pulsar.apache.org/sink-rest-api/?version=3.1.1): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector). +- [Terraform](https://github.com/hashicorp/terraform): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector). +- [Function Mesh](https://functionmesh.io/docs/connectors/run-connector): The docker image can be found at the beginning of the document. +{% /callout %} + ### 2. Send messages to the topic {% callout title="Note" type="note" %} diff --git a/connectors/kafka-sink/v3.0.2.1/kafka-sink.md b/connectors/kafka-sink/v3.0.2.1/kafka-sink.md index 15a861b98..91dd83feb 100644 --- a/connectors/kafka-sink/v3.0.2.1/kafka-sink.md +++ b/connectors/kafka-sink/v3.0.2.1/kafka-sink.md @@ -34,22 +34,41 @@ The prerequisites for connecting an Kafka sink connector to external systems inc Apache Kafka: Ensure you have a running Kafka instance. You can follow the official Kafka [Quickstart guide](https://kafka.apache.org/quickstart) to set up a Kafka instance if you don't have one already. ### 1. Create a connector -Depending on the environment, there are several ways to create an Kafka sink connector: - -- [Create a Connector on StreamNative Cloud](https://docs.streamnative.io/docs/connector-create). -- [Create a Connector with Function worker](https://pulsar.apache.org/docs/io-quickstart/). -Using this way requires you to download a **NAR** package to create a connector. You can download the version you need from the `download button` at the beginning of the article. -- [Create a Connector with Function mesh](https://functionmesh.io/docs/connectors/run-connector). -Using this way requires you to set the docker image. You can choose the version you want to launch from [here](https://hub.docker.com/r/streamnative/pulsar-io-kafka). - -No matter how you create an Kafka sink connector, the minimum connector configuration contains the following parameters: -```yaml -configs: - bootstrapServers: "localhost:9092" - topic: "kafka-topic-name" - ack: 1 + +The following command shows how to use [pulsarctl](https://github.com/streamnative/pulsarctl) to create a `builtin` connector. If you want to create a `non-builtin` connector, +you need to replace `--sink-type kafka` with `--archive /path/to/pulsar-io-kafka.nar`. You can find the button to download the `nar` package at the beginning of the document. + +{% callout title="For StreamNative Cloud User" type="note" %} +If you are a StreamNative Cloud user, you need [set up your environment](https://docs.streamnative.io/docs/connector-setup) first. +{% /callout %} + +```bash +pulsarctl sinks create \ + --sink-type kafka \ + --name kafka-sink \ + --tenant public \ + --namespace default \ + --inputs "Your topic name" \ + --parallelism 1 \ + --sink-config \ + '{ + "bootstrapServers": "localhost:9092", + "topic": "kafka-topic-name", + "ack": 1 + }' ``` +The `--sink-config` is the minimum necessary configuration for starting this connector, and it is a JSON string. You need to substitute the relevant parameters with your own. +If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference. + +{% callout title="Note" type="note" %} +You can also choose to use a variety of other tools to create a connector: +- [pulsar-admin](https://pulsar.apache.org/docs/3.1.x/io-use/): The command arguments for `pulsar-admin` are similar to those of `pulsarctl`. You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector ). +- [RestAPI](https://pulsar.apache.org/sink-rest-api/?version=3.1.1): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector). +- [Terraform](https://github.com/hashicorp/terraform): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector). +- [Function Mesh](https://functionmesh.io/docs/connectors/run-connector): The docker image can be found at the beginning of the document. +{% /callout %} + ### 2. Send messages to the topic {% callout title="Note" type="note" %} diff --git a/connectors/kafka-sink/v3.1.1.1/kafka-sink.md b/connectors/kafka-sink/v3.1.1.1/kafka-sink.md index e560b374e..030cd06c3 100644 --- a/connectors/kafka-sink/v3.1.1.1/kafka-sink.md +++ b/connectors/kafka-sink/v3.1.1.1/kafka-sink.md @@ -34,22 +34,41 @@ The prerequisites for connecting an Kafka sink connector to external systems inc Apache Kafka: Ensure you have a running Kafka instance. You can follow the official Kafka [Quickstart guide](https://kafka.apache.org/quickstart) to set up a Kafka instance if you don't have one already. ### 1. Create a connector -Depending on the environment, there are several ways to create an Kafka sink connector: - -- [Create a Connector on StreamNative Cloud](https://docs.streamnative.io/docs/connector-create). -- [Create a Connector with Function worker](https://pulsar.apache.org/docs/io-quickstart/). -Using this way requires you to download a **NAR** package to create a connector. You can download the version you need from the `download button` at the beginning of the article. -- [Create a Connector with Function mesh](https://functionmesh.io/docs/connectors/run-connector). -Using this way requires you to set the docker image. You can choose the version you want to launch from [here](https://hub.docker.com/r/streamnative/pulsar-io-kafka). - -No matter how you create an Kafka sink connector, the minimum connector configuration contains the following parameters: -```yaml -configs: - bootstrapServers: "localhost:9092" - topic: "kafka-topic-name" - ack: 1 + +The following command shows how to use [pulsarctl](https://github.com/streamnative/pulsarctl) to create a `builtin` connector. If you want to create a `non-builtin` connector, +you need to replace `--sink-type kafka` with `--archive /path/to/pulsar-io-kafka.nar`. You can find the button to download the `nar` package at the beginning of the document. + +{% callout title="For StreamNative Cloud User" type="note" %} +If you are a StreamNative Cloud user, you need [set up your environment](https://docs.streamnative.io/docs/connector-setup) first. +{% /callout %} + +```bash +pulsarctl sinks create \ + --sink-type kafka \ + --name kafka-sink \ + --tenant public \ + --namespace default \ + --inputs "Your topic name" \ + --parallelism 1 \ + --sink-config \ + '{ + "bootstrapServers": "localhost:9092", + "topic": "kafka-topic-name", + "ack": 1 + }' ``` +The `--sink-config` is the minimum necessary configuration for starting this connector, and it is a JSON string. You need to substitute the relevant parameters with your own. +If you want to configure more parameters, see [Configuration Properties](#configuration-properties) for reference. + +{% callout title="Note" type="note" %} +You can also choose to use a variety of other tools to create a connector: +- [pulsar-admin](https://pulsar.apache.org/docs/3.1.x/io-use/): The command arguments for `pulsar-admin` are similar to those of `pulsarctl`. You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector ). +- [RestAPI](https://pulsar.apache.org/sink-rest-api/?version=3.1.1): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector). +- [Terraform](https://github.com/hashicorp/terraform): You can find an example for [StreamNative Cloud Doc](https://docs.streamnative.io/docs/connector-create#create-a-built-in-connector). +- [Function Mesh](https://functionmesh.io/docs/connectors/run-connector): The docker image can be found at the beginning of the document. +{% /callout %} + ### 2. Send messages to the topic {% callout title="Note" type="note" %}