Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sync script and sync docs. #349

Merged
merged 5 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 161 additions & 0 deletions connectors/activemq-source/v2.5.1/io-activemq-source.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
---
description: ActiveMQ Connector integrates Apache Pulsar with Apache ActiveMQ.
author: gaoran10,dependabot[bot],shibd,sijie
contributors: gaoran10,dependabot[bot],shibd,sijie
language: Java,Shell,Dockerfile
document:
source: https://github.com/streamnative/pulsar-io-activemq
license: Apache License 2.0
license_link: "https://github.com/streamnative/pulsar-io-activemq/blob/master/LICENSE"
tags: apache-pulsar,pulsar-io,source-connector,sink-connector,apache-activemq
alias: ActiveMQ Sink
features: ["ActiveMQ Connector integrates Apache Pulsar with Apache ActiveMQ. "]
icon: "/images/connectors/activemq_logo_white_vertical.jpg"
download: https://api.github.com/repos/streamnative/pulsar-io-activemq/tarball/refs/tags/v2.5.1
support: streamnative
support_link: https://github.com/streamnative/pulsar-io-activemq
support_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
owner_name: "streamnative"
owner_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
dockerfile:
sn_available: ""
id: "io-activemq-source"
---


The ActiveMQ source connector receives messages from ActiveMQ clusters and writes messages to Pulsar topics.

# Installation

```
git clone https://github.com/streamnative/pulsar-io-activemq.git
cd pulsar-io-activemq/
mvn clean install -DskipTests
cp target/pulsar-io-activemq-0.0.1.nar $PULSAR_HOME/pulsar-io-activemq-0.0.1.nar
```

# Configuration

The configuration of the ActiveMQ source connector has the following properties.

## ActiveMQ source connector configuration

| Name | Type|Required | Default | Description
|------|----------|----------|---------|-------------|
| `protocol` |String| true | "tcp" | The ActiveMQ protocol. |
| `host` | String| true | " " (empty string) | The ActiveMQ host. |
| `port` | int |true | 5672 | The ActiveMQ port. |
| `username` | String|false | " " (empty string) | The username used to authenticate to ActiveMQ. |
| `password` | String|false | " " (empty string) | The password used to authenticate to ActiveMQ. |
| `queueName` | String|false | " " (empty string) | The ActiveMQ queue name that messages should be read from or written to. |
| `topicName` | String|false | " " (empty string) | The ActiveMQ topic name that messages should be read from or written to. |

## Configure ActiveMQ source connector

Before using the ActiveMQ source connector, you need to create a configuration file through one of the following methods.

* JSON

```json
{
"tenant": "public",
"namespace": "default",
"name": "activemq-source",
"topicName": "user-op-queue-topic",
"archive": "connectors/pulsar-io-activemq-0.0.1.nar",
"parallelism": 1,
"configs": {
"protocol": "tcp",
"host": "localhost",
"port": "61616",
"username": "admin",
"password": "admin",
"queueName": "user-op-queue"
}
}
```

* YAML

```yaml
tenant: "public"
namespace: "default"
name: "activemq-source"
topicName: "user-op-queue-topic"
archive: "connectors/pulsar-io-activemq-0.0.1.nar"
parallelism: 1

configs:
protocol: "tcp"
host: "localhost"
port: "61616"
username: "admin"
password: "admin"
queueName: "user-op-queue"
```

1. Prepare ActiveMQ service.

```
docker pull rmohr/activemq
docker run -p 61616:61616 -p 8161:8161 rmohr/activemq
```

2. Put the `pulsar-io-activemq-0.0.1.nar` in the pulsar connectors catalog.

```
cp pulsar-io-activemq-0.0.1.nar $PULSAR_HOME/connectors/pulsar-io-activemq-0.0.1.nar
```

3. Start Pulsar in standalone mode.

```
$PULSAR_HOME/bin/pulsar standalone
```

4. Run ActiveMQ source locally.

```
$PULSAR_HOME/bin/pulsar-admin source localrun --source-config-file activemq-source-config.yaml
```

5. Consume Pulsar messages.

```
bin/pulsar-client consume -s "sub-products" public/default/user-op-queue-topic -n 0
```

6. Send ActiveMQ messages.

Use the test method `sendMessage` of the `class org.apache.pulsar.ecosystem.io.activemq.ActiveMQDemo`
to send ActiveMQ messages.

```
@Test
private void sendMessage() throws JMSException {

ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://localhost:61616");

@Cleanup
Connection connection = connectionFactory.createConnection();
connection.start();

@Cleanup
Session session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);

Destination destination = session.createQueue("user-op-queue");

@Cleanup
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);

for (int i = 0; i < 10; i++) {
String msgContent = "Hello ActiveMQ - " + i;
ActiveMQTextMessage message = new ActiveMQTextMessage();
message.setText(msgContent);
producer.send(message);
}
}
```


Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,20 @@ author: gaoran10,Anonymitaet,dependabot[bot],freeznet
contributors: gaoran10,Anonymitaet,dependabot[bot],freeznet
language: Java,Shell,Dockerfile,Python
document:
source: "https://github.com/streamnative/pulsar-io-amqp-1-0"
source: https://github.com/streamnative/pulsar-io-amqp-1-0
license: Apache License 2.0
license_link: "https://github.com/streamnative/pulsar-io-amqp-1-0/blob/master/LICENSE"
tags:
alias: AMQP 1.0 Sink Connector
features: ["support sink/source for AMQP version 1.0.0"]
icon: "/images/connectors/amqp-logo.png"
download: "https://api.github.com/repos/streamnative/pulsar-io-amqp-1-0/tarball/refs/tags/v3.1.0.5"
download: https://api.github.com/repos/streamnative/pulsar-io-amqp-1-0/tarball/refs/tags/v2.10.5.9
support: streamnative
support_link: https://github.com/streamnative/pulsar-io-amqp-1-0
support_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
owner_name: "streamnative"
owner_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
dockerfile: "https://hub.docker.com/r/streamnative/pulsar-io-amqp-1-0"
dockerfile: https://hub.docker.com/r/streamnative/pulsar-io-amqp-1-0
sn_available: "true"
id: "amqp-1-0-sink"
---
Expand All @@ -27,7 +27,7 @@ id: "amqp-1-0-sink"

The AMQP 1.0 sink connector pulls messages from Pulsar topics and persists messages to [AMQP 1.0](https://www.amqp.org/).

![](https://raw.githubusercontent.com/streamnative/pulsar-io-amqp-1-0/v3.1.0.5/docs/amqp-1-0-sink.png)
![](https://raw.githubusercontent.com/streamnative/pulsar-io-amqp-1-0/v2.10.5.9/docs/amqp-1-0-sink.png)

## Quick start

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,20 @@ author: gaoran10,Anonymitaet,dependabot[bot],freeznet
contributors: gaoran10,Anonymitaet,dependabot[bot],freeznet
language: Java,Shell,Dockerfile,Python
document:
source: "https://github.com/streamnative/pulsar-io-amqp-1-0"
source: https://github.com/streamnative/pulsar-io-amqp-1-0
license: Apache License 2.0
license_link: "https://github.com/streamnative/pulsar-io-amqp-1-0/blob/master/LICENSE"
tags:
alias: AMQP 1.0 Sink Connector
features: ["support sink/source for AMQP version 1.0.0"]
icon: "/images/connectors/amqp-logo.png"
download: "https://api.github.com/repos/streamnative/pulsar-io-amqp-1-0/tarball/refs/tags/v2.10.5.8"
download: https://api.github.com/repos/streamnative/pulsar-io-amqp-1-0/tarball/refs/tags/v3.1.0.6
support: streamnative
support_link: https://github.com/streamnative/pulsar-io-amqp-1-0
support_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
owner_name: "streamnative"
owner_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
dockerfile: "https://hub.docker.com/r/streamnative/pulsar-io-amqp-1-0"
dockerfile: https://hub.docker.com/r/streamnative/pulsar-io-amqp-1-0
sn_available: "true"
id: "amqp-1-0-sink"
---
Expand All @@ -27,7 +27,7 @@ id: "amqp-1-0-sink"

The AMQP 1.0 sink connector pulls messages from Pulsar topics and persists messages to [AMQP 1.0](https://www.amqp.org/).

![](https://raw.githubusercontent.com/streamnative/pulsar-io-amqp-1-0/v2.10.5.8/docs/amqp-1-0-sink.png)
![](https://raw.githubusercontent.com/streamnative/pulsar-io-amqp-1-0/v3.1.0.6/docs/amqp-1-0-sink.png)

## Quick start

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,20 @@ author: gaoran10,Anonymitaet,dependabot[bot],freeznet
contributors: gaoran10,Anonymitaet,dependabot[bot],freeznet
language: Java,Shell,Dockerfile,Python
document:
source: "https://github.com/streamnative/pulsar-io-amqp-1-0"
source: https://github.com/streamnative/pulsar-io-amqp-1-0
license: Apache License 2.0
license_link: "https://github.com/streamnative/pulsar-io-amqp-1-0/blob/master/LICENSE"
tags:
alias: AMQP 1.0 Source Connector
features: ["support sink/source for AMQP version 1.0.0"]
icon: "/images/connectors/amqp-logo.png"
download: "https://api.github.com/repos/streamnative/pulsar-io-amqp-1-0/tarball/refs/tags/v3.1.0.5"
download: https://api.github.com/repos/streamnative/pulsar-io-amqp-1-0/tarball/refs/tags/v2.10.5.9
support: streamnative
support_link: https://github.com/streamnative/pulsar-io-amqp-1-0
support_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
owner_name: "streamnative"
owner_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
dockerfile: "https://hub.docker.com/r/streamnative/pulsar-io-amqp-1-0"
dockerfile: https://hub.docker.com/r/streamnative/pulsar-io-amqp-1-0
sn_available: "true"
id: "amqp-1-0-source"
---
Expand All @@ -27,7 +27,7 @@ id: "amqp-1-0-source"

The AMQP 1.0 source connector receives messages from [AMQP 1.0](https://www.amqp.org/) and writes messages to Pulsar topics.

![](https://raw.githubusercontent.com/streamnative/pulsar-io-amqp-1-0/v3.1.0.5/docs/amqp-1-0-source.png)
![](https://raw.githubusercontent.com/streamnative/pulsar-io-amqp-1-0/v2.10.5.9/docs/amqp-1-0-source.png)

## Quick start

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,20 @@ author: gaoran10,Anonymitaet,dependabot[bot],freeznet
contributors: gaoran10,Anonymitaet,dependabot[bot],freeznet
language: Java,Shell,Dockerfile,Python
document:
source: "https://github.com/streamnative/pulsar-io-amqp-1-0"
source: https://github.com/streamnative/pulsar-io-amqp-1-0
license: Apache License 2.0
license_link: "https://github.com/streamnative/pulsar-io-amqp-1-0/blob/master/LICENSE"
tags:
alias: AMQP 1.0 Source Connector
features: ["support sink/source for AMQP version 1.0.0"]
icon: "/images/connectors/amqp-logo.png"
download: "https://api.github.com/repos/streamnative/pulsar-io-amqp-1-0/tarball/refs/tags/v2.10.5.8"
download: https://api.github.com/repos/streamnative/pulsar-io-amqp-1-0/tarball/refs/tags/v3.1.0.6
support: streamnative
support_link: https://github.com/streamnative/pulsar-io-amqp-1-0
support_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
owner_name: "streamnative"
owner_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
dockerfile: "https://hub.docker.com/r/streamnative/pulsar-io-amqp-1-0"
dockerfile: https://hub.docker.com/r/streamnative/pulsar-io-amqp-1-0
sn_available: "true"
id: "amqp-1-0-source"
---
Expand All @@ -27,7 +27,7 @@ id: "amqp-1-0-source"

The AMQP 1.0 source connector receives messages from [AMQP 1.0](https://www.amqp.org/) and writes messages to Pulsar topics.

![](https://raw.githubusercontent.com/streamnative/pulsar-io-amqp-1-0/v2.10.5.8/docs/amqp-1-0-source.png)
![](https://raw.githubusercontent.com/streamnative/pulsar-io-amqp-1-0/v3.1.0.6/docs/amqp-1-0-source.png)

## Quick start

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,18 @@ language: Java,Shell,Dockerfile
document:
source: Private source
license: Business License
license_link: "https://github.com/streamnative/pulsar-io-aws-eventbridge/blob/master/LICENSE"
license_link:
tags:
alias: AWS EventBridge Sink Connector
features: [""]
icon: "/images/connectors/aws-eventbridge.png"
download:
download:
support: streamnative
support_link:
support_link: https://streamnative.io
support_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
owner_name: "streamnative"
owner_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
dockerfile:
dockerfile:
sn_available: "true"
id: "aws-eventbridge-sink"
---
Expand All @@ -26,7 +26,7 @@ id: "aws-eventbridge-sink"
The [Amazon EventBridge](https://aws.amazon.com/eventbridge/) sink connector pulls data from Pulsar topics and persists
data to Amazon EventBridge.

![](https://raw.githubusercontent.com/streamnative/pulsar-io-aws-eventbridge/v2.10.5.8/docs/aws-eventbridge-sink.png)
![](https://raw.githubusercontent.com/streamnative/pulsar-io-aws-eventbridge/v2.10.5.9/docs/aws-eventbridge-sink.png)

## Quick start

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ language: Java,Shell,Dockerfile
document:
source: Private source
license: Business License
license_link: "https://github.com/streamnative/pulsar-io-aws-eventbridge/blob/master/LICENSE"
license_link:
tags:
alias: AWS EventBridge Sink Connector
features: [""]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ language: Java,Shell,Dockerfile
document:
source: Private source
license: Business License
license_link: "https://github.com/streamnative/pulsar-io-aws-eventbridge/blob/master/LICENSE"
license_link:
tags:
alias: AWS EventBridge Sink Connector
features: [""]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ language: Java,Shell,Dockerfile
document:
source: Private source
license: Business License
license_link: "https://github.com/streamnative/pulsar-io-aws-eventbridge/blob/master/LICENSE"
license_link:
tags:
alias: AWS EventBridge Sink Connector
features: [""]
icon: "/images/connectors/aws-eventbridge.png"
download: Business License
download:
support: streamnative
support_link:
support_link: https://streamnative.io
support_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
owner_name: "streamnative"
owner_img: "https://avatars.githubusercontent.com/u/44651383?v=4"
Expand All @@ -26,7 +26,7 @@ id: "aws-eventbridge-sink"
The [Amazon EventBridge](https://aws.amazon.com/eventbridge/) sink connector pulls data from Pulsar topics and persists
data to Amazon EventBridge.

![](https://raw.githubusercontent.com/streamnative/pulsar-io-aws-eventbridge/v3.1.0.5/docs/aws-eventbridge-sink.png)
![](https://raw.githubusercontent.com/streamnative/pulsar-io-aws-eventbridge/v3.1.0.6/docs/aws-eventbridge-sink.png)

## Quick start

Expand Down
Loading