Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
enhancement(pulsar sink): Refactor to use StreamSink
This commit heavily refactors the Pulsar Sink to use the StreamSink interface and is modeled after the Kafka Sink. It also adds additional features that bring it in line with Kafka Sink feature set. This includes: * Refactoring to use StreamSink instead of Sink interace. See #9261 * Supports dynamic topics using a topic template * Refactor configurations in advance of adding Pulsar source * Rework message parsing to support logs and metrics, with support for dynamic keys and properties This work is heavily modeled after Kafka sink. This means there has been some duplication of some utility code. However, it has not been refactored to remove the duplication as there wasn't a clear pattern of where such shared code should be put. Additionally, this refactor seems to be much simpler by using StreamSink but does require some workarounds limitations in the Pulsar client library by wrapping certain resources in Arc<Mutex> that *may* have performance implications. I am not famaliar enough to know if there might be some efficiencies by structuring this differently. Remaining work: * Add a few more tests
- Loading branch information