v4.35.0-rc1

For installation instructions check out the getting started guide.

Added

• Auth fields added to the schema_registry input and output. (@mihaitodor)
• New experimental kafka_migrator and kafka_migrator_bundle inputs and outputs. (@mihaitodor)
• New experimental kafka_migrator_offsets output. (@mihaitodor)
• Field job_project added to the gcp_bigquery output. (@Roviluca)

The full change log can be found here.

v4.34.0

For installation instructions check out the getting started guide.

Fixed

• The schema_registry output now allows pushing schemas if the target Schema Registry instance is in IMPORT mode. (@mihaitodor)
• Fixed an issue where the azure_blob_storage input would fail to delete blobs when using targets_input with delete_objects: true. (@mihaitodor)
• New experimental gcp_vertex_ai_chat processor. (@rockwotj)
• New experimental aws_bedrock_chat processor. (@rockwotj)

The full change log can be found here.

v4.34.0-rc2

For installation instructions check out the getting started guide.

Fixed

• The schema_registry output now allows pushing schemas if the target Schema Registry instance is in IMPORT mode. (@mihaitodor)
• Fixed an issue where the azure_blob_storage input would fail to delete blobs when using targets_input with delete_objects: true. (@mihaitodor)
• New experimental gcp_vertex_ai_chat processor. (@rockwotj)
• New experimental aws_bedrock_chat processor. (@rockwotj)

The full change log can be found here.

v4.33.0

For installation instructions check out the getting started guide.

Added

• Field content_md5 added to the aws_s3 output. (@dom-lee-naimuri)
• Field send_ack added to the nats input. (@plejd-sebman)
• New Bloblang method vector. (@rockwotj)
• New experimental ockam_kafka input and output. (@mrinalwadhwa, @davide-baldo)
• Field credentials_json added to all GCP components. (@tomasz-sadura)
• (Benthos) The list subcommand now supports the format jsonschema. (@Jeffail)
• New experimental schema_registry input and output. (@mihaitodor)
• New experimental qdrant output. (@Anush008)
• (Benthos) The --set run flag now supports structured values, e.g. --set input={}. (@Jeffail)

The full change log can be found here.

v4.33.0-rc4

For installation instructions check out the getting started guide.

Added

• Field content_md5 added to the aws_s3 output. (@dom-lee-naimuri)
• Field send_ack added to the nats input. (@plejd-sebman)
• New Bloblang method vector. (@rockwotj)
• New experimental ockam_kafka input and output. (@mrinalwadhwa, @davide-baldo)
• Field credentials_json added to all GCP components. (@tomasz-sadura)
• (Benthos) The list subcommand now supports the format jsonschema. (@Jeffail)
• New experimental schema_registry input and output. (@mihaitodor)
• New experimental qdrant output. (@Anush008)
• (Benthos) The --set run flag now supports structured values, e.g. --set input={}. (@Jeffail)

The full change log can be found here.

v4.32.1

For installation instructions check out the getting started guide.

Added

• Field app_name added to the MongoDB components. (@mihaitodor)
• New openai_chat_completion processor. (@rockwotj)
• New openai_embeddings processor. (@rockwotj)
• New openai_image_generation processor. (@rockwotj)
• New openai_speech processor. (@rockwotj)
• New openai_transcription processor. (@rockwotj)
• New openai_translation processor. (@rockwotj)
• New ollama_chat processor. (@rockwotj)
• New ollama_embeddings processor. (@rockwotj)

Changed

• The gcp_pubsub output now rejects messages with metadata values which contain invalid UTF-8-encoded runes. (@AndreasBergmeier6176)
• The .goreleaser.yml configuration has been set back to version 1. (@Jeffail)
• The number of release build artifacts for the community and cloud flavours have been reduced due to Github Action Runner disk space limitations.

The full change log can be found here.

v4.31.0

For installation instructions check out the getting started guide.

4.31.0 - 2024-07-19

Added

• The splunk input and splunk_hec output now support custom tls configuration. (@mihaitodor)
• Field timestamp added to the kafka and kafka_franz outputs. (@mihaitodor)
• (Benthos) Field max_retries added to the retry processor. (@mihaitodor)
• (Benthos) Metadata fields retry_count and backoff_duration added to the retry processor. (@mihaitodor)
• (Benthos) Parameter escape_html added to the format_json() Bloblang method. (@mihaitodor)
• (Benthos) New array bloblang method. (@gramian)
• (Benthos) Algorithm fnv32 added to the hash bloblang method. (@CallMeMhz)
• New experimental redpanda_data_transform. (@rockwotj)
• New -community suffixed build included in release artifacts, containing only FOSS functionality. (@Jeffail)
• New -cloud suffixed build included in release artifacts, containing components enabled in Redpanda Cloud. (@Jeffail)
• Field status_topic added to the global redpanda config block. (@Jeffail)
• New pinecone output. (@rockwotj)
• (Benthos) The /ready endpoint in regular operation now provides a detailed summary of all inputs and outputs, including connection errors where applicable. (@Jeffail)

Changed

• (Benthos) All cli subcommands that previously relied on root-level flags (streams, lint, test, echo) now explicitly define those flags such that they appear in help-text and can be specified after the subcommand itself. This means previous commands such as connect -r ./foo.yaml streams ./bar.yaml can now be more intuitively written as connect streams -r ./foo.yaml ./bar.yaml and so on. The old style will still work in order to preserve backwards compatibility, but the help-text for these root-level flags has been hidden. (@Jeffail)

v4.30.1

For installation instructions check out the getting started guide.

Added

• (Benthos) Field omit_empty added to the lines scanner. (@mihaitodor)
• (Benthos) New scheme gcm added to the encrypt_aes and decrypy_aes Bloblang methods. (@abergmeier)
• (Benthos) New Bloblang method pow. (@mfamador)
• (Benthos) New sin, cos, tan and pi bloblang methods. (@mfamador)
• (Benthos) Field proxy_url added to the websocket input and output. (@mihaitodor)
• New experimental splunk input. (@mihaitodor)

Fixed

• The sql_insert and sql_raw components no longer fail when inserting large binary blobs into Oracle BLOB columns. (@mihaitodor)
• (Benthos) The websocket input and output now obey the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables. (@mihaitodor)
• AWS Lambda serverless build artifacts have been added back to official releases.

Changed

• The splunk_hec output is now implemented as a native Go component. (@mihaitodor)

The full change log can be found here.

v4.30.0

For installation instructions check out the getting started guide.

Added

• (Benthos) Field omit_empty added to the lines scanner. (@mihaitodor)
• (Benthos) New scheme gcm added to the encrypt_aes and decrypy_aes Bloblang methods. (@abergmeier)
• (Benthos) New Bloblang method pow. (@mfamador)
• (Benthos) New sin, cos, tan and pi bloblang methods. (@mfamador)
• (Benthos) Field proxy_url added to the websocket input and output. (@mihaitodor)
• New experimental splunk input. (@mihaitodor)

Fixed

• The sql_insert and sql_raw components no longer fail when inserting large binary blobs into Oracle BLOB columns. (@mihaitodor)
• (Benthos) The websocket input and output now obey the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables. (@mihaitodor)

Changed

• The splunk_hec output is now implemented as a native Go component. (@mihaitodor)

The full change log can be found here.

v4.29.0

For installation instructions check out the getting started guide.

Added

• Go API: New packages public/bundle/free and public/bundle/enterprise with explicit licensing for bundles of component imports.
• Field auth.oauth2.scope added to the pulsar input and output. (@srenatus)
• Field subscription_initial_position added to the pulsar input. (@srenatus)

Fixed

• The pulsar input and output should no longer ignore auth.oauth2 fields. (@srenatus)
• Creating builds using make no longer prints warnings when the repository does not contain a tag. (@mkysel)
• Messages resulting from the redis processor are no longer invalid when using hash commands. (@mkysel)
• The nats_jetstream input no longer fails to initialise when a stream is specified and a subject is not. (@maxarndt)

The full change log can be found here.