fs2-kafka v2.2.0

This is the third release in the fs2-kafka 2.x series, based on cats-effect 3. If focuses on usability, bringing the same improvements as v1.8.0. It also includes various dependency updates.

fs2-kafka v1.8.0

This release focuses on improving usability, with new helper methods for working with Kafka producers and consumers. This release is for users stuck on cats-effect 2. For cats-effect 3 users, v2.2.0 will be coming shortly.

New KafkaProducer syntax (#632 @keirlawson)

We've added several new helper methods for KafkaProducer to simplify common use cases.

Producing one record and getting the resulting record metadata - before:

producer.produce(ProducerRecords.one(ProducerRecord(topic, key, value))).map(_.map(_.records.head.get._2))

After:

producer.produceOne_(topic, key, value)

New KafkaConsumer syntax (#662 #664 @bplommer )

We've also added new extension methods for KafkaConsumer in the Stream context (no new imports needed!), so the most common operations no longer require explicit use of flatMap and evalTap. The stream and partitionedStream methods on KafkaConsumer can now be invoked as records and partitionedRecords respectively, for greater clarity.

Allocating a consumer, subscribing it to a topic, and streaming records - before:

KafkaConsumer.stream(consumerSettings)
  .evalTap(_.subscribeTo("topic"))
  .flatMap(_.stream)

After:

KafkaConsumer.stream(consumerSettings)
  .subscribeTo("topic")
  .records

Vulcan testkit (#629 @keirlawson)

A new module, fs2-kafka-vulcan-testkit-munit, makes it easier to test the schema compatibility of Vulcan codecs against a live schema registry instance.

Other changes

Various dependency updates

fs2-kafka v2.1.0

Scala 3.0.0

The main feature of this release is support for Scala 3.0.0 (and dropping support for 3.0.0-RC3).

Updates

• Scala 3.0.0
• Scala 2.13.6
• Vulcan 1.7.1
• Cats-effect 3.1.1
• Cats 2.6.1
• fs2 3.0.4
• Discipline-scalatest 2.1.5
• Kind-projector 0.13.0
• testcontainers-scala 0.39.4

fs2-kafka v1.7.0

This release brings support for Scala 3.0.0 to the fs2-kafka 1.x series, which uses cats-effect 2.

Updates

• Scala 3.0.0
• Scala 2.13.6
• Vulcan 1.7.1
• Cats-effect 2.5.1
• Cats 2.6.1
• fs2 2.5.6
• Discipline-scalatest 2.1.5
• Kind-projector 0.13.0
• testcontainers-scala 0.39.4

fs2-kafka v2.0.0

This is the first release in the fs2-kafka 2.x series. Its main difference with previous releases is that it uses cats-effect 3 rather than cats-effect 2, but there are some other changes and improvements. It is identical to v2.0.0-RC3. See release notes for previous release candidates for further details.

fs2-kafka v2.0.0-RC3

This is likely to be the final release candidate for fs2-kafka v2.0.0. It contains the same changes as v1.6.1.

fs2-kafka v1.6.1

Bug fix

• Fixed an issue in KafkaCredentialStore with handling of newlines in PEMs (#606)

Dependency updates

• Support Scala 3.0.0-RC3, drop Scala 3.0.0-RC2, update Cats to 2.6.0, update Vulcan to 1.6.0
• Update fs2-core to 2.5.5
• Update discipline-scalatest, cats-testkit-scalatest to 2.1.4

fs2-kafka v2.0.0-RC2

This release candidate contains the same changes as v1.6.0, in addition to the following.

User-facing changes

• Add dual-effect constructors for KafkaAdminClient and KafkaProducerConnection (#595)
• Allow a custom ExecutionContext to be provided for blocking operations in KafkaProducerSettinngs and KafkaConsumerSettings (#590, #594)

Internals

• Use a single-threaded ExecutionContext for producers (#584)
• Add missing MkProducer argument, force compiler errors when they're missing (#585)
• Decouple effect type of Mk traits from effect type of settings (#588)
• Replace Deferred + Dispatcher with Promise in producer code (#587)

fs2-kafka v1.6.0

Documentation improvements

• Clarified that custom blocker provided to ConsumerSettings#withBlocker must be single-threaded (#583)

Internals

• Use blocker for scheduling commitAsync, remove semaphore from WithConsumer (#586)

Dependency updates

• cats-effect 2.5.0 (#597)
• kafka-clients 2.8.0 (#596)

fs2-kafka v2.0.0-RC1

This is the first release candidate for fs2-kafka v2.0.0, based on cats-effect 3.0. It is available for Scala 2.12, 2.13, and 3.0.0-RC2.

Revised roadmap

Previously we planned to release two new major versions: 2.0, based on cats-effect 2, and 3.0 based on cats-effect 3. However we no longer intend to release a new major version based on cats-effect 2, so fs2-kafka 2.0 will be based on cats-effect 3. This release candidate is therefore the successor to v3.0.0-M3.

We expect to release fs2-kafka 3.0, with further breaking changes, in the second half of 2021. The 1.x series will continue to receive bug fixes and some new features may be backported, but development will focus on the 2.x and 3.x series.

Migrating from v1.x

Before updating to v2.0.0-RC1, you should first update to v1.5.0 and fix any deprecation warnings. This process can mostly be automated with ScalaFix using scalafix --rules=github:fd4s/fs2-kafka/Fs2Kafka?sha=v1.5.0. (See here for how to add ScalaFix to your project.)

Apart from the migration to cats-effect 3.0, v2.0.0-RC1 makes only a few source-breaking changes from v1.5.0. Again, the migration can mostly be automated with Scalafix, using scalafix --rules=github:fd4s/fs2-kafka/Fs2Kafka?sha=v2.0.0-RC1.

• Reordering of passthrough parameters (#466): the P type parameter in ProducerRecords, TransactionalProducerRecords and ProducerResult has been moved to the left.
• Moving resource instantiation out of settings classes (#581). Classes such as ProducerSettings no longer have a createProducer method (etc) - instead, this capability is represented by separately by the MkProducer trait. Most user code will be unaffected by this change.

It is not binary-compatible with previous releases.