Crash: KafkaJSNonRetriableError: Specified group generation id is not valid

[oleh-poberezhets]

Hello

We have the problem with consumers crash.
It's look like the same problem described in #1466 and #1474
But we have "kafkajs": "2.2.4", module with fix from #1474.

Logs from crashed pod:

Logs from all pods at the same time

Maybe it has something to do with how we commit the offset:

        const autoCommit = false;
        const partitionsConsumedConcurrently = 1;
        
        await this.consumer.run({
            partitionsConsumedConcurrently,
            autoCommit,
            eachMessage: async ({ topic, partition, message }) => {
                const { error, decodedMessage } = await this.deserialize(
                    message,
                    formatOptions,
                );

                try {
                    await onMessageHandler(
                        error,
                        decodedMessage,
                        message.offset,
                        topic,
                        partition,
                        message.timestamp,
                    );

                    if (!autoCommit) {
                        await this.consumer.commitOffsets([
                            {
                                topic,
                                partition,
                                offset: (parseInt(message.offset, 10) + 1).toString(),
                            },
                        ]);
                    }
                } catch (e) {
                    this.logger.logError(e);

                    throw e;
                }
            },
        });

Environment:

• KafkaJS version - 2.2.4
• NodeJS version - 20.11.0

[HassanRazaMuhammadTufail]

I am also facing same issue, logs shows that consumer crashes after I get this error few times with in a shorter span. @oleh-poberezhets have you found any work around?

[oleh-poberezhets]

I am also facing same issue, logs shows that consumer crashes after I get this error few times with in a shorter span. @oleh-poberezhets have you found any work around?

We will try to test consumers with enabled AutoCommit.
I will let you know when the results are in