Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tx: enforce limit on client set transaction timeout #21504

Merged
merged 3 commits into from
Jul 22, 2024

Conversation

bharathv
Copy link
Contributor

@bharathv bharathv commented Jul 18, 2024

Adds a new configuration transaction_max_timeout_ms defaulting to 15mins.
Producers with transaction timeout greater than this configuration are rejected.
Hanging transactions (from producers with unset transaction timeouts) can potentially
cause blocked consumer due to LSO being blocked, so enforcing this limit on how long
a transaction can hang before it is auto aborted prevents such situations.

Fixes #17984

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.1.x
  • v23.3.x
  • v23.2.x

Release Notes

Features

  • Adds a new broker configuration transaction_max_timeout_ms. The configuration controls the maximum allowed user set timeout for transactions. If a client requested transaction timeout exceeds this configuration, the broker will return
    an error during transactional producer initialization. This guardrail prevents hanging transactions from blocking consumer progress. The default value is 15mins.

@bharathv
Copy link
Contributor Author

/dt

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Jul 18, 2024

new failures in https://buildkite.com/redpanda/redpanda/builds/51737#0190c76d-8d40-44fc-80cf-c12ef197a02e:

"rptest.transactions.tx_admin_api_test.TxAdminTest.test_mark_transaction_expired"
"rptest.transactions.tx_verifier_test.TxVerifierTest.test_all_tx_tests"

new failures in https://buildkite.com/redpanda/redpanda/builds/51737#0190c76d-8d42-4728-9a2d-2d2092092fa4:

"rptest.transactions.transactions_test.TransactionsTest.graceful_leadership_transfer_tx_coordinator_test"
"rptest.transactions.transactions_test.TransactionsTest.transaction_id_expiration_test"

new failures in https://buildkite.com/redpanda/redpanda/builds/51737#0190c785-b344-4570-8ccf-b41977facf6f:

"rptest.transactions.transactions_test.TransactionsTest.graceful_leadership_transfer_tx_coordinator_test"
"rptest.transactions.transactions_test.TransactionsTest.transaction_id_expiration_test"

new failures in https://buildkite.com/redpanda/redpanda/builds/51737#0190c785-b346-4e53-a871-d56fe362d491:

"rptest.transactions.tx_admin_api_test.TxAdminTest.test_mark_transaction_expired"
"rptest.transactions.tx_verifier_test.TxVerifierTest.test_all_tx_tests"

@bharathv
Copy link
Contributor Author

/dt

@vbotbuildovich
Copy link
Collaborator

bharathv added 3 commits July 18, 2024 22:39
The maximum allowed timeout for transactions. If a client requested
transaction timeout exceeds this configuration, the broker will return
an error during transactional producer initialization. This guardrail
prevents hanging transactions from blocking consumer progress.
…t_ms

error code is inline with Apache Kafka return code
invalid_transaction_timeout(50)
@bharathv bharathv marked this pull request as ready for review July 19, 2024 05:42
@bharathv bharathv requested a review from a team as a code owner July 19, 2024 05:42
@bharathv bharathv requested review from mmaslankaprv, bashtanov, ztlpn and a team and removed request for a team July 19, 2024 05:42
@mmaslankaprv mmaslankaprv merged commit bd93ade into redpanda-data:dev Jul 22, 2024
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enforce a limit on user set transaction timeout
3 participants