Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry for range exceed error (#2774) #2777

Merged
merged 1 commit into from
Mar 28, 2024

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #2774

What problem does this PR solve?

TiSpark may set the wrong range to TiKV when using FetchHandleRDD. We have two guesses about this:

  1. TiSpark has a bug when splitting range with index scan. This bug only occurs under certain data.
  2. TiSpark supports cluster index but client-java does not support it. There is a problem with the coordination between them.

What is changed and how it works?

Since it is hard to find the root cause, we just log it and retry once when this error occurs. We use client-java's splitRangeByRegion method to avoid exceeding the bound issue. It seems this method can split the range correctly.

Spark Plan

= Physical Plan == *(1) ColumnarToRow +- TiSpark RegionTaskExec{downgradeThreshold=1000000000,downgradeFilter=[] +- RowToColumnar +- TiKV FetchHandleRDD{[table: items] IndexLookUp, Columns: item_primary_key@BYTES, item_id@VARCHAR(45), item_set_id@VARCHAR(45), product_id@VARCHAR(45), product_set_id@VARCHAR(45), point_of_sale_country@VARCHAR(2), merchant_id@LONG, merchant_item_id@VARCHAR(127), merchant_item_set_id@VARCHAR(127), domains@JSON, product_sources@JSON, image_signatures@JSON, normalized_short_link_clusters@JSON, canonical_links@JSON, feed_item_ids@JSON, feed_profile_ids@JSON, reconciled_data@JSON, source_data@JSON, cdc_change_indicator@JSON, cdc_new_values@JSON, cdc_old_values@JSON, created_time@LONG, arrival_time@LONG, updated_time@LONG, timestamp_data@JSON: { {IndexRangeScan(Index:item_id(item_id)): { RangeFilter: [], Range: [([t\200\000\000\000\000\000\023\226_i\200\000\000\000\000\000\000\003\000], [t\200\000\000\000\000\000\023\226_i\200\000\000\000\000\000\000\003\372])] }}; {TableRowIDScan} }, startTs: 448636486137151521}

* print range

* update version

* retry once

* RC2

* Revert "RC2"

This reverts commit 1e38743.

* opt

* revert version
Copy link

ti-chi-bot bot commented Mar 28, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign xuanyu66 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot requested a review from zhangyangyu March 28, 2024 04:02
@shiyuhang0 shiyuhang0 merged commit df1dc00 into pingcap:master Mar 28, 2024
12 of 13 checks passed
@shiyuhang0 shiyuhang0 deleted the cherry-pick-2774-to-master branch March 28, 2024 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants