This repository has been archived by the owner on Dec 8, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 66
Support multi-engine per table (batching) #113
Merged
Merged
Commits on Jan 2, 2019
-
config,restore: introduced
[mydumper] batch-size
Removed `[tikv-importer] batch-size` to avoid confusion. Removed `[mydumper] min-region-size` since it is useless now.
Configuration menu - View commit details
-
Copy full SHA for 44dc8d1 - Browse repository at this point
Copy the full SHA 44dc8d1View commit details -
Configuration menu - View commit details
-
Copy full SHA for e61bacf - Browse repository at this point
Copy the full SHA e61bacfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8e00a50 - Browse repository at this point
Copy the full SHA 8e00a50View commit details -
Configuration menu - View commit details
-
Copy full SHA for c621266 - Browse repository at this point
Copy the full SHA c621266View commit details -
Configuration menu - View commit details
-
Copy full SHA for 142dd9a - Browse repository at this point
Copy the full SHA 142dd9aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8b2498c - Browse repository at this point
Copy the full SHA 8b2498cView commit details -
Configuration menu - View commit details
-
Copy full SHA for ed0471a - Browse repository at this point
Copy the full SHA ed0471aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 485fc9d - Browse repository at this point
Copy the full SHA 485fc9dView commit details -
Configuration menu - View commit details
-
Copy full SHA for cc34c4e - Browse repository at this point
Copy the full SHA cc34c4eView commit details -
Configuration menu - View commit details
-
Copy full SHA for c7f0a37 - Browse repository at this point
Copy the full SHA c7f0a37View commit details -
Configuration menu - View commit details
-
Copy full SHA for 294f228 - Browse repository at this point
Copy the full SHA 294f228View commit details
Commits on Jan 3, 2019
-
Configuration menu - View commit details
-
Copy full SHA for 92e8dad - Browse repository at this point
Copy the full SHA 92e8dadView commit details
Commits on Jan 14, 2019
-
Support non-uniform batch size (#114)
* mydump: non-uniform batch size * *: make the `batch-size-scale` configurable * *: implemented the optimized non-uniform strategy * tests: due to change of strategy, checkpoint_engines count becomes 4 again * mydump/region: slightly adjust the batch size computation * Use the exact result of 1/Beta(N, R) instead of an approximation * When the number of engines is small and the total engine size of the first (table-concurrency) batches exceed the table size, the last batch was truncated, and disrupt the pipeline. Now in these case we will reduce the batch size to avoid this disruption. * restore: log the SQL size and KV size of each engine for debugging * config: change default batch size and ratio given experiment result * config: added more explanation about batch-import-ratio
Configuration menu - View commit details
-
Copy full SHA for 8ba38ed - Browse repository at this point
Copy the full SHA 8ba38edView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.