Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: support distributed merge_into #13151

Merged
merged 72 commits into from
Oct 31, 2023
Merged

feat: support distributed merge_into #13151

merged 72 commits into from
Oct 31, 2023

Commits on Oct 17, 2023

  1. add settings

    @JackTan25
    JackTan25 committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    1f13597 View commit details
    Browse the repository at this point in the history

  2. right join for merge into first

    @JackTan25
    JackTan25 committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    879ee6e View commit details
    Browse the repository at this point in the history

  3. add distribution optimization for merge into join

    @JackTan25
    JackTan25 committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    92538e1 View commit details
    Browse the repository at this point in the history

  4. split merge into plan

    @JackTan25
    JackTan25 committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    19a1999 View commit details
    Browse the repository at this point in the history

  5. fix update identify error

    @JackTan25
    JackTan25 committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    60f79f7 View commit details
    Browse the repository at this point in the history

  6. finish distibuted baisc codes

    @JackTan25
    JackTan25 committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    662af10 View commit details
    Browse the repository at this point in the history

  7. fix typo

    @JackTan25
    JackTan25 committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    76e1352 View commit details
    Browse the repository at this point in the history

  8. uniform row_kind and mutation_log

    @JackTan25
    JackTan25 committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    e4f7450 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 17, 2023
    Configuration menu
    Copy the full SHA
    cd92873 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2023

  1. fix MixRowKindAndLog serialize and deserialize

    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    93a1cdf View commit details
    Browse the repository at this point in the history

  2. add tests

    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    0a070b1 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 18, 2023
    Configuration menu
    Copy the full SHA
    35ff1f5 View commit details
    Browse the repository at this point in the history

  4. fix check

    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    f258ac0 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'distributed_merge_into' of https://github.com/JackTan25… 

    …/databend into distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    fe15639 View commit details
    Browse the repository at this point in the history

  6. fix check

    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    f292341 View commit details
    Browse the repository at this point in the history

  7. fix check

    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    137b52f View commit details
    Browse the repository at this point in the history

  8. fix test

    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    f39bfd1 View commit details
    Browse the repository at this point in the history

  9. fix test

    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    9b114f4 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'main' of https://github.com/datafuselabs/databend into … 

    …distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    d177d15 View commit details
    Browse the repository at this point in the history

  11. fix

    @JackTan25
    JackTan25 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    7bcf760 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 18, 2023
    Configuration menu
    Copy the full SHA
    636825f View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2023

  1. remove memory size limit

    @JackTan25
    JackTan25 committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    6bb03e6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'distributed_merge_into' of https://github.com/JackTan25… 

    …/databend into distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    df8acbf View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 19, 2023
    Configuration menu
    Copy the full SHA
    b262af2 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2023

  1. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 21, 2023
    Configuration menu
    Copy the full SHA
    c08cf13 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2023

  1. Merge branch 'main' of https://github.com/datafuselabs/databend into … 

    …distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    1c64166 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 23, 2023
    Configuration menu
    Copy the full SHA
    ceae57f View commit details
    Browse the repository at this point in the history

  3. optmizie merge source and add row_number processor

    @JackTan25
    JackTan25 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    0d0eba9 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'distributed_merge_into' of https://github.com/JackTan25… 

    …/databend into distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    74c1ec1 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' of https://github.com/datafuselabs/databend into … 

    …distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    ab2469d View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2023

  1. Merge branch 'main' of https://github.com/datafuselabs/databend into … 

    …distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    db00e4e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/datafuselabs/databend into … 

    …distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    e95a280 View commit details
    Browse the repository at this point in the history

  3. fix delete bug

    @JackTan25
    JackTan25 committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    f1f2f94 View commit details
    Browse the repository at this point in the history

  4. add row number plan

    @JackTan25
    JackTan25 committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    bd3be35 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 24, 2023
    Configuration menu
    Copy the full SHA
    a2fec57 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2023

  1. fix row number

    @JackTan25
    JackTan25 committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    b60479e View commit details
    Browse the repository at this point in the history

  2. refactor merge into pipeline

    @JackTan25
    JackTan25 committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    c79a04b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of https://github.com/datafuselabs/databend into … 

    …distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    551d854 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. split row_number and log, try to get hash table source data

    @JackTan25
    JackTan25 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    126da66 View commit details
    Browse the repository at this point in the history

  2. finish distributed codes, need to get data from hashtable

    @JackTan25
    JackTan25 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    4112417 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of https://github.com/datafuselabs/databend into … 

    …distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    2450df7 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. finish not macthed append data

    @JackTan25
    JackTan25 committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    622a13a View commit details
    Browse the repository at this point in the history

  2. fix conflict

    @JackTan25
    JackTan25 committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    8a56035 View commit details
    Browse the repository at this point in the history

  3. fix filter

    @JackTan25
    JackTan25 committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    a15df12 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 27, 2023
    Configuration menu
    Copy the full SHA
    18d25d0 View commit details
    Browse the repository at this point in the history

  5. fix filter

    @JackTan25
    JackTan25 committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    672218e View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' of https://github.com/datafuselabs/databend into … 

    …distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    03ac2a8 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2023

  1. fix distributed bugs,many bugs, need to support insert

    @JackTan25
    JackTan25 committed Oct 28, 2023
    Configuration menu
    Copy the full SHA
    2ba7b7d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 28, 2023
    Configuration menu
    Copy the full SHA
    9eebe16 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2023

  1. fix bugs

    @JackTan25
    JackTan25 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    4e168db View commit details
    Browse the repository at this point in the history

  2. fix check and clean codes

    @JackTan25
    JackTan25 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    73e6210 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 29, 2023
    Configuration menu
    Copy the full SHA
    6ffdee9 View commit details
    Browse the repository at this point in the history

  4. fix check

    @JackTan25
    JackTan25 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    1930589 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'distributed_merge_into' of https://github.com/JackTan25… 

    …/databend into distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    f9200b4 View commit details
    Browse the repository at this point in the history

  6. add more tests

    @JackTan25
    JackTan25 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    2236178 View commit details
    Browse the repository at this point in the history

  7. fix flaky

    @JackTan25
    JackTan25 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    8f1739d View commit details
    Browse the repository at this point in the history

  8. fix test result

    @JackTan25
    JackTan25 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    68aa9be View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 30, 2023
    Configuration menu
    Copy the full SHA
    2da2a21 View commit details
    Browse the repository at this point in the history

  2. fix order

    @JackTan25
    JackTan25 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    d7fc27b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'distributed_merge_into' of https://github.com/JackTan25… 

    …/databend into distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    4afa8a2 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 30, 2023
    Configuration menu
    Copy the full SHA
    d1150e6 View commit details
    Browse the repository at this point in the history

  5. clean codes

    @JackTan25
    JackTan25 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    edffda8 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'distributed_merge_into' of https://github.com/JackTan25… 

    …/databend into distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    7796bdf View commit details
    Browse the repository at this point in the history

  7. remove local builder branch

    @JackTan25
    JackTan25 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    deb4d1f View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 30, 2023
    Configuration menu
    Copy the full SHA
    c646f3b View commit details
    Browse the repository at this point in the history

  9. refactor logic

    @JackTan25
    JackTan25 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    9fcf4b4 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'distributed_merge_into' of https://github.com/JackTan25… 

    …/databend into distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    30f5b3b View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2023

  1. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 31, 2023
    Configuration menu
    Copy the full SHA
    fa3e09e View commit details
    Browse the repository at this point in the history

  2. clean codes

    @JackTan25
    JackTan25 committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    cb08da0 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'distributed_merge_into' of https://github.com/JackTan25… 

    …/databend into distributed_merge_into
    @JackTan25
    JackTan25 committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    269c0de View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 31, 2023
    Configuration menu
    Copy the full SHA
    60fcaee View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into distributed_merge_into

    @JackTan25
    JackTan25 authored Oct 31, 2023
    Configuration menu
    Copy the full SHA
    4defaec View commit details
    Browse the repository at this point in the history