Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support NebulaGraph #5116

Merged
merged 1 commit into from
Oct 16, 2024
Merged

feat: Support NebulaGraph #5116

merged 1 commit into from
Oct 16, 2024

Conversation

GG2002
Copy link
Contributor

@GG2002 GG2002 commented Sep 12, 2024

Which issue does this PR close?

Closes #4553.

Rationale for this change

What changes are included in this PR?

Support read, write, list, delete ops on NebulaGraph.

To access NebulaGraph, users must provide the username and password for NebulaGraph, as well as specify the space, tag, and the key field and value field of the tag.

Since each vertex in NebulaGraph must have a unique ID(called VID), but NebulaGraph lacks a self-incrementing ID, the unique IDs are generated using the Snowflake algorithm. Additionally, the set operation involves first delete existing original vertex, followed by inserting a new vertex.

NebulaGraph does not support the Blob type, hence Base64+string type is employed for file access. If it's supported in the future, just remove Base64.

Are there any user-facing changes?

No(?)

@GG2002
Copy link
Contributor Author

GG2002 commented Sep 18, 2024

Test results are as follows:

$ OPENDAL_TEST=nebulagraph cargo test behavior --features tests -- --nocapture --test-threads=20
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.32s
     Running unittests src/lib.rs (target/debug/deps/opendal-20c211e669cef355)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 128 filtered out; finished in 0.00s

     Running tests/behavior/main.rs (target/debug/deps/behavior-0bda2893f3ecdc95)

running 90 tests
test behavior::test_delete_with_version                       ... ok
test behavior::test_delete_with_not_existing_version          ... ok
test behavior::test_delete_not_existing                       ... ok
test behavior::test_list_with_start_after                     ... ok
test behavior::test_check                                     ... ok
test behavior::test_list_non_exist_dir                        ... ok
test behavior::test_delete_empty_dir                          ... ok
test behavior::test_create_dir                                ... ok
test behavior::test_create_dir_existing                       ... ok
test behavior::test_list_files_with_version                   ... ok
test behavior::test_list_with_version_and_limit               ... ok
test behavior::test_list_with_version_and_start_after         ... ok
test behavior::test_list_root_with_recursive                  ... ok
test behavior::test_list_sub_dir                              ... ok
test behavior::test_list_file_with_recursive                  ... ok
test behavior::test_list_empty_dir                            ... ok
test behavior::test_read_with_if_match                        ... ok
test behavior::test_read_with_if_none_match                   ... ok
test behavior::test_list_dir                                  ... ok
test behavior::test_read_not_exist                            ... ok
test behavior::test_read_with_override_cache_control          ... ok
test behavior::test_read_with_override_content_disposition    ... ok
test behavior::test_read_with_override_content_type           ... ok
test behavior::test_read_with_version                         ... ok
test behavior::test_read_with_not_existing_version            ... ok
test behavior::test_read_with_dir_path                        ... ok
test behavior::test_stat_dir                                  ... ok
test behavior::test_list_nested_dir                           ... ok
test behavior::test_list_dir_with_recursive_no_trailing_slash ... ok
test behavior::test_list_dir_with_recursive                   ... ok
test behavior::test_stat_not_exist                            ... ok
test behavior::test_stat_with_if_match                        ... ok
test behavior::test_stat_with_if_none_match                   ... ok
test behavior::test_stat_with_override_cache_control          ... ok
test behavior::test_stat_with_override_content_disposition    ... ok
test behavior::test_stat_with_override_content_type           ... ok
test behavior::test_stat_root                                 ... ok
test behavior::test_stat_with_version                         ... ok
test behavior::stat_with_not_existing_version                 ... ok
test behavior::test_remove_all                                ... ok
test behavior::test_write_only                                ... ok
test behavior::test_write_with_dir_path                       ... ok
test behavior::test_write_with_empty_content                  ... ok
test behavior::test_write_with_cache_control                  ... ok
test behavior::test_write_with_content_type                   ... ok
test behavior::test_write_with_content_disposition            ... ok
test behavior::test_write_with_user_metadata                  ... ok
test behavior::test_writer_write                              ... ok
test behavior::test_stat_nested_parent_dir                    ... ok
test behavior::test_writer_write_with_concurrent              ... ok
test behavior::test_writer_sink                               ... ok
test behavior::test_writer_sink_with_concurrent               ... ok
test behavior::test_list_rich_dir                             ... ok
test behavior::test_writer_abort                              ... ok
test behavior::test_writer_futures_copy                       ... ok
test behavior::test_writer_futures_copy_with_concurrent       ... ok
test behavior::test_blocking_create_dir                       ... ok
test behavior::test_write_with_special_chars                  ... ok
test behavior::test_blocking_create_dir_existing              ... ok
test behavior::test_blocking_delete_file                      ... ok
test behavior::test_reader                                    ... ok
test behavior::test_writer_abort_with_concurrent              ... ok
test behavior::test_read_range                                ... ok
test behavior::test_blocking_read_not_exist                   ... ok
test behavior::test_read_with_special_chars                   ... ok
test behavior::test_blocking_stat_dir                         ... ok
test behavior::test_list_dir_with_file_path                   ... ok
test behavior::test_blocking_stat_not_exist                   ... ok
test behavior::test_blocking_stat_with_special_chars          ... ok
test behavior::test_list_prefix                               ... ok
test behavior::test_blocking_write_with_dir_path              ... ok
test behavior::test_blocking_stat_file                        ... ok
test behavior::test_stat_file                                 ... ok
test behavior::test_remove_one_file                           ... ok
test behavior::test_blocking_write_file                       ... ok
test behavior::test_blocking_remove_all_basic                 ... ok
test behavior::test_stat_not_cleaned_path                     ... ok
test behavior::test_delete_with_special_chars                 ... ok
test behavior::test_delete_file                               ... ok
test behavior::test_read_full                                 ... ok
test behavior::test_list_dir_with_metakey                     ... ok
test behavior::test_stat_with_special_chars                   ... ok
test behavior::test_blocking_read_full                        ... ok
test behavior::test_blocking_remove_one_file                  ... ok
test behavior::test_blocking_read_range                       ... ok
test behavior::test_list_dir_with_metakey_complete            ... ok
test behavior::test_delete_stream                             ... ok
test behavior::test_blocking_write_with_special_chars         ... ok
test behavior::test_remove_all_basic                          ... ok
test behavior::test_writer_write_with_overwrite               ... ok

test result: ok. 90 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.56s

core/Cargo.lock Outdated Show resolved Hide resolved
core/Cargo.toml Outdated Show resolved Hide resolved
core/Cargo.toml Outdated Show resolved Hide resolved
@Xuanwo
Copy link
Member

Xuanwo commented Sep 23, 2024

Hi, @GG2002. It might take us a while to complete the communication. Would you like to split this PR into parts so we can merge some of them first? I believe we can add the builder, config first without introduce the client impl in.

@GG2002
Copy link
Contributor Author

GG2002 commented Sep 23, 2024

Hi, @GG2002. It might take us a while to complete the communication. Would you like to split this PR into parts so we can merge some of them first? I believe we can add the builder, config first without introduce the client impl in.

It's ok.

@GG2002 GG2002 force-pushed the nebulagraph-support branch 2 times, most recently from bcc646c to 5388a61 Compare October 11, 2024 15:20
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM, thank you @GG2002 a lot for working on this!

core/src/types/scheme.rs Outdated Show resolved Hide resolved
core/src/types/scheme.rs Outdated Show resolved Hide resolved
Signed-off-by: feathercyc <feathercyc@163.com>
@GG2002 GG2002 force-pushed the nebulagraph-support branch from 5388a61 to 2bfc4b6 Compare October 16, 2024 12:21
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@Xuanwo Xuanwo merged commit e2be539 into apache:main Oct 16, 2024
233 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for NebulaGraph
2 participants