chore: rebase main #16
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chore/remove-struct-size-assertion: Remove unit tests for parquet_meta_size function in cache_size.rs
* basic impl Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * implement physical rule Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * feat: install windowed sort physical rule and optimize partition ranges Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * add logs and sqlness test Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * feat: introduce PartSortExec for partitioned sorting Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * tune exec nodes' properties and metrics Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * clean up Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix typo Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * debug: add more info on very wrong * debug: also print overlap ranges * feat: add check when emit PartSort Stream * dbg: info on overlap working range * feat: check batch range is inside part range * set distinguish partition range param Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * chore: more logs * update sqlness Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * tune optimizer Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * clean up Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix lints Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix windowed sort rule Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix: early terminate sort stream Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * chore: remove min/max check * chore: remove unused windowed_sort module, uuid feature and refactor region_scanner to synchronous Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * chore: print more fuzz log * chore: more log * fix: part sort should skip empty part * chore: remove insert logs * tests: empty PartitionRange * refactor: testcase * docs: update comment&tests: all empty * ci: enlarge etcd cpu limit --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com> Co-authored-by: discord9 <discord9@163.com> Co-authored-by: evenyag <realevenyag@gmail.com>
…Mito (#4887) feat(index): support building inverted index for the field column Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
chore: install numpy in CI
* chore: update greptime-proto * feat: add json datatype for grpc protocol
* refactor: json type update * test: update test * fix: convert when needed * revert: leave sqlness tests unchanged * fix: fmt * refactor: just refactor * Apply suggestions from code review Co-authored-by: Weny Xu <wenymedia@gmail.com> * refactor: parse jsonb first * test: add bad cases * Update src/datatypes/src/vectors/binary.rs Co-authored-by: Weny Xu <wenymedia@gmail.com> * fix: fmt * fix: fix clippy/check --------- Co-authored-by: Weny Xu <wenymedia@gmail.com>
* test: add fuzz tests for migrate metric regions * test: insert values before migrating metric region * feat: correct table num * chore: apply suggestions from CR
* feat/alter-ttl: Update greptime-proto source and add ChangeTableOptions handling - Change greptime-proto source repository and revision in Cargo.lock and Cargo.toml - Implement handling for ChangeTableOptions in grpc-expr and meta modules - Add support for parsing and applying region option changes in mito2 - Introduce new error type for invalid change table option requests - Add humantime dependency to store-api - Fix SQL syntax in tests for changing column types * chore: remove write buffer size option handling since we don't support specifying write_buffer_size for single table or region * persist ttl to manifest * chore: add sqlness * fix: sqlness * fix: typo and toml format * fix: tests * update: change alter syntax * feat/alter-ttl: Add Clone trait to RegionFlushRequest and remove redundant Default derive in region_request.rs. * feat/alter-ttl: Refactor code to replace 'ChangeTableOption' with 'ChangeRegionOption' and handle TTL as a region option • Rename ChangeTableOption to ChangeRegionOption across various files. • Update AlterKind::ChangeTableOptions to AlterKind::ChangeRegionOptions. • Modify TTL handling to treat '0d' as None for TTL in table options. • Adjust related function names and comments to reflect the change from table to region options. • Include test case updates to verify the new TTL handling behavior. * chore: update format * refactor: update region options in DatanodeTableValue * feat/alter-ttl: Remove TTL handling from RegionManifest and related structures - Eliminate TTL fields from `RegionManifest`, `RegionChange`, and associated handling logic. - Update tests and checksums to reflect removal of TTL. - Refactor `RegionOpener` and `handle_alter` to adjust to TTL removal. - Simplify `RegionChangeResult` by replacing `change` with `new_meta`. * chore: fmt * remove useless delete op * feat/alter-ttl: Updated Cargo.lock and gRPC expression Cargo.toml to include store-api dependency. Refactored alter.rs to use ChangeOption from store-api instead of ChangeTableOptionRequest. Adjusted error handling in error.rs to use MetadataError. Modified handle_alter.rs to handle TTL changes with ChangeOption. Simplified region_request.rs by replacing ChangeRegionOption with ChangeOption and removing redundant code. Removed UnsupportedTableOptionChange error in table/src/error.rs. Updated metadata.rs to use ChangeOption for table options. Removed ChangeTableOptionRequest enum and related conversion code from requests.rs. * feat/alter-ttl: Update greptime-proto dependency to revision 53ab9a9553 * chore: format code * chore: update greptime-proto
…4875) * feat: dynamic definition of plugin options * rebase * revert * fix ci
…cs table (#4909) * feat: adds index size to region statistics * feat: adds the number of rows for region statistics * test: adds sqlness test for region_statistics * fix: test
* chore: provide more info in check message * chore: set timeout to 240s --------- Co-authored-by: WenyXu <wenymedia@gmail.com>
* feat: implement parse_query api * chore: switch to upstream * fix: add post method for parse_query * chore: bump promql-parser * test: use latest promql ast serialization
* feat: add an iter to prune by time range * feat: filter rows from mem range
* add RegionScanner::metadata Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * skip PartSort when there is no tag column Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * add more sqlness test Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * handle desc Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix: should keep part sort on DESC Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix clippy Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* refactor: simplify WeightedChoose * chore: remove unused errors
* feat: get part range min-max from cache for unordered scan * feat: seq scan push row groups if num_row_groups > 0 * test: test split * feat: update comment * test: fix split test * refactor: rename get meta data method
* chore: tiny doc about change log level * chore: per review * chore
* feat: simple limit impl in PartSort Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix: update time_index method to return a non-optional String Co-authored-by: Yingwen <realevenyag@gmail.com> Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * use builtin limit Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * add more info to analyze display Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * update sqlness Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com> Co-authored-by: Yingwen <realevenyag@gmail.com>
fix/database-base-ttl: Fix typos in comments and function names across multiple modules - Correct spelling of 'parallelism' in region_server, engine, and scan_region modules - Amend typo in TODO comment from 'persisent' to 'persistent' in server module - Update incorrect test query from 'versiona' to 'version' in federated module tests Co-authored-by: Lei, HUANG <mrsatangel@gmail.com>
* feat: heartbeat_flush_threshold * chore: rename to flush_stats_factor * Update src/meta-srv/src/handler/collect_stats_handler.rs
* chore: update default cache size to 1Gib for object storage read/write cache * feat: update docs * fix: test
* refactor: json type update * test: update test * fix: convert when needed * revert: leave sqlness tests unchanged * fix: fmt * refactor: just refactor * Apply suggestions from code review Co-authored-by: Weny Xu <wenymedia@gmail.com> * refactor: parse jsonb first * test: add bad cases * Update src/datatypes/src/vectors/binary.rs Co-authored-by: Weny Xu <wenymedia@gmail.com> * fix: fmt * fix: fix clippy/check * fix: corrupted jsonb panic * chore(deps): change to rev --------- Co-authored-by: Weny Xu <wenymedia@gmail.com>
* fix: data_length, index_length, table_rows in tables * feat: table stats only works for mito engine currently * fix: tests * fix: typo * chore: log error when region_stats fails
* feat: support to insert json data via grpc protocol * chore: handle error * feat: introduce `prepare_rows` * chore: fmt toml * test: add row deletion test * test: fix unit test * chore: remove log * chore: apply suggestions from CR
refactor: alter parser
* main: Add common-meta dependency and implement SchemaMetadataManager - Introduce `common-meta` as a new dependency in `mito2`. - Implement `SchemaMetadataManager` for managing schema-level metadata. - Update `DatanodeBuilder` and `MitoEngine` to pass `KvBackendRef` for schema metadata management. - Add `SchemaMetadataManager` to `RegionWorkerLoop` for compaction handling. - Include `SchemaNameKey` usage in compaction-related code. - Add `database_metadata_manager` module with `SchemaMetadataManager` struct and associated logic. * fix/database-base-ttl: Refactor metadata management and update compaction logic - Remove `database_metadata_manager` and introduce `schema_metadata_manager` - Update compaction logic to handle TTL based on schema metadata - Adjust tests to use `schema_metadata_manager` for setting up schema options - Fix engine creation in tests to pass `kv_backend` explicitly - Remove unused imports and apply minor code cleanups * fix/database-base-ttl: Extend CREATE TABLE LIKE to inherit schema options - Implement inheritance of database level options for CREATE TABLE LIKE - Add schema options to SHOW CREATE TABLE output - Refactor create_table_stmt to include schema_options in SQL generation - Update error handling to include TableMetadataManagerSnafu * fix/database-base-ttl: Refactor error handling and remove schema dependency in table creation - Replace expect with the ? operator for error handling in open_compaction_region - Simplify create_logical_tables by removing catalog and schema name parameters - Remove unnecessary schema retrieval and merging of schema options in create_table_info - Clean up unused imports and redundant code * fix/database-base-ttl: Refactor error handling and update documentation comments - Update comment to reflect retrieval of schema options instead of metadata - Introduce new error type `GetSchemaMetadataSnafu` for schema metadata retrieval failures - Implement error handling for schema metadata retrieval in `find_ttl` function * fix: toml * fix/database-base-ttl: Refactor SchemaMetadataManager and adjust Cargo.toml dependencies - Remove unused imports in schema_metadata_manager.rs - Add conditional compilation for SchemaMetadataManager::new - Update Cargo.toml to remove "testing" feature from common-meta dependency in main section and add it to dev-dependencies * fix/database-base-ttl: Fix typos in comments and function names across multiple modules - Correct spelling of 'parallelism' in region_server, engine, and scan_region modules - Amend typo in TODO comment from 'persisent' to 'persistent' in server module - Update incorrect test query from 'versiona' to 'version' in federated module tests * fix/database-base-ttl: Add schema existence check in StatementExecutor for CREATE TABLE operation * fix/database-base-ttl: Add warning log for failed TTL retrieval in compaction region open function * fix/database-base-ttl: Refactor to use SchemaMetadataManagerRef in Datanode and MitoEngine - Replace KvBackendRef with SchemaMetadataManagerRef across various components. - Update DatanodeBuilder and MitoEngine to pass SchemaMetadataManagerRef instead of KvBackendRef. - Adjust test cases to use get_schema_metadata_manager method for consistency.
* chore: minor refactor for weighted choose * chore: by comment, remove the fast path of choose_multiple
* chore: add type conversion for array types * feat: add h3_cells_contains * refactor: resolve lint issues * feat: add sphere distance function * feat: euclidean distance between h3 centroids * test: round float number * feat: add more geospatial functions * test: add tests for geometry functions * refactor: move wkt function to dedicated module * feat: add st_area * refactor: only allow sphere distance between points
chore: fix typos in change log level
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the GreptimeDB CLA
What's changed and what's your intention?
PLEASE DO NOT LEAVE THIS EMPTY !!!
Please explain IN DETAIL what the changes are in this PR and why they are needed:
Checklist
Refer to a related PR or issue link (optional)