Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dbnode] AggregateTiles RPC - minimal E2E flow #2466

Merged
merged 36 commits into from
Aug 19, 2020
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
3aa84cb
[dbnode] A noop AggregateTiles thrift RPC
linasm Jul 10, 2020
be16984
Add AggregateTilesRequest.rangeType
linasm Jul 10, 2020
5a4a6e7
sourceNameSpace / targetNameSpace
linasm Jul 10, 2020
663c201
Drop AggregateTilesRequest.shardId
linasm Jul 10, 2020
1a87fd3
A partial implementation of AggregateTiles
linasm Jul 13, 2020
ac1bc66
Open DataFileSetReader and iterate through it
linasm Jul 14, 2020
c150858
Decompress the data read
linasm Jul 14, 2020
aa9617b
Add explicit FileSetType
linasm Jul 14, 2020
76dc3e2
Remove dbShard.TagsFromSeriesID
linasm Jul 15, 2020
fd222ef
Regenerate mocks
linasm Jul 15, 2020
eeab8fd
Unit tests
linasm Jul 15, 2020
9a6dbc0
Mockgen
linasm Jul 16, 2020
20c07e1
Fix test
linasm Jul 16, 2020
efa16e7
Resurrect rpc_mock.go
linasm Jul 17, 2020
13fd5e1
Remove accidentally committed files
linasm Jul 17, 2020
7d03515
Trigger build
linasm Jul 17, 2020
bd248a0
Add step parameter
linasm Jul 20, 2020
7359124
Write aggregated data to other namespace
gediminasgu Jul 20, 2020
573d4a1
Merge branch 'gg/large-tiles-aggregation-write' into linas/large-tile…
linasm Jul 21, 2020
12ebe54
Fix tests
linasm Jul 21, 2020
3ef6920
Introduced AggregateTilesOptions
linasm Jul 21, 2020
3e4de5c
Minor improvements
linasm Jul 21, 2020
5579b76
Cleanup
linasm Jul 21, 2020
c1de7cd
Compute and validate block time frames
linasm Jul 23, 2020
51c43b9
Merge branch 'large-tiles-aggregation' into linas/large-tiles-aggrega…
linasm Jul 23, 2020
81fc3ce
[dbnode] Integration test for large tiles (#2478)
gediminasgu Jul 23, 2020
07a091c
Return processedBlockCount from AggregateTiles
linasm Jul 24, 2020
f94ca76
Improve error handling
linasm Jul 24, 2020
0075388
Validate AggregateTilesOptions
linasm Jul 24, 2020
5c85771
Unnest read locks
linasm Jul 24, 2020
d22f278
Use default instead of constant
linasm Jul 24, 2020
66a37a1
Fix test
linasm Jul 24, 2020
f6a4d43
minor refactoring
gediminasgu Jul 24, 2020
9dddd50
Merge branch 'large-tiles-aggregation' into linas/large-tiles-aggrega…
linasm Aug 5, 2020
d336b81
Add HandleCounterResets to AggregateTilesOptions
linasm Aug 7, 2020
aecba62
Merge branch 'large-tiles-aggregation' into linas/large-tiles-aggrega…
linasm Aug 11, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions src/dbnode/generated/thrift/rpc.thrift
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ service Node {
void repair() throws (1: Error err)
TruncateResult truncate(1: TruncateRequest req) throws (1: Error err)

AggregateTilesResult aggregateTiles(1: AggregateTilesRequest req) throws (1: Error err)

// Management endpoints
NodeHealthResult health() throws (1: Error err)
// NB: bootstrapped is for use with cluster management tools like k8s.
Expand Down Expand Up @@ -491,6 +493,19 @@ struct Query {
7: optional FieldQuery field
}

struct AggregateTilesRequest {
1: required string sourceNameSpace
2: required string targetNameSpace
3: required i64 rangeStart
4: required i64 rangeEnd
5: required string step
6: bool removeResets
7: optional TimeType rangeType = TimeType.UNIX_SECONDS
}

struct AggregateTilesResult {
linasm marked this conversation as resolved.
Show resolved Hide resolved
}

struct DebugProfileStartRequest {
1: required string name
2: required string filePathTemplate
Expand Down
Loading