Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify M3DB config #1371

Merged
merged 54 commits into from
Feb 14, 2019
Merged

Simplify M3DB config #1371

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
a32c9f9
wip
Feb 12, 2019
f221c68
pooling passing
Feb 12, 2019
108904f
delete test yolo shard
Feb 12, 2019
55a4eec
rename method
Feb 12, 2019
6f186e5
Revert "rename method"
Feb 12, 2019
41b8fb0
Add default pool policy
Feb 12, 2019
84483fe
fix broken test
Feb 12, 2019
26f150c
valid refills
Feb 12, 2019
c54c3f4
delete unused var block
Feb 12, 2019
de21c88
fix bug in parsing values and fix tests
Feb 12, 2019
d1a29ed
fix broken config and test
Feb 12, 2019
6725b03
default values for filesystem;
Feb 12, 2019
59ba7f9
Fix more config stuff
Feb 12, 2019
b85a430
remove now-default values from config yamls
Feb 12, 2019
1b55e90
fix validaiton
Feb 12, 2019
828878d
make repair config optional for now
Feb 12, 2019
dd49fe2
dont run repair code in server.go
Feb 12, 2019
e844550
update pooling
Feb 12, 2019
c1c1d2a
restructure constants
Feb 12, 2019
31050b5
remove TODO
Feb 12, 2019
b2f6ea2
fix broken config
Feb 12, 2019
3d486f0
fix typo
Feb 12, 2019
40ffc3f
set correct defaults for m3db client
Feb 12, 2019
f09c896
remove client config from example YAMLs
Feb 12, 2019
a7137f8
delete more client config from YAMLs
Feb 12, 2019
e9d3be4
chain function calls
Feb 12, 2019
95e1790
fix broken test
Feb 12, 2019
f0b8f48
fix lint issue
Feb 12, 2019
3e2bc91
fix compilation
Feb 12, 2019
4d34b8e
set no retries for client tests
Feb 12, 2019
efdbb90
Validate configs
Feb 13, 2019
86a09a6
fix broken tests
Feb 13, 2019
5afac43
fix tests
Feb 13, 2019
f9fab84
wip
Feb 13, 2019
b4fe06b
fix broken test
Feb 13, 2019
9bad14f
remove bounds checks
Feb 13, 2019
5fffe3c
Add commented all-config file
Feb 13, 2019
b031b75
fix broken test
Feb 13, 2019
2b0d2f2
add pool validation
Feb 13, 2019
bb5b371
more pooling config nonsense
Feb 13, 2019
7cc246b
rewrite everything
Feb 13, 2019
06211ca
use constants
Feb 13, 2019
6d152bf
Fix comment explaining how to add new pool
Feb 13, 2019
067dbee
add constant for default pooling type
Feb 13, 2019
2559486
fix error message
Feb 13, 2019
44ae437
add comment
Feb 13, 2019
1ef370b
fix broken test
Feb 13, 2019
1ff90a3
use validate method on client
Feb 14, 2019
088d102
fix broken test
Feb 14, 2019
93ac499
fix integration tests timing out
Feb 14, 2019
521d31c
fix broken test
Feb 14, 2019
d2dffb6
fix last broken test
Feb 14, 2019
5fd8cae
fix last broken test for eral
Feb 14, 2019
1c955dd
remove print statements
Feb 14, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 0 additions & 127 deletions kube/m3dbnode-configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,21 +47,6 @@ data:
client:
writeConsistencyLevel: majority
readConsistencyLevel: unstrict_majority
writeTimeout: 10s
fetchTimeout: 15s
connectTimeout: 20s
writeRetry:
initialBackoff: 500ms
backoffFactor: 3
maxRetries: 2
jitter: true
fetchRetry:
initialBackoff: 500ms
backoffFactor: 2
maxRetries: 3
jitter: true
backgroundHealthCheckFailLimit: 4
backgroundHealthCheckFailThrottleFactor: 0.5

gcPercentage: 100

Expand All @@ -88,119 +73,7 @@ data:

fs:
filePathPrefix: /var/lib/m3db
writeBufferSize: 65536
dataReadBufferSize: 65536
infoReadBufferSize: 128
seekReadBufferSize: 4096
throughputLimitMbps: 100.0
throughputCheckEvery: 128

repair:
enabled: false
interval: 2h
offset: 30m
jitter: 1h
throttle: 2m
checkInterval: 1m

pooling:
blockAllocSize: 16
type: simple
seriesPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
blockPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
encoderPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
closersPool:
size: 104857
lowWatermark: 0.7
highWatermark: 1.0
contextPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
segmentReaderPool:
size: 16384
lowWatermark: 0.7
highWatermark: 1.0
iteratorPool:
size: 2048
lowWatermark: 0.7
highWatermark: 1.0
fetchBlockMetadataResultsPool:
size: 65536
capacity: 32
lowWatermark: 0.7
highWatermark: 1.0
fetchBlocksMetadataResultsPool:
size: 32
capacity: 4096
lowWatermark: 0.7
highWatermark: 1.0
hostBlockMetadataSlicePool:
size: 131072
capacity: 3
lowWatermark: 0.7
highWatermark: 1.0
blockMetadataPool:
size: 65536
lowWatermark: 0.7
highWatermark: 1.0
blockMetadataSlicePool:
size: 65536
capacity: 32
lowWatermark: 0.7
highWatermark: 1.0
blocksMetadataPool:
size: 65536
lowWatermark: 0.7
highWatermark: 1.0
blocksMetadataSlicePool:
size: 32
capacity: 4096
lowWatermark: 0.7
highWatermark: 1.0
identifierPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
bytesPool:
buckets:
- capacity: 16
size: 524288
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 32
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 64
size: 131072
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 128
size: 65536
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 256
size: 65536
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 1440
size: 16384
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 4096
size: 8192
lowWatermark: 0.7
highWatermark: 1.0
config:
service:
env: default_env
Expand Down
17 changes: 0 additions & 17 deletions scripts/development/m3_stack/m3coordinator.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,23 +32,6 @@ clusters:
- zone: embedded
endpoints:
- m3db_seed:2379
writeConsistencyLevel: majority
readConsistencyLevel: unstrict_majority
writeTimeout: 10s
fetchTimeout: 15s
connectTimeout: 20s
writeRetry:
initialBackoff: 500ms
backoffFactor: 3
maxRetries: 2
jitter: true
fetchRetry:
initialBackoff: 500ms
backoffFactor: 2
maxRetries: 3
jitter: true
backgroundHealthCheckFailLimit: 4
backgroundHealthCheckFailThrottleFactor: 0.5

ingest:
ingester:
Expand Down
128 changes: 0 additions & 128 deletions scripts/development/m3_stack/m3dbnode.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,21 +22,6 @@ db:
client:
writeConsistencyLevel: majority
readConsistencyLevel: unstrict_majority
writeTimeout: 10s
fetchTimeout: 15s
connectTimeout: 20s
writeRetry:
initialBackoff: 500ms
backoffFactor: 3
maxRetries: 2
jitter: true
fetchRetry:
initialBackoff: 500ms
backoffFactor: 2
maxRetries: 3
jitter: true
backgroundHealthCheckFailLimit: 4
backgroundHealthCheckFailThrottleFactor: 0.5

gcPercentage: 100

Expand Down Expand Up @@ -67,119 +52,6 @@ db:

fs:
filePathPrefix: /var/lib/m3db
writeBufferSize: 65536
dataReadBufferSize: 65536
infoReadBufferSize: 128
seekReadBufferSize: 4096
throughputLimitMbps: 100.0
throughputCheckEvery: 128

repair:
enabled: false
interval: 2h
offset: 30m
jitter: 1h
throttle: 2m
checkInterval: 1m

pooling:
blockAllocSize: 16
type: simple
seriesPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
blockPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
encoderPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
closersPool:
size: 104857
lowWatermark: 0.7
highWatermark: 1.0
contextPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
segmentReaderPool:
size: 16384
lowWatermark: 0.7
highWatermark: 1.0
iteratorPool:
size: 2048
lowWatermark: 0.7
highWatermark: 1.0
fetchBlockMetadataResultsPool:
size: 65536
capacity: 32
lowWatermark: 0.7
highWatermark: 1.0
fetchBlocksMetadataResultsPool:
size: 32
capacity: 4096
lowWatermark: 0.7
highWatermark: 1.0
hostBlockMetadataSlicePool:
size: 131072
capacity: 3
lowWatermark: 0.7
highWatermark: 1.0
blockMetadataPool:
size: 65536
lowWatermark: 0.7
highWatermark: 1.0
blockMetadataSlicePool:
size: 65536
capacity: 32
lowWatermark: 0.7
highWatermark: 1.0
blocksMetadataPool:
size: 65536
lowWatermark: 0.7
highWatermark: 1.0
blocksMetadataSlicePool:
size: 32
capacity: 4096
lowWatermark: 0.7
highWatermark: 1.0
identifierPool:
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
bytesPool:
buckets:
- capacity: 16
size: 524288
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 32
size: 262144
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 64
size: 131072
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 128
size: 65536
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 256
size: 65536
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 1440
size: 16384
lowWatermark: 0.7
highWatermark: 1.0
- capacity: 4096
size: 8192
lowWatermark: 0.7
highWatermark: 1.0

config:
service:
Expand Down
15 changes: 0 additions & 15 deletions scripts/docker-integration-tests/carbon/m3coordinator.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,21 +34,6 @@ clusters:
- dbnode01:2379
writeConsistencyLevel: majority
readConsistencyLevel: unstrict_majority
writeTimeout: 10s
fetchTimeout: 15s
connectTimeout: 20s
writeRetry:
initialBackoff: 500ms
backoffFactor: 3
maxRetries: 2
jitter: true
fetchRetry:
initialBackoff: 500ms
backoffFactor: 2
maxRetries: 3
jitter: true
backgroundHealthCheckFailLimit: 4
backgroundHealthCheckFailThrottleFactor: 0.5

carbon:
ingester:
Expand Down
15 changes: 0 additions & 15 deletions scripts/docker-integration-tests/prometheus/m3coordinator.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,3 @@ clusters:
- dbnode01:2379
writeConsistencyLevel: majority
readConsistencyLevel: unstrict_majority
writeTimeout: 10s
fetchTimeout: 15s
connectTimeout: 20s
writeRetry:
initialBackoff: 500ms
backoffFactor: 3
maxRetries: 2
jitter: true
fetchRetry:
initialBackoff: 500ms
backoffFactor: 2
maxRetries: 3
jitter: true
backgroundHealthCheckFailLimit: 4
backgroundHealthCheckFailThrottleFactor: 0.5
Loading