Beanstalk support in WPA #69
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…oxMessagesNotVisible is greater than 0
Old format: bs|test-tube|beanstalkd|11300 New format: beanstalk://beanstalkd:11300/test-tube
Using standard url format for beanstalk queue uri
Old format: bs|test-tube|beanstalkd|11300 New format: beanstalk://beanstalkd:11300/test-tube
…pod-autoscaler into feature/beanstalk
…oxMessagesNotVisible is greater than 0
Old format: bs|test-tube|beanstalkd|11300 New format: beanstalk://beanstalkd:11300/test-tube
…pod-autoscaler into feature/beanstalk
alok87
commented
Jun 12, 2020
There was a problem hiding this comment.
UPDATE: fixed in 8153761
There is a concurrent write issue, that needs to be fixed.
fatal error: concurrent map read and map write
goroutine 2493 [running]:
runtime.throw(0x1679af0, 0x21)
/usr/local/go/src/runtime/panic.go:1116 +0x72 fp=0xc001224c28 sp=0xc001224bf8 pc=0x433912
runtime.mapaccess1_faststr(0x149e320, 0xc000243290, 0xc0009a6a00, 0x48, 0x460010)
/usr/local/go/src/runtime/map_faststr.go:21 +0x43c fp=0xc001224c98 sp=0xc001224c28 pc=0x4129cc
github.com/practo/k8s-worker-pod-autoscaler/pkg/queue.(*Beanstalk).getBeanstalkClient(0xc000131e80, 0xc0009a6a00, 0x48, 0xc0003a170e, 0x2, 0x2)
/src/pkg/queue/beanstalk.go:48 +0x59 fp=0xc001224d10 sp=0xc001224c98 pc=0x134fb49
github.com/practo/k8s-worker-pod-autoscaler/pkg/queue.(*Beanstalk).receiveQueueLength(0xc000131e80, 0xc0009a6a00, 0x48, 0x5c0000000040505d, 0xc001224dc8, 0xc001224dc8)
/src/pkg/queue/beanstalk.go:72 +0x43 fp=0xc001224d70 sp=0xc001224d10 pc=0x134fde3
github.com/practo/k8s-worker-pod-autoscaler/pkg/queue.(*Beanstalk).poll(0xc000131e80, 0xc0025e2720, 0x2f, 0xc0009a6a21, 0x27, 0xc000d3f680, 0x9, 0xc0009a6a00, 0x48, 0xc0009a6a0c, ...)
/src/pkg/queue/beanstalk.go:138 +0x49 fp=0xc001224dd8 sp=0xc001224d70 pc=0x1350329
github.com/practo/k8s-worker-pod-autoscaler/pkg/queue.(*Poller).runPollThread(0xc0002433e0, 0xc0025e2720, 0x2f)
/src/pkg/queue/poller.go:46 +0x88 fp=0xc001224fc8 sp=0xc001224dd8 pc=0x13504e8
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc001224fd0 sp=0xc001224fc8 pc=0x4634e1
created by github.com/practo/k8s-worker-pod-autoscaler/pkg/queue.(*Poller).Run
/src/pkg/queue/poller.go:93 +0x1db
map only supports concurrent reads, but not concurrent read and write. sync.Map was used to solve this which supports this. https://golang.org/doc/faq#atomic_maps https://golang.org/pkg/sync/ #69 (review)
Any changes made to sqs should not go with this PR
alok87
force-pushed
the
beanstalk
branch
2 times, most recently
from
79434d6 to
ff981fb
Compare
Reason: Pave the way for unit tests
Beanstalk queues are only present when there are jobs. This behaviour is one of the behaviours of beanstalk which is not same as SQS.
Queue: Beanstalk tests This covers most of the beanstalk testing scenarios
* master: Error instead of fatal, and log infor of min and maxDisruptable (#72)
|
Fixed in 2f6a0c9 |
SQS poller should operate on only sqs WPA resources and same goes for beanstalk. Also change the default to start both. #69 (comment)
justjkk
approved these changes
Jun 19, 2020
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Moved from #57
Changes made on top of #57
ok github.com/practo/k8s-worker-pod-autoscaler/pkg/queue 43.028s coverage: 41.3% of statements@sushant-115