Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] - Concurrent map read and map write in Scheduler #585

Closed
mikayzo opened this issue Oct 5, 2023 · 0 comments · Fixed by #589
Closed

[BUG] - Concurrent map read and map write in Scheduler #585

mikayzo opened this issue Oct 5, 2023 · 0 comments · Fixed by #589
Labels
bug Something isn't working

Comments

@mikayzo
Copy link

mikayzo commented Oct 5, 2023

Describe the bug

Application gets fatal error with the following message:

fatal error: concurrent map read and map write

goroutine 20038 [running]:
github.com/go-co-op/gocron.(*Scheduler)._jobPresent(0x0?, 0xc0016bddd0?, 0x771d17?)
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/scheduler.go:892 +0xaa
github.com/go-co-op/gocron.(*Scheduler).jobPresent(0xc000140c60?, 0x44c4b4?)
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/scheduler.go:899 +0x37
github.com/go-co-op/gocron.(*Scheduler).scheduleNextRun(0x0?, 0xc001b77680)
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/scheduler.go:205 +0x65
github.com/go-co-op/gocron.(*Scheduler).runContinuous(0xc000140c60, 0xc001b77680)
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/scheduler.go:613 +0x2d
github.com/go-co-op/gocron.(*Scheduler).runContinuous.func1()
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/scheduler.go:646 +0x1c5
created by time.goFunc
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/time/sleep.go:176 +0x32

goroutine 1 [chan receive]:
main.main()
        /Users/myname/company/scripts/myproject/cmd/main.go:117 +0xb2a

goroutine 20 [syscall]:
os/signal.signal_recv()
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/runtime/sigqueue.go:152 +0x2f
os/signal.loop()
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/os/signal/signal_unix.go:23 +0x19
created by os/signal.Notify.func1.1
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/os/signal/signal.go:151 +0x2a

goroutine 36 [select]:
github.com/go-co-op/gocron.(*executor).run(0xc0001ce0a0)
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/executor.go:229 +0xc7
created by github.com/go-co-op/gocron.(*executor).start
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/executor.go:158 +0x20a

goroutine 37 [chan receive]:
example.github.com/myproject/internal/executor.(*Executor).CollectDynamicRerouteResults(0xc000024240)
        /Users/myname/company/scripts/myproject/internal/executor/dynamic_reroute.go:39 +0xdd
created by main.main
        /Users/myname/company/scripts/myproject/cmd/main.go:103 +0x7ca

goroutine 38 [chan receive]:
example.github.com/myproject/internal/monitoring.(*Monitoring).PublishResults(0xc000070600)
        /Users/myname/company/scripts/myproject/internal/monitoring/monitoring.go:25 +0xad
created by main.main
        /Users/myname/company/scripts/myproject/cmd/main.go:104 +0x835

goroutine 12 [IO wait]:
internal/poll.runtime_pollWait(0x4027be2758, 0x72)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0004da080?, 0xc000469cb0?, 0x0)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc0004da080)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/internal/poll/fd_unix.go:614 +0x2bd
net.(*netFD).accept(0xc0004da080)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/net/fd_unix.go:172 +0x35
net.(*TCPListener).accept(0xc001686048)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/net/tcpsock_posix.go:148 +0x25
net.(*TCPListener).Accept(0xc001686048)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/net/tcpsock.go:297 +0x3d
net/http.(*Server).Serve(0xc00168a000, {0xa736f0, 0xc001686048})
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/net/http/server.go:3059 +0x385
net/http.(*Server).ListenAndServe(0xc00168a000)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/net/http/server.go:2988 +0x7d
net/http.ListenAndServe(...)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/net/http/server.go:3242
example.github.com/myproject/internal/api.(*API).Start(0xc000010978)
        /Users/myname/company/scripts/myproject/internal/api/api.go:48 +0x2f1
created by main.main
        /Users/myname/company/scripts/myproject/cmd/main.go:112 +0xa14

goroutine 11 [chan receive]:
example.github.com/myproject/internal/metrics.(*Metrics).Collect(0xc000012068)
        /Users/myname/company/scripts/myproject/internal/metrics/metrics.go:51 +0x111
created by main.main
        /Users/myname/company/scripts/myproject/cmd/main.go:111 +0x9aa

goroutine 20052 [sync.RWMutex.Lock]:
sync.runtime_SemacquireRWMutex(0x85b7e5?, 0xa8?, 0x98dc00?)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/runtime/sema.go:87 +0x26
sync.(*RWMutex).Lock(0x8?)
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/sync/rwmutex.go:152 +0x71
github.com/go-co-op/gocron.(*Scheduler).getCurrentJob(0xc000140c60)
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/scheduler.go:1315 +0xd6
github.com/go-co-op/gocron.(*Scheduler).EveryRandom(0x85e06b?, 0x0?, 0x94b000?)
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/scheduler.go:539 +0x28
example.github.com/myproject/internal/executor.(*Executor).scheduleTask(0xc000024240, {{{0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...})
        /Users/myname/company/scripts/myproject/internal/executor/tasks.go:70 +0x47
example.github.com/myproject/internal/executor.(*Executor).runTask.func2(0xc000024240, 0xc003ce2ee8, {0xa70500, 0xc0034d43f0}, 0x3?, 0xc003ce2c78, 0x0, 0xc003ce2e20, 0xc003ce2c20)
        /Users/myname/company/scripts/myproject/internal/executor/tasks.go:131 +0x5a5
example.github.com/myproject/internal/executor.(*Executor).runTask(_, {{{0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...}, ...)
        /Users/myname/company/scripts/myproject/internal/executor/tasks.go:170 +0x311
reflect.Value.call({0x903040?, 0xc000022d40?, 0x4027f86450?}, {0x9ad087, 0x4}, {0xc0034d40f0, 0x2, 0x0?})
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/reflect/value.go:586 +0xb07
reflect.Value.Call({0x903040?, 0xc000022d40?, 0x10?}, {0xc0034d40f0?, 0x2?, 0x4027fbd5b8?})
        /opt/homebrew/Cellar/go/1.20.2/libexec/src/reflect/value.go:370 +0xbc
github.com/go-co-op/gocron.callJobFuncWithParams({0x903040?, 0xc000022d40?}, {0xc0031f11a0, 0x2, 0xc003a4065c?})
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/gocron.go:118 +0x28a
github.com/go-co-op/gocron.runJob({{0x54, 0xc, 0x6d, 0xc2, 0x60, 0x59, 0x44, 0x57, 0xbc, 0xd1, ...}, ...})
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/executor.go:88 +0x1a5
github.com/go-co-op/gocron.(*jobFunction).singletonRunner(0xc003e13680)
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/executor.go:117 +0x1b8
created by github.com/go-co-op/gocron.(*executor).runJob
        /Users/myname/go/pkg/mod/github.com/go-co-op/gocron@v1.35.0/executor.go:219 +0x3ed
make: *** [run] Error 2

To Reproduce

Unfortunately cannot share company code for this.

However, there is no issue when using RemoveByTags instead of RemoveByID

Version

v1.35.0

Expected behavior

There should be no fatal error: concurrent map read and map write. Somewhere Lock() is missing.

@mikayzo mikayzo added the bug Something isn't working label Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant