You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Starting up, Keda Operator has a segmentation violation in logger. Presumably because Azure Blob Scaler is missing to store the initialized logger on line 70 in azure_blob_scaler.go. I will contribute a PR and link to this issue.
Expected Behavior
I expect Keda Operator to not have a segmentation violation. If my scaler credentials are bad, I expect Keda to handle it gracefully.
Actual Behavior
Keda crashes cyclically.
Steps to Reproduce the Problem
Add a ScaledJob with azure blob scaling with something wrong in the credentials to blob storage (I presume)
Sit back and watch keda operator crash
Logs from KEDA operator
LogEntry
" /workspace/pkg/scaling/scale_handler.go:108 +0x465"
created by github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).HandleScalableObject
" /workspace/pkg/scaling/scale_handler.go:149 +0x31c"
"github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop(0xc000863650, {0x3cae988, 0xc0011982c0}, 0xc0004e9900, {0x34ee760, 0xc0011a4240}, {0x3c8b6e8, 0xc000e7ff68})"
" /workspace/pkg/scaling/scale_handler.go:286 +0x317"
"github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers(0xc000863650, {0x3cae988, 0xc0011982c0}, {0x34ee760, 0xc0011a4240}, {0x3c8b6e8, 0xc000e7ff68})"
" /workspace/pkg/scaling/cache/scalers_cache.go:125 +0xda"
"github.com/kedacore/keda/v2/pkg/scaling/cache.(*ScalersCache).IsScaledJobActive(0xc00004dd10, {0x3cae988, 0xc0011982c0}, 0xc0011a4240)"
" /workspace/pkg/scaling/cache/scalers_cache.go:263 +0x332"
"github.com/kedacore/keda/v2/pkg/scaling/cache.(*ScalersCache).getScaledJobMetrics(0xc0002dc690, {0x3cae988, 0xc0011982c0}, 0xc0011a4240)"
" /workspace/pkg/scalers/azure_blob_scaler.go:193 +0xa5"
"github.com/kedacore/keda/v2/pkg/scalers.(*azureBlobScaler).IsActive(0xc0011a21e0, {0x3cae988, 0xc0011982c0})"
" /go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 +0x80"
"github.com/go-logr/logr.Logger.Error({{0x0, 0x0}, 0x35a828d}, {0x3c59720, 0xc000412dc0}, {0x35a49bb, 0x6}, {0x0, 0x0, 0x0})"
goroutine 350 [running]:
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x585a40]
panic: runtime error: invalid memory address or nil pointer dereference
"2022-11-03T17:37:27Z INFO Initializing Scaling logic according to ScaledJob Specification {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""scaledJob"": {""name"":""reconstruction"",""namespace"":""app-env-dtu""}, ""namespace"": ""app-env-dtu"", ""name"": ""reconstruction"", ""reconcileID"": ""386dff46-d3f6-4e03-99bc-a2d86ca1a35d""}"
"2022-11-03T17:37:27Z INFO RolloutStrategy: gradual, Not deleting jobs owned by the previous version of the scaleJob {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""scaledJob"": {""name"":""reconstruction"",""namespace"":""app-env-dtu""}, ""namespace"": ""app-env-dtu"", ""name"": ""reconstruction"", ""reconcileID"": ""386dff46-d3f6-4e03-99bc-a2d86ca1a35d""}"
"2022-11-03T17:37:27Z INFO Reconciling ScaledJob {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""scaledJob"": {""name"":""reconstruction"",""namespace"":""app-env-dtu""}, ""namespace"": ""app-env-dtu"", ""name"": ""reconstruction"", ""reconcileID"": ""386dff46-d3f6-4e03-99bc-a2d86ca1a35d""}"
"2022-11-03T17:37:27Z INFO Initializing Scaling logic according to ScaledJob Specification {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""scaledJob"": {""name"":""preprocessing"",""namespace"":""app-env-dtu""}, ""namespace"": ""app-env-dtu"", ""name"": ""preprocessing"", ""reconcileID"": ""6e92f8cf-d4c8-43f2-848a-5060ef1dbeae""}"
"2022-11-03T17:37:27Z INFO RolloutStrategy: gradual, Not deleting jobs owned by the previous version of the scaleJob {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""scaledJob"": {""name"":""preprocessing"",""namespace"":""app-env-dtu""}, ""namespace"": ""app-env-dtu"", ""name"": ""preprocessing"", ""reconcileID"": ""6e92f8cf-d4c8-43f2-848a-5060ef1dbeae""}"
"2022-11-03T17:37:27Z INFO Reconciling ScaledJob {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""scaledJob"": {""name"":""preprocessing"",""namespace"":""app-env-dtu""}, ""namespace"": ""app-env-dtu"", ""name"": ""preprocessing"", ""reconcileID"": ""6e92f8cf-d4c8-43f2-848a-5060ef1dbeae""}"
"2022-11-03T17:37:27Z INFO RolloutStrategy: gradual, Not deleting jobs owned by the previous version of the scaleJob {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""scaledJob"": {""name"":""big-reconstruction"",""namespace"":""app-env-dtu""}, ""namespace"": ""app-env-dtu"", ""name"": ""big-reconstruction"", ""reconcileID"": ""ba3f1142-a3bd-4c3d-a1c7-a07b1e103dc9""}"
"2022-11-03T17:37:27Z INFO Reconciling ScaledJob {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""scaledJob"": {""name"":""big-reconstruction"",""namespace"":""app-env-dtu""}, ""namespace"": ""app-env-dtu"", ""name"": ""big-reconstruction"", ""reconcileID"": ""ba3f1142-a3bd-4c3d-a1c7-a07b1e103dc9""}"
"2022-11-03T17:37:27Z INFO Starting workers {""controller"": ""triggerauthentication"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""TriggerAuthentication"", ""worker count"": 1}"
"2022-11-03T17:37:27Z INFO Starting workers {""controller"": ""scaledobject"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledObject"", ""worker count"": 5}"
"2022-11-03T17:37:27Z INFO Starting workers {""controller"": ""clustertriggerauthentication"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ClusterTriggerAuthentication"", ""worker count"": 1}"
"2022-11-03T17:37:27Z INFO Starting workers {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""worker count"": 1}"
"2022-11-03T17:37:27Z INFO KubeAPIWarningLogger autoscaling/v2beta2 HorizontalPodAutoscaler is deprecated in v1.23+, unavailable in v1.26+ use autoscaling/v2 HorizontalPodAutoscaler"
"2022-11-03T17:37:27Z INFO Starting Controller {""controller"": ""clustertriggerauthentication"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ClusterTriggerAuthentication""}"
"2022-11-03T17:37:27Z INFO Starting EventSource {""controller"": ""clustertriggerauthentication"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ClusterTriggerAuthentication"", ""source"": ""kind source: *v1alpha1.ClusterTriggerAuthentication""}"
"2022-11-03T17:37:27Z INFO Starting Controller {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob""}"
"2022-11-03T17:37:27Z INFO Starting EventSource {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""source"": ""kind source: *v1alpha1.ScaledJob""}"
"2022-11-03T17:37:27Z INFO Starting Controller {""controller"": ""triggerauthentication"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""TriggerAuthentication""}"
"2022-11-03T17:37:27Z INFO Starting EventSource {""controller"": ""triggerauthentication"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""TriggerAuthentication"", ""source"": ""kind source: *v1alpha1.TriggerAuthentication""}"
"2022-11-03T17:37:27Z INFO Starting Controller {""controller"": ""scaledobject"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledObject""}"
"2022-11-03T17:37:27Z INFO Starting EventSource {""controller"": ""scaledobject"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledObject"", ""source"": ""kind source: *v2beta2.HorizontalPodAutoscaler""}"
"2022-11-03T17:37:27Z INFO Starting EventSource {""controller"": ""scaledobject"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledObject"", ""source"": ""kind source: *v1alpha1.ScaledObject""}"
I1103 17:37:27.219554 1 leaderelection.go:258] successfully acquired lease keda/operator.keda.sh
"2022-11-03T17:37:27Z INFO Initializing Scaling logic according to ScaledJob Specification {""controller"": ""scaledjob"", ""controllerGroup"": ""keda.sh"", ""controllerKind"": ""ScaledJob"", ""scaledJob"": {""name"":""big-reconstruction"",""namespace"":""app-env-dtu""}, ""namespace"": ""app-env-dtu"", ""name"": ""big-reconstruction"", ""reconcileID"": ""ba3f1142-a3bd-4c3d-a1c7-a07b1e103dc9""}"
"2022-11-03T17:37:11Z INFO Starting server {""path"": ""/metrics"", ""kind"": ""metrics"", ""addr"": ""[::]:8080""}"
"2022-11-03T17:37:11Z INFO Starting server {""kind"": ""health probe"", ""addr"": ""[::]:8081""}"
I1103 17:37:11.962951 1 leaderelection.go:248] attempting to acquire leader lease keda/operator.keda.sh...
2022-11-03T17:37:11Z INFO setup Go OS/Arch: linux/amd64
2022-11-03T17:37:11Z INFO setup Go Version: go1.17.13
2022-11-03T17:37:11Z INFO setup Git Commit: a4a118201214e7abdeebad72cbe337b9856f8191
2022-11-03T17:37:11Z INFO setup KEDA Version: 2.8.0
2022-11-03T17:37:11Z INFO setup Starting manager
"2022-11-03T17:37:11Z INFO setup Running on Kubernetes 1.24 {""version"": ""v1.24.6""}"
"2022-11-03T17:37:11Z INFO controller-runtime.metrics Metrics server is starting to listen {""addr"": "":8080""}"```
### KEDA Version
2.8.0
### Kubernetes Version
1.24
### Platform
Microsoft Azure
### Scaler Details
Azube Blob Scaler
### Anything else?
_No response_
The text was updated successfully, but these errors were encountered:
Report
Starting up, Keda Operator has a segmentation violation in logger. Presumably because Azure Blob Scaler is missing to store the initialized logger on line 70 in azure_blob_scaler.go. I will contribute a PR and link to this issue.
Expected Behavior
I expect Keda Operator to not have a segmentation violation. If my scaler credentials are bad, I expect Keda to handle it gracefully.
Actual Behavior
Keda crashes cyclically.
Steps to Reproduce the Problem
Logs from KEDA operator
The text was updated successfully, but these errors were encountered: