Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

arm64: unprotected error in call to Lua API (bad light userdata pointer) #2802

Closed
brezerk opened this issue Jul 18, 2018 · 6 comments
Closed

arm64: unprotected error in call to Lua API (bad light userdata pointer) #2802

brezerk opened this issue Jul 18, 2018 · 6 comments
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@brezerk
Copy link

brezerk commented Jul 18, 2018

Is this a BUG REPORT:

NGINX Ingress controller version: quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm64:0.17.1

Kubernetes version (use kubectl version): v1.10.5

Environment:

  • Cloud provider or hardware configuration: rock64
  • OS (e.g. from /etc/os-release): Gentoo/Linux
  • Kernel (e.g. uname -a): Linux cerberus02 4.17.2-gentoo Add presubmit checks #4 SMP Sat Jun 23 01:04:06 EEST 2018 aarch64 GNU/Linux
  • Install tools:
  • Others:

What happened:

Jul 18 02:56:24 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:24 cerberus02 8467cda57149[20943]: NGINX Ingress controller
Jul 18 02:56:24 cerberus02 8467cda57149[20943]:   Release:    0.17.1
Jul 18 02:56:24 cerberus02 8467cda57149[20943]:   Build:      git-12f7966
Jul 18 02:56:24 cerberus02 8467cda57149[20943]:   Repository: https://github.com/kubernetes/ingress-nginx.git
Jul 18 02:56:24 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:24 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:24 cerberus02 8467cda57149[20943]: nginx version: nginx/1.13.12
Jul 18 02:56:24 cerberus02 8467cda57149[20943]: W0717 23:56:24.116206       8 client_config.go:552] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
Jul 18 02:56:24 cerberus02 8467cda57149[20943]: I0717 23:56:24.117206       8 main.go:191] Creating API client for https://10.96.0.1:443
Jul 18 02:56:24 cerberus02 8467cda57149[20943]: I0717 23:56:24.332529       8 main.go:235] Running in Kubernetes cluster version v1.10 (v1.10.5) - git (clean) commit 32ac1c9073b132b8ba18aa830f46b77dcceb0723 - platform linux/arm64
Jul 18 02:56:24 cerberus02 8467cda57149[20943]: I0717 23:56:24.384943       8 main.go:100] Validated ingress-nginx/default-http-backend as the default backend.
Jul 18 02:56:31 cerberus02 8467cda57149[20943]: I0717 23:56:31.169042       8 nginx.go:255] Starting NGINX Ingress controller
Jul 18 02:56:31 cerberus02 8467cda57149[20943]: I0717 23:56:31.219234       8 event.go:221] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"nginx-configuration", UID:"36639499-8a1c-11e8-bc57-e602c5d7d22a", APIVersion:"v1", ResourceVersion:"3943", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/nginx-configuration
Jul 18 02:56:31 cerberus02 8467cda57149[20943]: I0717 23:56:31.236387       8 event.go:221] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"tcp-services", UID:"366c3fca-8a1c-11e8-bc57-e602c5d7d22a", APIVersion:"v1", ResourceVersion:"3948", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/tcp-services
Jul 18 02:56:31 cerberus02 8467cda57149[20943]: I0717 23:56:31.236574       8 event.go:221] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"udp-services", UID:"3672e0fe-8a1c-11e8-bc57-e602c5d7d22a", APIVersion:"v1", ResourceVersion:"3949", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/udp-services
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: I0717 23:56:32.283653       8 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"test-ingress", UID:"b4589798-8a1c-11e8-bc57-e602c5d7d22a", APIVersion:"extensions/v1beta1", ResourceVersion:"4311", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress default/test-ingress
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: I0717 23:56:32.370490       8 nginx.go:276] Starting NGINX process
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: I0717 23:56:32.371058       8 leaderelection.go:185] attempting to acquire leader lease  ingress-nginx/ingress-controller-leader-nginx...
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: I0717 23:56:32.378071       8 controller.go:926] Ingress "default/test-ingress" defines a backend but no rule. Using it to configure the catch-all server "_"
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: I0717 23:56:32.378539       8 controller.go:169] Configuration changes detected, backend reload required.
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: I0717 23:56:32.487013       8 leaderelection.go:194] successfully acquired lease ingress-nginx/ingress-controller-leader-nginx
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: I0717 23:56:32.487592       8 status.go:197] new leader elected: nginx-ingress-controller-6598dcf768-hx7l9
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: E0717 23:56:32.494595       8 controller.go:181] Unexpected failure reloading the backend:
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: Error: exit status 1
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: W0717 23:56:32.494670       8 queue.go:130] requeuing initial-sync, err
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: Error: exit status 1
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:32 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: I0717 23:56:35.711299       8 controller.go:926] Ingress "default/test-ingress" defines a backend but no rule. Using it to configure the catch-all server "_"
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: I0717 23:56:35.711612       8 controller.go:169] Configuration changes detected, backend reload required.
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: E0717 23:56:35.831007       8 controller.go:181] Unexpected failure reloading the backend:
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: Error: exit status 1
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: W0717 23:56:35.831101       8 queue.go:130] requeuing kube-system/kube-controller-manager, err
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: Error: exit status 1
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:35 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: I0717 23:56:39.044671       8 controller.go:926] Ingress "default/test-ingress" defines a backend but no rule. Using it to configure the catch-all server "_"
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: I0717 23:56:39.044988       8 controller.go:169] Configuration changes detected, backend reload required.
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: E0717 23:56:39.156152       8 controller.go:181] Unexpected failure reloading the backend:
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: Error: exit status 1
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: W0717 23:56:39.156230       8 queue.go:130] requeuing kube-system/kube-dns, err
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: Error: exit status 1
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:39 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: I0717 23:56:42.377723       8 controller.go:926] Ingress "default/test-ingress" defines a backend but no rule. Using it to configure the catch-all server "_"
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: I0717 23:56:42.377935       8 controller.go:169] Configuration changes detected, backend reload required.
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: E0717 23:56:42.512025       8 controller.go:181] Unexpected failure reloading the backend:
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: Error: exit status 1
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: W0717 23:56:42.512152       8 queue.go:130] requeuing default/nginx, err
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: Error: exit status 1
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: 
Jul 18 02:56:42 cerberus02 8467cda57149[20943]: -------------------------------------------------------------------------------
Jul 18 02:56:44 cerberus02 8467cda57149[20943]: I0717 23:56:44.629530       8 main.go:158] Received SIGTERM, shutting down
Jul 18 02:56:44 cerberus02 8467cda57149[20943]: I0717 23:56:44.629624       8 nginx.go:340] Shutting down controller queues
Jul 18 02:56:44 cerberus02 8467cda57149[20943]: I0717 23:56:44.629698       8 status.go:119] updating status of Ingress rules (remove)
Jul 18 02:56:44 cerberus02 8467cda57149[20943]: I0717 23:56:44.657504       8 status.go:138] removing address from ingress status ([])
Jul 18 02:56:44 cerberus02 8467cda57149[20943]: I0717 23:56:44.658124       8 nginx.go:348] Stopping NGINX process
Jul 18 02:56:44 cerberus02 8467cda57149[20943]: PANIC: unprotected error in call to Lua API (bad light userdata pointer)
Jul 18 02:56:44 cerberus02 8467cda57149[20943]: I0717 23:56:44.693864       8 main.go:162] Error during shutdown: exit status 1
Jul 18 02:56:44 cerberus02 8467cda57149[20943]: I0717 23:56:44.694053       8 main.go:166] Handled quit, awaiting Pod deletion
Jul 18 02:56:54 cerberus02 8467cda57149[20943]: I0717 23:56:54.694330       8 main.go:169] Exiting with 1

What you expected to happen:
The ingress controller starts

How to reproduce it (as minimally and precisely as possible):
Boot the ingress controller on arm64 architecture

Anything else we need to know:

Looks like the infamous light userdata issue LuaJIT/LuaJIT#230 https://groups.google.com/forum/#!topic/openresty-en/Gp0NGN9lMXk etc. is affecting ingress-nginx on arm64 platform.

Same issue was observer on some other projects like: neovim/neovim#7879
@brezerk
Copy link
Author

brezerk commented Jul 18, 2018

LuaJIT/LuaJIT#245

LuaJIT on 64 bit systems provides a limited range of 47 bits for the legacy lightuserdata data type. This is only relevant on x64 systems which use the negative part of the virtual address space in user mode, e.g. Solaris/x64, and on ARM64 systems configured with a 48 bit or 52 bit VA. Avoid using lightuserdata to hold pointers that may point outside of that range, e.g. variables on the stack. In general, avoid this data type for new code and replace it with (much more performant) FFI bindings. FFI cdata pointers can address the full 64 bit range.

@antoineco
Copy link
Contributor

antoineco commented Jul 22, 2018 •
edited
Loading

/kind bug

ping @ElvinEfendi

@k8s-ci-robot k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Jul 22, 2018
@ElvinEfendi
Copy link
Member

lua-nginx-module currently does not support ARM64, there's a pending PR: openresty/lua-nginx-module#1379. We will update as soon as it gets merged and released.

@ElvinEfendi ElvinEfendi mentioned this issue Oct 10, 2018
Merged
@ktsakalozos ktsakalozos mentioned this issue Nov 1, 2018
Merged
@aledbf
Copy link
Member

aledbf commented Nov 4, 2018 •
edited
Loading

@brezerk please use quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm64:dev
This image contains current master with the luajit fork from openresty with support for arm64

@jianghang8421 jianghang8421 mentioned this issue Nov 9, 2018
Closed
@aledbf
Copy link
Member

aledbf commented Nov 28, 2018

Closing. Please update to 0.21.0. Includes openresty/lua-nginx-module#1379.

@aledbf aledbf closed this as completed Nov 28, 2018
@brezerk
Copy link
Author

brezerk commented Dec 4, 2018

Thanks a lot. Looks good.

@btrepp btrepp mentioned this issue Aug 1, 2020
Open
@afbjorklund afbjorklund mentioned this issue Apr 17, 2021
Closed
@nathancoleman nathancoleman mentioned this issue Jun 7, 2022
Closed
@pmalek pmalek mentioned this issue Jun 8, 2022
Closed
2 tasks
carlosabalde added a commit to carlosabalde/libvmod-cfg that referenced this issue May 29, 2024
@carlosabalde
Stop using Lua light userdata to avoid issues in platforms like ARM64 #…
118b94e 
…34

[1] kubernetes/ingress-nginx#2802
[2] neovim/neovim#7879
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@brezerk @aledbf @ElvinEfendi @antoineco @k8s-ci-robot