naiveproxy crash with openwrt master

[openips]

naiveproxy version 102.0.5005.61-1-1 x64 with openwrt x64 master

linux kernel 5.10.120
crash radome and Very frequently

config here:

{ "listen": "socks://127.0.0.1:1080", "proxy": "https://123:4563@789.com", "log": "", "padding": "true" }

system log below：

[186320.997544] traps: naive[22977] trap int3 ip:560b2fc4ceb5 sp:7ffd7db86570 error:0 in naive[560b2f909000+400000] [186497.246306] traps: naive[22982] trap int3 ip:558ee5a3beb5 sp:7fff0a7c2fe0 error:0 in naive[558ee56f8000+400000] [186507.269056] traps: naive[22979] trap int3 ip:55e25a7e7eb5 sp:7ffc340faa30 error:0 in naive[55e25a4a4000+400000] [186517.276855] traps: naive[22980] trap int3 ip:558fd9888eb5 sp:7ffda6d58e50 error:0 in naive[558fd9545000+400000] [186527.297564] traps: naive[22978] trap int3 ip:56485fe8beb5 sp:7ffe08db68b0 error:0 in naive[56485fb48000+400000] [186537.314067] traps: naive[22981] trap int3 ip:55e0a2b5beb5 sp:7fffab4b5c50 error:0 in naive[55e0a2818000+400000] [186597.419801] traps: naive[9023] trap int3 ip:5625dd1dfeb5 sp:7ffcff901ba0 error:0 in naive[5625dce9c000+400000] [186607.438475] traps: naive[9025] trap int3 ip:55ab3c392eb5 sp:7fffee56a430 error:0 in naive[55ab3c04f000+400000] [186617.449854] traps: naive[9028] trap int3 ip:55f2964e3eb5 sp:7fff846834f0 error:0 in naive[55f2961a0000+400000] [186627.466299] traps: naive[9026] trap int3 ip:5562e1fc4eb5 sp:7fff6d0e7ce0 error:0 in naive[5562e1c81000+400000] [186637.488861] traps: naive[9024] trap int3 ip:557db164deb5 sp:7ffd3c2fb3b0 error:0 in naive[557db130a000+400000] [186647.502356] traps: naive[9027] trap int3 ip:5636ce56feb5 sp:7ffc121768b0 error:0 in naive[5636ce22c000+400000] [186657.522302] traps: naive[9514] trap int3 ip:5604301b1eb5 sp:7ffc9f6f78d0 error:0 in naive[56042fe6e000+400000] [186667.547698] traps: naive[9510] trap int3 ip:56246f75deb5 sp:7fff7397cbe0 error:0 in naive[56246f41a000+400000] [186677.566323] traps: naive[9512] trap int3 ip:559b8855ceb5 sp:7ffca9aca8d0 error:0 in naive[559b88219000+400000] [186687.581480] traps: naive[9515] trap int3 ip:55a1e3686eb5 sp:7fff40062960 error:0 in naive[55a1e3343000+400000] [186697.601905] traps: naive[9511] trap int3 ip:56317af9ceb5 sp:7ffe6d7daaf0 error:0 in naive[56317ac59000+400000] [186707.623420] traps: naive[9513] trap int3 ip:5579d5f4ceb5 sp:7fffe9337ef0 error:0 in naive[5579d5c09000+400000] [186717.642531] traps: naive[9798] trap int3 ip:55adb5489eb5 sp:7ffd92c85f90 error:0 in naive[55adb5146000+400000] [186757.683232] traps: naive[9803] trap int3 ip:55a7b99b1eb5 sp:7ffcf86b2690 error:0 in naive[55a7b966e000+400000] [186767.711725] traps: naive[9800] trap int3 ip:5609eb76deb5 sp:7ffefec7a070 error:0 in naive[5609eb42a000+400000] [186777.714214] traps: naive[9801] trap int3 ip:5555a9eb1eb5 sp:7ffd713e5fe0 error:0 in naive[5555a9b6e000+400000] [186787.739815] traps: naive[9799] trap int3 ip:56123a543eb5 sp:7fff16289280 error:0 in naive[56123a200000+400000] [186797.752365] traps: naive[9802] trap int3 ip:5589f8fc9eb5 sp:7fff77a22260 error:0 in naive[5589f8c86000+400000] [186837.831589] traps: naive[10336] trap int3 ip:5575d78c9eb5 sp:7ffee98a5c00 error:0 in naive[5575d7586000+400000] [187108.210293] traps: naive[10338] trap int3 ip:561799dfeeb5 sp:7fffd2d307e0 error:0 in naive[561799abb000+400000] [187118.224422] traps: naive[10337] trap int3 ip:5583ef075eb5 sp:7ffcd8c754b0 error:0 in naive[5583eed32000+400000] [187709.345786] traps: naive[10341] trap int3 ip:5598cc95eeb5 sp:7ffc18ed4060 error:0 in naive[5598cc61b000+400000] [187719.365745] traps: naive[10340] trap int3 ip:55d437da5eb5 sp:7ffec6c27070 error:0 in naive[55d437a62000+400000] [187729.384975] traps: naive[10339] trap int3 ip:5583518a1eb5 sp:7ffec2fa0d90 error:0 in naive[55835155e000+400000] [187739.397227] traps: naive[14099] trap int3 ip:55f822423eb5 sp:7ffd9ecbf520 error:0 in naive[55f8220e0000+400000] [187749.411304] traps: naive[14096] trap int3 ip:5603ea741eb5 sp:7ffc9a74a830 error:0 in naive[5603ea3fe000+400000]

[klzgrad]

How to reproduce? Is there a public url that can trigger this by repeated requests?

[openips]

you can download the Firmware in https://t.me/mwan3/219?single with named 2022-06-18-clash.zip and running more than 24h.this crash bug will be repeat again. @klzgrad

[klzgrad]

I can't use this. To debug with, you need to create a build with debug info (in build.sh change symbol level to 1) and use ulimit -c unlimited to create a coredump (if space is an issue, run it in gdb). The information I need is the line number at which this issue happens. int3 is usually a check.

[openips]

OK i will build a new one with with debug info

[openips]

@klzgrad naive debug info is below
if you need some info elst tell me . thks
core-naive-13249-1655860919.zip

[openips]

core-naive-13254-1655863352.zip
core-naive-13251-1655863545.zip
another two coedump

[klzgrad]

You're almost there. Coredumps must be used with the original binary. You can get the line number with this: gdb ./naive coredump and use bt to show the stack trace. I need the stack trace.

[openips]

ok i will try

[openips]

log here

For bug reporting instructions, please see:
https://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/naive...
(No debugging symbols found in /usr/bin/naive)
[New LWP 3209]
[New LWP 3219]
[New LWP 3218]
Core was generated by `/usr/bin/naive /etc/naiveproxy/config_sg.json'.
Program terminated with signal SIGTRAP, Trace/breakpoint trap.
#0 0x0000562d01246eb5 in ?? ()
[Current thread is 1 (LWP 3209)]
(gdb) bt
#0 0x0000562d01246eb5 in ?? ()
#1 0x00007f64cae17990 in ?? ()
#2 0x00007fffd3216338 in ?? ()
#3 0x00007fffd3216030 in ?? ()
#4 0x00007f64cae428f8 in ?? ()
#5 0x00000005cad45300 in ?? ()
#6 0x0000000000000000 in ?? ()
(gdb) quit
root@OpenWrt:~# gdb /usr/bin/naive core-naive-3212-1655949314
GNU gdb (GDB) 11.2
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-openwrt-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
https://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/naive...
(No debugging symbols found in /usr/bin/naive)
[New LWP 3212]
[New LWP 3215]
[New LWP 3221]
[New LWP 5241]
Core was generated by `/usr/bin/naive /etc/naiveproxy/config_sg_http.json'.
Program terminated with signal SIGTRAP, Trace/breakpoint trap.
#0 0x00005618dd240eb5 in ?? ()
[Current thread is 1 (LWP 3212)]
(gdb) bt
#0 0x00005618dd240eb5 in ?? ()
#1 0x00007f5a7d38b900 in ?? ()
#2 0x00007ffd216415d8 in ?? ()
#3 0x00007ffd216412d0 in ?? ()
#4 0x00007f5a7d3b6918 in ?? ()
#5 0x000000057d389330 in ?? ()
#6 0x0000000000000000 in ?? ()

[klzgrad]

Reading symbols from /usr/bin/naive...
(No debugging symbols found in /usr/bin/naive)

This means it's not built with debug symbols. I referred to it above (in build.sh set symbol level to 1).

[openips]

Do you means to chang symbol_level to 1 in scr/bulid.sh ?
OK i see，i can fix the buid.sh but could you tell me how to build the fixed naiveproxy in my local ubuntu server thks

[openips]

could you give me a bin build with et symbol level to 1 thks

[klzgrad]

(Edit build.sh to symbol_level=1)
export EXTRA_FLAGS='target_cpu="x64" target_os="openwrt" use_allocator="none" use_allocator_shim=false use_partition_alloc=false'
export OPENWRT_FLAGS='arch=x86_64 release=21.02.2 gcc_ver=8.4.0 target=x86 subtarget=64'
./get-clang.sh
./build.sh

[openips]

OK i will try

[klzgrad]

https://github.com/klzgrad/naiveproxy/releases/download/v103.0.5060.53-1/naiveproxy-v103.0.5060.53-1-openwrt-x86_64.tar.xz has some debug info but has no line numbers. See what the stack trace looks like.

[openips]

ok i will test today

[openips]

i have already install this version ,running and waiting for crash log, take some time

https://github.com/klzgrad/naiveproxy/releases/download/v103.0.5060.53-1/naiveproxy-v103.0.5060.53-1-openwrt-x86_64.tar.xz has some debug info but has no line numbers. See what the stack trace looks like.

[openips]

core crash log below @klzgrad

gdb /usr/bin/naive core-naive-3128-1656375721

Core was generated by `/usr/bin/naive /etc/naiveproxy/config_us_http.json'.
Program terminated with signal SIGTRAP, Trace/breakpoint trap.
#0  0x0000564e125ae1c5 in net::ClientSocketHandle::Init(net::ClientSocketPool::GroupId const&, scoped_refptr<net::ClientSocketPool::SocketParams>, absl::optional<net::NetworkTrafficAnnotationTag> const&, net::RequestPriority, net::SocketTag const&, net::ClientSocketPool::RespectLimits, base::OnceCallback<void (int)>, base::RepeatingCallback<void (net::HttpResponseInfo const&, net::HttpAuthController*, base::OnceCallback<void ()>)> const&, net::ClientSocketPool*, net::NetLogWithSource const&) ()
[Current thread is 1 (LWP 3128)]
(gdb) bt
#0  0x0000564e125ae1c5 in net::ClientSocketHandle::Init(net::ClientSocketPool::GroupId const&, scoped_refptr<net::ClientSocketPool::SocketParams>, absl::optional<net::NetworkTrafficAnnotationTag> const&, net::RequestPriority, net::SocketTag const&, net::ClientSocketPool::RespectLimits, base::OnceCallback<void (int)>, base::RepeatingCallback<void (net::HttpResponseInfo const&, net::HttpAuthController*, base::OnceCallback<void ()>)> const&, net::ClientSocketPool*, net::NetLogWithSource const&) ()
#1  0x0000564e12417417 in net::(anonymous namespace)::InitSocketPoolHelper(url::SchemeHostPort, int, net::RequestPriority, net::HttpNetworkSession*, net::ProxyInfo const&, net::SSLConfig const&, net::SSLConfig const&, bool, net::PrivacyMode, net::NetworkIsolationKey, net::SecureDnsPolicy, net::SocketTag const&, net::NetLogWithSource const&, int, net::ClientSocketHandle*, net::HttpNetworkSession::SocketPoolType, base::OnceCallback<void (int)>, base::RepeatingCallback<void (net::HttpResponseInfo const&, net::HttpAuthController*, base::OnceCallback<void ()>)> const&) ()
#2  0x0000564e125aea3b in net::InitSocketHandleForRawConnect2(net::HostPortPair const&, net::HttpNetworkSession*, int, net::RequestPriority, net::ProxyInfo const&, net::SSLConfig const&, net::SSLConfig const&, net::PrivacyMode, net::NetworkIsolationKey, net::NetLogWithSource const&, net::ClientSocketHandle*, base::OnceCallback<void (int)>) ()
#3  0x0000564e124af6e8 in net::NaiveConnection::DoLoop(int) ()
#4  0x0000564e124aefd3 in net::NaiveConnection::OnIOComplete(int) ()
#5  0x0000564e124b7f39 in net::Socks5ServerSocket::OnIOComplete(int) ()
#6  0x0000564e125be17d in net::TCPClientSocket::DidCompleteRead(int) ()
#7  0x0000564e1238f37d in net::TCPSocketPosix::ReadCompleted(scoped_refptr<net::IOBuffer> const&, base::OnceCallback<void (int)>, int) ()
#8  0x0000564e1238f32e in base::internal::Invoker<base::internal::BindState<void (net::TCPSocketPosix::*)(scoped_refptr<net::IOBuffer> const&, base::OnceCallback<void (int)>, int), base::internal::UnretainedWrapper<net::TCPSocketPosix>, scoped_refptr<net::IOBuffer>, base::OnceCallback<void (int)> >, void (int)>::RunOnce(base::internal::BindStateBase*, int) ()
#9  0x0000564e125f3c55 in net::SocketPosix::RetryRead(int) ()
#10 0x0000564e125f3ef8 in net::SocketPosix::OnFileCanReadWithoutBlocking(int) ()
#11 0x0000564e1261c509 in base::MessagePumpLibevent::OnLibeventNotification(int, short, void*) ()
#12 0x0000564e1266b2c9 in event_base_loop ()
#13 0x0000564e1261c87b in base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) ()
#14 0x0000564e124029b4 in base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run(bool, base::TimeDelta) ()
#15 0x0000564e12446b5c in base::RunLoop::Run(base::Location const&) ()
#16 0x0000564e124b4087 in main ()

[openips]

cat /etc/naiveproxy/config_us.json

{
  "listen": "socks://127.0.0.1:1080",
  "proxy": "https://123:456@789",
  "log": "",
  "padding": "true"
}

[klzgrad]

What is the "Connection 123 to xxx" log before the crash? The crash has something to do with the origin domain name.

[klzgrad]

See if this fixes https://github.com/klzgrad/naiveproxy/releases/tag/v103.0.5060.53-2

[openips]

What is the "Connection 123 to xxx" log before the crash? The crash has something to do with the origin domain name.

my domain name is abc.cyou but i think the domain name should be secret

[openips]

See if this fixes https://github.com/klzgrad/naiveproxy/releases/tag/v103.0.5060.53-2

i will try and wait for the crash or not

[openips]

i tried103.0.5060.53-2 and 103.0.5060.53-3 both more than 5 days ,more stable then before ,thks your great work .