Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LDPd crash in RB Tree rbe_insert_color() #841

Closed
mwinter-osr opened this issue Jul 21, 2017 · 3 comments
Closed

LDPd crash in RB Tree rbe_insert_color() #841

mwinter-osr opened this issue Jul 21, 2017 · 3 comments
Assignees
Labels
Milestone

Comments

@mwinter-osr
Copy link
Member

Debug information is from master branch at dceb5f8 (July 20). However the bug exists at least since f30a732 (June 26), but not on or before bade23d (June 2)
Bug does not exist on 2.0 or 3.0

LDPd crashes in ANVL Compliance Test ANVL-LDP-28.12

From the LDP log:

2017/07/20 18:04:01 LDP: msg[out]: initialization: lsr-id 192.168.1.1
2017/07/20 18:04:01 LDP: kalive[out]: keepalive: lsr-id 192.168.1.1
2017/07/20 18:04:01 LDP: kalive[in]: keepalive: lsr-id 192.168.1.1
2017/07/20 18:04:01 LDP: nbr_fsm: event KEEPALIVE RECEIVED resulted in action START NEIGHBOR SESSION and changing state for lsr-id 192.168.1.1 from OPENREC to OPERATIONAL
2017/07/20 18:04:02 LDP: waiting for children to terminate
2017/07/20 18:04:02 LDP: terminating

Backtrace info from the coredump:

#0  0x00007fde6d308428 in __GI_raise (sig=sig@entry=6)
	at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007fde6d30a02a in __GI_abort () at abort.c:89
#2  0x00007fde6db1397b in core_handler (signo=11, siginfo=0x7ffdab726170, 
	context=0x7ffdab726040) at sigevent.c:249
#3  <signal handler called>
#4  rbe_insert_color (rbe=0x113daa0, rbt=<optimized out>, t=<optimized out>)
	at openbsd-tree.c:169
#5  _rb_insert (t=t@entry=0x42b1d0 <nbr_pid_head_RB_INFO>, 
	rbt=rbt@entry=0x63d220 <nbrs_by_pid>, elm=elm@entry=0x113da60)
	at openbsd-tree.c:432
#6  0x0000000000417153 in nbr_pid_head_RB_INSERT (elm=0x113da60, 
	head=0x63d220 <nbrs_by_pid>) at ldpe.h:124
#7  nbr_update_peerid (nbr=nbr@entry=0x113da60) at neighbor.c:338
#8  0x0000000000417196 in nbr_act_session_operational (nbr=nbr@entry=0x113da60)
	at neighbor.c:750
#9  0x00000000004174f1 in nbr_fsm (nbr=nbr@entry=0x113da60, 
	event=event@entry=NBR_EVT_KEEPALIVE_RCVD) at neighbor.c:182
#10 0x00000000004272eb in recv_keepalive (nbr=nbr@entry=0x113da60, 
	buf=buf@entry=0x113d3ea "\002\001", len=len@entry=8) at keepalive.c:60
#11 0x0000000000419b63 in session_read (thread=<optimized out>) at packet.c:546
#12 0x00007fde6daf5ef7 in thread_call (thread=thread@entry=0x7ffdab726970)
	at thread.c:1470
#13 0x00000000004140f5 in ldpe () at ldpe.c:136
#14 0x000000000040b4eb in main (argc=0, argv=<optimized out>) at ldpd.c:280

#4 is here:

#4  rbe_insert_color (rbe=0x113daa0, rbt=<optimized out>, t=<optimized out>)
	at openbsd-tree.c:169
169			if (parent == RBE_LEFT(gparent)) {
(gdb) list
164	
165		while ((parent = RBE_PARENT(rbe)) != NULL
166		       && RBE_COLOR(parent) == RB_RED) {
167			gparent = RBE_PARENT(parent);
168	
169			if (parent == RBE_LEFT(gparent)) {
170				tmp = RBE_RIGHT(gparent);
171				if (tmp != NULL && RBE_COLOR(tmp) == RB_RED) {
172					RBE_COLOR(tmp) = RB_BLACK;
173					rbe_set_blackred(parent, gparent);

and it crashes as gparent is NULL in line 169

parent is {rbt_parent = 0x0, rbt_left = 0x0, rbt_right = 0x113daa0, rbt_color = 1}

Issue is reliable reproducable. Access info of the bug is mailed to @rwestphal to look at.

@mwinter-osr mwinter-osr added this to the next milestone Jul 21, 2017
@dwalton76
Copy link
Contributor

I am able to reproduce this on one of our internal tests (tests.nclu.bond_and_vrr_test.VrrNcluTest.test06_bond_lacp_bypass) as well

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/lib/frr/zebra -M snmp -s 90000000 --daemon -A 127.0.0.1'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f1e882ac067 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#0  0x00007f1e882ac067 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007f1e882ad448 in __GI_abort () at abort.c:89
#2  0x00007f1e895f8c8f in core_handler (signo=11, siginfo=0x7ffcbc660e70, context=0x7ffcbc660d40) at sigevent.c:249
#3  <signal handler called>
#4  rbe_insert_color (rbe=0x1a517f0, rbt=<optimized out>, t=<optimized out>) at openbsd-tree.c:169
#5  _rb_insert (t=t@entry=0x7f1e8982b430 <vrf_id_head_RB_INFO>, rbt=0x6615c0 <vrfs_by_id>, rbt@entry=0x7f1e89834dc0 <vrfs_by_id>, elm=elm@entry=0x1a517f0) at openbsd-tree.c:432
#6  0x00007f1e895fc36c in vrf_id_head_RB_INSERT (head=0x7f1e89834dc0 <vrfs_by_id>, elm=0x1a517f0) at vrf.h:92
#7  vrf_get (vrf_id=<optimized out>, name=name@entry=0x7ffcbc661674 "vrf1") at vrf.c:124
#8  0x00000000004355e2 in netlink_vrf_change (h=h@entry=0x7ffcbc661650, tb=<optimized out>, name=name@entry=0x7ffcbc661674 "vrf1") at if_netlink.c:343
#9  0x00000000004368d1 in netlink_link_change (snl=<optimized out>, h=0x7ffcbc661650, ns_id=<optimized out>, startup=<optimized out>) at if_netlink.c:1083
#10 0x000000000043ad13 in netlink_parse_info (filter=filter@entry=0x43a457 <netlink_information_fetch>, nl=nl@entry=0x19412e8, zns=zns@entry=0x19412c0, count=count@entry=5, startup=startup@entry=0) at kernel_netlink.c:636
#11 0x000000000043ada0 in kernel_read (thread=<optimized out>) at kernel_netlink.c:273
#12 0x00007f1e895e0789 in thread_call (thread=thread@entry=0x7ffcbc663870) at thread.c:1470
#13 0x00007f1e896019ca in frr_run (master=0x18a81b0) at libfrr.c:435
#14 0x000000000040d754 in main (argc=8, argv=0x7ffcbc663a48) at main.c:344

rwestphal added a commit to opensourcerouting/frr that referenced this issue Jul 24, 2017
Commit 8f942af introduced a bug while silencing a clang warning. Silence
the warning in a different way to fix our red-black tree implementation.

Fixes FRRouting#841.

Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
@rwestphal
Copy link
Member

@dwalton76 could you check if #848 fixes the problem for you?

@dwalton76
Copy link
Contributor

@rwestphal yep this fixes the crash for me

qlyoung pushed a commit to qlyoung/frr that referenced this issue Sep 15, 2017
Commit 8f942af introduced a bug while silencing a clang warning. Silence
the warning in a different way to fix our red-black tree implementation.

Fixes FRRouting#841.

Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
(cherry picked from commit 145b205)
qlyoung pushed a commit to qlyoung/frr that referenced this issue Nov 6, 2017
Commit 8f942af introduced a bug while silencing a clang warning. Silence
the warning in a different way to fix our red-black tree implementation.

Fixes FRRouting#841.

Signed-off-by: Renato Westphal <renato@opensourcerouting.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants