Invalid payload in membership brakes cluster completely

[yngvar-antonsson]

The invalid payload in membership members (or the invalid member itself) leads to broken webui (or some undiscovered problems).

membership.members['!!binary something'] = {
    payload = 98,
}

[Bob4anckii]

TDG 2.6.9
During running destructive tests on a cluster of 8 hosts, a 15% network packet corruption scenario was run for 10 minutes, resulting to this bug
The problem was detected when called

require'fun'.iter(require'membership'.members()):filter(function(member)
        return type(member.payload) ~= 'table'
    end):totable()

On instance of tarantool

[grafin]

How is that possible? I thought Tarantool relies on TCP to detect broken packets and resend them. Were the packets modified on higher levels of OSI (like in a proxy before Tarantool)?

[yngvar-antonsson]

How is that possible? I thought Tarantool relies on TCP to detect broken packets and resend them. Were the packets modified on higher levels of OSI (like in a proxy before Tarantool)?

I'm not sure about the place where the packets were corrupted, but membership works with UDP.
Anyway, for the four years in Tarantool I've never seen this bug.

[grafin]

I'm not sure about the place where the packets were corrupted, but membership works with UDP

Thanks for the info, then it is plausible that the packets might be corrupted in any way during the transmission (although not probable in real setups, thats why it probably didn't happen before intentional destruction testing)