Summary
When executing the following simple query, the vtgate will go into an endless loop that also keeps consuming memory and eventually will OOM.
Details
When running the following query, the evalengine will try evaluate it and runs forever.
select _utf16 0xFF
The source of the bug lies in the collation logic that we have. The bug applies to all utf16, utf32 and ucs2 encodings. In general, the bug is there for any encoding where the minimal byte length for a single character is more than 1 byte.
The decoding functions for these collations all implement logic like the following to enforce the minimal character length:
https://github.com/vitessio/vitess/blob/8f6cfaaa643a08dc111395a75a2d250ee746cfa8/go/mysql/collations/charset/unicode/utf16.go#L69-L71
The problem is that all the callers of DecodeRune expect progress by returning the number of bytes consumed. This means that if there's only 1 byte left in an input, it will here return still 0 and the caller(s) don't consume the character.
One example of such a caller is the following:
https://github.com/vitessio/vitess/blob/8f6cfaaa643a08dc111395a75a2d250ee746cfa8/go/mysql/collations/charset/convert.go#L73-L79
The logic here moves forward the pointer in the input []byte but if DecodeRune returns 0 in case of error, it will keep running forever. The OOM happens since it keeps adding the ? as the invalid character to the destination buffer infinitely, growing forever until it runs out of memory.
The fix here would be to always return forward progress also on invalid strings.
There's also a separate bug here that even if progress is guaranteed, select _utf16 0xFF will return the wrong result currently. MySQL will pad here the input when the _utf16 introducer is used with leading 0x00 bytes and then decode to UTF-16, resulting in the output of ÿ here.
PoC
select _utf16 0xFF
Impact
Denial of service attack by triggering unbounded memory usage.
References
dbussink Remediation developer
mattrobenolt Finder
vmg Remediation reviewer
Summary
When executing the following simple query, the
vtgatewill go into an endless loop that also keeps consuming memory and eventually will OOM.Details
When running the following query, the
evalenginewill try evaluate it and runs forever.The source of the bug lies in the collation logic that we have. The bug applies to all
utf16,utf32anducs2encodings. In general, the bug is there for any encoding where the minimal byte length for a single character is more than 1 byte.The decoding functions for these collations all implement logic like the following to enforce the minimal character length:
https://github.com/vitessio/vitess/blob/8f6cfaaa643a08dc111395a75a2d250ee746cfa8/go/mysql/collations/charset/unicode/utf16.go#L69-L71
The problem is that all the callers of
DecodeRuneexpect progress by returning the number of bytes consumed. This means that if there's only 1 byte left in an input, it will here return still0and the caller(s) don't consume the character.One example of such a caller is the following:
https://github.com/vitessio/vitess/blob/8f6cfaaa643a08dc111395a75a2d250ee746cfa8/go/mysql/collations/charset/convert.go#L73-L79
The logic here moves forward the pointer in the input
[]bytebut ifDecodeRunereturns0in case of error, it will keep running forever. The OOM happens since it keeps adding the?as the invalid character to the destination buffer infinitely, growing forever until it runs out of memory.The fix here would be to always return forward progress also on invalid strings.
There's also a separate bug here that even if progress is guaranteed,
select _utf16 0xFFwill return the wrong result currently. MySQL will pad here the input when the_utf16introducer is used with leading0x00bytes and then decode to UTF-16, resulting in the output ofÿhere.PoC
Impact
Denial of service attack by triggering unbounded memory usage.
References