nginx reload fails (msc_rules_cleanup)

[tomsommer]

When doing nginx -s reload in master (3.0.9), this happens:

*** Error in `nginx: master process /usr/local/nginx/sbin/nginx': double free or corruption (!prev): 0x00000000019d3940 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81329)[0x7f495bce4329]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x38)[0x7f495c6600e8]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x1e)[0x7f495c6600ce]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x2a)[0x7f495c6600da]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x1e)[0x7f495c6600ce]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x2a)[0x7f495c6600da]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x2a)[0x7f495c6600da]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x1e)[0x7f495c6600ce]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x2a)[0x7f495c6600da]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x1e)[0x7f495c6600ce]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x2a)[0x7f495c6600da]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x1e)[0x7f495c6600ce]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x2a)[0x7f495c6600da]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x2a)[0x7f495c6600da]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x2a)[0x7f495c6600da]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x1e)[0x7f495c6600ce]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTree18postOrderTraversalEP8TreeNode+0x2a)[0x7f495c6600da]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity5Utils6IpTreeD1Ev+0x20)[0x7f495c6601a0]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity9operators8IpMatchFD0Ev+0x21)[0x7f495c590f71]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity16RuleWithOperatorD2Ev+0x29)[0x7f495c5f05f9]
/usr/local/modsecurity/lib/libmodsecurity.so.3(_ZN11modsecurity16RuleWithOperatorD0Ev+0x9)[0x7f495c5f0679]
/usr/local/modsecurity/lib/libmodsecurity.so.3(msc_rules_cleanup+0xb0)[0x7f495c5e34c0]
nginx: master process /usr/local/nginx/sbin/nginx[0x4f05b2]
nginx: master process /usr/local/nginx/sbin/nginx(ngx_destroy_pool+0x20)[0x45cf81]
nginx: master process /usr/local/nginx/sbin/nginx(ngx_init_cycle+0x10c6)[0x46d5b1]
nginx: master process /usr/local/nginx/sbin/nginx(ngx_master_process_cycle+0x711)[0x48206a]
nginx: master process /usr/local/nginx/sbin/nginx(main+0xc0b)[0x45ba31]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f495bc85555]

[martinhsv]

Hello @tomsommer ,

I'm not currently able to reproduce what you have reported.

Could you please provide more information? For example the various configuration information suggested in the issue template (like nginx version, O/S and version, etc.).

Also:

• the exact text of the rule that triggers this might be useful
• does this only occur, for example, with IPv6 addresses? Or is it with either IPv4 or IPv6?
• does it happen every time you do a reload, or is it sporadic?
• are there any other conditions in the file that are necessary to trigger this (e.g. if a very small file of only 1 or 2 IP addresses does not trigger the problem, that might also be useful information.)

[tomsommer]

Any tips to how to find the specific rule? It happens pretty reliably every time I reload nginx.

Nginx 1.23.3, modsec compiled with --with-lmdb

[martinhsv]

Given the backtrace you provided, it will be a rule that uses the @ipMatchF operator (or possibly the @ipMatchFromFile operator). I doubt that you have very many such rules.

And the contents of the related file would be the relevant point for other inquiries (does it still happen if you substitute a very small file, IPv4 vs IPv6, etc.)

[tomsommer]

I believe it happens on duplicate CIDR-entries in @ipMatchF, I had two 147.75.208.0/20 in the file

[martinhsv]

Thanks @tomsommer , for the information. I believe I have identified an issue in the code and will continue with the analysis.

[martinhsv]

Resolved via #2890 .