Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #370: Go regex breakage #372

Merged

Conversation

tsellers-r7
Copy link
Contributor

@tsellers-r7 tsellers-r7 commented Sep 2, 2021

Description

This PR changes the maximum repetition size so as to be compatible with Go's regex requirements.

Motivation and Context

The character class repetition limits were added to some fingerprints in PR #368 with the goal of reducing the impact that matches of unbounded sizes could have on performance. The value of 1024 was somewhat arbitrarily selected as a value that was large enough to allow for all expected values while still having some reasonable limit. 1000 will serve just as well and is supported by Go.

Reference: https://pkg.go.dev/regexp/syntax

Implementation restriction: The counting forms x{n,m}, x{n,}, and x{n} reject forms that create a minimum or maximum repetition count above 1000. Unlimited repetitions are not subject to this restriction.

We have plans to implement better cross-language testing in the near future so this issue will be caught before the change is landed.

How Has This Been Tested?

$ ~/go/bin/recog_verify ~/git/recog/xml/
INFO[0000] loaded 297 fingerprints from ~/git/recog/xml/apache_modules.xml 
INFO[0000] loaded 38 fingerprints from ~/git/recog/xml/apache_os.xml 
INFO[0000] loaded 7 fingerprints from ~/git/recog/xml/architecture.xml 
INFO[0000] loaded 17 fingerprints from ~/git/recog/xml/dhcp_vendor_class.xml 
INFO[0000] loaded 75 fingerprints from ~/git/recog/xml/dns_versionbind.xml 
INFO[0000] loaded 180 fingerprints from ~/git/recog/xml/favicons.xml 
INFO[0000] loaded 148 fingerprints from ~/git/recog/xml/ftp_banners.xml 
INFO[0000] loaded 99 fingerprints from ~/git/recog/xml/h323_callresp.xml 
INFO[0000] loaded 35 fingerprints from ~/git/recog/xml/hp_pjl_id.xml 
INFO[0000] loaded 338 fingerprints from ~/git/recog/xml/html_title.xml 
INFO[0000] loaded 66 fingerprints from ~/git/recog/xml/http_cookies.xml 
INFO[0000] loaded 397 fingerprints from ~/git/recog/xml/http_servers.xml 
INFO[0000] loaded 62 fingerprints from ~/git/recog/xml/http_wwwauth.xml 
INFO[0000] loaded 18 fingerprints from ~/git/recog/xml/imap_banners.xml 
INFO[0000] loaded 55 fingerprints from ~/git/recog/xml/ldap_searchresult.xml 
INFO[0000] loaded 171 fingerprints from ~/git/recog/xml/mdns_device-info_txt.xml 
INFO[0000] loaded 1 fingerprints from ~/git/recog/xml/mdns_workstation_txt.xml 
INFO[0000] loaded 127 fingerprints from ~/git/recog/xml/mysql_banners.xml 
INFO[0000] loaded 105 fingerprints from ~/git/recog/xml/mysql_error.xml 
INFO[0000] loaded 7 fingerprints from ~/git/recog/xml/nntp_banners.xml 
INFO[0000] loaded 75 fingerprints from ~/git/recog/xml/ntp_banners.xml 
INFO[0000] loaded 59 fingerprints from ~/git/recog/xml/operating_system.xml 
INFO[0000] loaded 30 fingerprints from ~/git/recog/xml/pop_banners.xml 
INFO[0000] loaded 8 fingerprints from ~/git/recog/xml/rsh_resp.xml 
INFO[0000] loaded 11 fingerprints from ~/git/recog/xml/rtsp_servers.xml 
INFO[0000] loaded 52 fingerprints from ~/git/recog/xml/sip_banners.xml 
INFO[0000] loaded 46 fingerprints from ~/git/recog/xml/sip_user_agents.xml 
INFO[0000] loaded 8 fingerprints from ~/git/recog/xml/smb_native_lm.xml 
INFO[0000] loaded 77 fingerprints from ~/git/recog/xml/smb_native_os.xml 
INFO[0000] loaded 139 fingerprints from ~/git/recog/xml/smtp_banners.xml 
INFO[0000] loaded 3 fingerprints from ~/git/recog/xml/smtp_debug.xml 
INFO[0000] loaded 2 fingerprints from ~/git/recog/xml/smtp_ehlo.xml 
INFO[0000] loaded 9 fingerprints from ~/git/recog/xml/smtp_expn.xml 
INFO[0000] loaded 19 fingerprints from ~/git/recog/xml/smtp_help.xml 
INFO[0000] loaded 2 fingerprints from ~/git/recog/xml/smtp_mailfrom.xml 
INFO[0000] loaded 3 fingerprints from ~/git/recog/xml/smtp_noop.xml 
INFO[0000] loaded 2 fingerprints from ~/git/recog/xml/smtp_quit.xml 
INFO[0000] loaded 1 fingerprints from ~/git/recog/xml/smtp_rcptto.xml 
INFO[0000] loaded 1 fingerprints from ~/git/recog/xml/smtp_rset.xml 
INFO[0000] loaded 1 fingerprints from ~/git/recog/xml/smtp_turn.xml 
INFO[0000] loaded 10 fingerprints from ~/git/recog/xml/smtp_vrfy.xml 
INFO[0000] loaded 553 fingerprints from ~/git/recog/xml/snmp_sysdescr.xml 
INFO[0000] loaded 41 fingerprints from ~/git/recog/xml/snmp_sysobjid.xml 
INFO[0000] loaded 150 fingerprints from ~/git/recog/xml/ssh_banners.xml 
INFO[0000] loaded 133 fingerprints from ~/git/recog/xml/telnet_banners.xml 
INFO[0000] loaded 15 fingerprints from ~/git/recog/xml/tls_jarm.xml 
INFO[0000] loaded 22 fingerprints from ~/git/recog/xml/x11_banners.xml 
INFO[0000] loaded 32 fingerprints from ~/git/recog/xml/x509_issuers.xml 
INFO[0000] loaded 150 fingerprints from ~/git/recog/xml/x509_subjects.xml 

Types of changes

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • I have updated the documentation accordingly (or changes are not required).
  • I have added tests to cover my changes (or new tests are not required).
  • All new and existing tests passed.

@tsellers-r7
Copy link
Contributor Author

CC @hdm @pberry25

@mkienow-r7 mkienow-r7 self-assigned this Sep 2, 2021
Copy link
Contributor

@mkienow-r7 mkienow-r7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I've verified the test steps locally.

@tsellers-r7 tsellers-r7 merged commit 471bf7d into rapid7:master Sep 2, 2021
@tsellers-r7 tsellers-r7 deleted the issue_370_fix_go_regex_compat branch September 2, 2021 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants