CPE Updates #267

tsellers-r7 · 2020-05-27T16:59:45Z

Description

This PR:

Updates the CPE data per the instructions found here: https://github.com/rapid7/recog/blob/master/CONTRIBUTING.md#updating-cpes
- The CPE mapping file was updated so there are improvements
- I fixed a few duplicate values in fingerprints which allowed those fingerprint databases to be processed.
- Note: The script will remove any CPE that hasn't actually been issued by NIST or that can't be constructed using Vendor/Product. The CPE mapping file assists where needed.
Manually adjusts the description of a Ubuntu and Debian MySQL banners to be a bit more consistent.

Motivation and Context

Improved CPE output.

How Has This Been Tested?

rspec
rake tests

Types of changes

Bug fix (non-breaking change which fixes an issue)
Content updates

Checklist:

I have updated the documentation accordingly (or changes are not required).
I have added tests to cover my changes (or new tests are not required).
All new and existing tests passed.

tsellers-r7 · 2020-05-27T17:00:44Z

FYI @hdm I've changed quite a bit here...

tsellers-r7 · 2020-05-27T17:01:35Z

xml/dns_versionbind.xml

+    <param pos="0" name="service.family" value="Pi-hole"/>
+    <param pos="0" name="service.product" value="Pi-hole"/>
+    <param pos="1" name="service.version"/>
+    <param pos="0" name="service.cpe23" value="cpe:/a:pi-hole:pi-hole:{service.version}"/>


FYI @brudis-r7 - This fingerprint has changed quite a bit.

tsellers-r7 · 2020-05-27T17:01:58Z

xml/http_cookies.xml

+    <param pos="0" name="service.family" value="NetScaler"/>
+    <param pos="0" name="service.device" value="Network Management Device"/>
+    <param pos="0" name="service.product" value="NetScaler"/>
+    <param pos="0" name="service.cpe23" value="cpe:/a:citrix:netscaler:-"/>
  </fingerprint>


FYI @brudis-r7 - This fingerprint has changed quite a bit.

This should probably assert os/hw like the rest of them

tsellers-r7 · 2020-05-27T17:02:18Z

xml/http_servers.xml

+    <param pos="0" name="service.device" value="Network Management Device"/>
+    <param pos="0" name="service.product" value="NetScaler"/>
+    <param pos="1" name="service.version"/>
+    <param pos="0" name="service.cpe23" value="cpe:/a:citrix:netscaler:{service.version}"/>


FYI @brudis-r7 - This fingerprint has changed quite a bit.

OS fields need to be re-added. Add service too, but we can't remove OS without breaking things.

tsellers-r7 · 2020-05-27T17:02:33Z

xml/ntp_banners.xml

    <param pos="2" name="os.arch"/>
    <param pos="3" name="os.version"/>
+    <param pos="0" name="os.cpe23" value="cpe:/o:freebsd:freebsd:{os.version}"/>


FYI @brudis-r7 - This fingerprint has changed quite a bit.

tsellers-r7 · 2020-05-27T17:05:08Z

@hdm I can fix that. It does pass tests, etc.

xml/dns_versionbind.xml

cpe-remap.yaml

hdm · 2020-05-27T17:08:21Z

xml/http_cookies.xml

-    <param pos="0" name="os.family" value="NetScaler"/>
-    <param pos="0" name="os.device" value="Network Management Device"/>
-    <param pos="0" name="os.product" value="NetScaler"/>
+    <param pos="0" name="service.vendor" value="Citrix"/>


This removes the OS assert, it should not. This should probably assert HW too.

I can assert that as FreeBSD

It should be asserted as what it was before, not FreeBSD (its a custom OS)

This one in particular is a big deal, as we use the os and hw fields to set the device type and hardware type. Folks trying to find their possibly-exploitable NetScalers need this fingerprint to remain how it was (from our perspective).

Yup, I can definitely sort this out. I'll try to review everywhere Netscaler exists and try to make it consistent.

@hdm - I'm going to sort this out this morning. RE: hardware assertions - there are virtual Netscalers. I'm not sure if we have an established policy on how to handle hw.* vs virtual machines. I'm leaning towards going ahead an using it here. Any thoughts?

hdm · 2020-05-27T17:11:33Z

xml/sip_banners.xml

  </fingerprint>
-


Line breaks between fingerprints makes them much more readable, I guess you disagree?

The automation removes it.

OK, we can address in a future PR to auto-newline after each fingerprint, and update the automation to match, if yall agree.

I've no issues with adding line breaks.

xml/snmp_sysdescr.xml

xml/ntp_banners.xml

tsellers-r7 · 2020-06-03T12:32:22Z

xml/snmp_sysdescr.xml

-    <param pos="2" name="os.version"/>
+    <param pos="0" name="os.product" value="NetScaler"/>
+    <param pos="1" name="os.version"/>
+    <param pos="2" name="os.version.version"/>


@hdm @gwiseman-r7 - FYI, I've changed this fingerprint so as to return os.version that is consistent with other Netscaler fingerprints that return that same value. In doing so, I've made the fingerprint more specific by adding NS in ^NetScaler NS. In my surveys I don't see a banner without that NS.
The build number being captured in os.version.version should match what is found here: https://support.citrix.com/article/CTX121840#NS13

Any objections?

hdm · 2020-06-03T12:34:41Z

This all looks good here, thanks!

gwiseman-r7

lgtm

tsellers-r7 added 2 commits May 27, 2020 09:34

Fixup: Normalize descriptions

2628ef5

Update CPEs

72f9f69

tsellers-r7 requested a review from gwiseman-r7 May 27, 2020 16:59

tsellers-r7 commented May 27, 2020

View reviewed changes

hdm reviewed May 27, 2020

View reviewed changes

xml/dns_versionbind.xml Show resolved Hide resolved

hdm reviewed May 27, 2020

View reviewed changes

cpe-remap.yaml Outdated Show resolved Hide resolved

hdm reviewed May 27, 2020

View reviewed changes

xml/snmp_sysdescr.xml Show resolved Hide resolved

hdm reviewed May 27, 2020

View reviewed changes

xml/ntp_banners.xml Outdated Show resolved Hide resolved

Address Netscaler OS

718b8b8

tsellers-r7 commented Jun 3, 2020

View reviewed changes

gwiseman-r7 approved these changes Jun 3, 2020

View reviewed changes

MySQL: add some debian matches

f7a4f62

tsellers-r7 merged commit 41a57bf into rapid7:master Jun 3, 2020

tsellers-r7 deleted the fixup_post_264 branch June 3, 2020 18:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CPE Updates #267

CPE Updates #267

tsellers-r7 commented May 27, 2020

tsellers-r7 commented May 27, 2020

tsellers-r7 May 27, 2020

tsellers-r7 May 27, 2020

hdm May 27, 2020

tsellers-r7 May 27, 2020

hdm May 27, 2020

tsellers-r7 May 27, 2020

tsellers-r7 commented May 27, 2020

hdm May 27, 2020

tsellers-r7 May 27, 2020

hdm May 27, 2020

hdm May 27, 2020

tsellers-r7 May 27, 2020

tsellers-r7 Jun 3, 2020

hdm May 27, 2020

tsellers-r7 May 27, 2020

hdm May 27, 2020

tsellers-r7 May 27, 2020 •

edited

Loading

tsellers-r7 Jun 3, 2020

hdm commented Jun 3, 2020

gwiseman-r7 left a comment

CPE Updates #267

CPE Updates #267

Conversation

tsellers-r7 commented May 27, 2020

Description

Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

tsellers-r7 commented May 27, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tsellers-r7 commented May 27, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tsellers-r7 May 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hdm commented Jun 3, 2020

gwiseman-r7 left a comment

Choose a reason for hiding this comment

tsellers-r7 May 27, 2020 •

edited

Loading