Unify HTTP and UPnP Server header fingerprinting

[jhart-r7]

Description

I've removed the UPnP database (hence the version bump) and moved it as is (minus the recently added Sonos fingerprints, which would be duplicate) to the HTTP Server header fingerprinting because for our purposes these are the same protocols and keeping them separate only causes maintenance headaches.

Motivation and Context

Drastically improve HTTP and UPnP coverage while simplifying maintenance.

How Has This Been Tested?

rspec passes

I also used the most recent set of HTTP and UPnP study data from recog to see if/how fingerprinting would improve. Previously using the HTTP Server fingerprint against the HTTP studies failed to matched 37074 unique Server headers, and after this change it failed to match 34994. Given that each unique header generally corresponds to several IPs, the actual impact is much larger than the difference of 2080 implies.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• I have updated the documentation accordingly (or changes are not required).
• I have added tests to cover my changes (or new tests are not required).
• All new and existing tests passed.

[rkirk-r7]

Will need to get Nexpose updated for this change

[tsellers-r7]

Are there any concerns about the loss of the protocol="ssdp" that would have been contained in the results emitted when using /upnp_banners.xml?

[hdm]

For Metasploit, it doesn't care about the protocol field, but MDM is hard-wired for the upnp_banners.xml. This is an easy fix on the MDM side: https://github.com/rapid7/metasploit_data_models/blob/master/lib/mdm/host/operating_system_normalization.rb#L222

[jhart-r7]

@tsellers-r7 yes, it is somewhat unfortunate that we are losing this, but other than private/personal code I don't believe any integrating products are currently using the protocol field.

[jhart-r7]

Thanks for the reminder on MDM @hdm, I've opened rapid7/metasploit_data_models#175