Use centralized function/objects for all network access

[pombredanne]

The network calls are scattered in many places and this is problem.

• this is technical debt
• we cannot log things easily Implement structured logging #590
• we should disable etags but this is in many places Disable Etag #321
• we cannot easily add proxy support at once Support for http/socks proxy #559
• we cannot implement API politeness at once Be polite to APIs #506 nginx import -> Name or service not known #568

[abhi-kr-2100]

I skimmed the code and found a fetch_response utility function in vulnerabilities/utils.py, more fetch_* functions/methods purpose-built for their respective importers, and fetch methods various VersionAPI subclasses.

Before I try to fix this issue, I would like to know if this is what you're talking about as well? Perhaps, you know of some more?

My plan to fix this:

1. Understand how each importer/VersionAPI fetches data.
2. Separate what's general to every importer/VersionAPI from what's special to each.
3. Design a base Fetcher class (analogous to the Importer class).
4. Specialize (through inheritance) the Fetcher class for an importer/VersionAPI as necessary.

I'm contributing for the first time and will appreciate all the advice and constructive criticism I get get. :)