Add request-id to log entries, log as json, add sql log

[alikins]

This is more or less the config I'm using for local devel stuff at the moment.
Can simplify/split commits/move to defaults.py etc as needed.

Add 'request_id' and json formatting to galaxy logs

This adds a log_request_id.filters.RequestIDFilter log filter
to the console/stdout log handlers. The filter adds the
request-id to log records so they can be used by the formatters.

Also add and setup a 'jog.JogFormatter' for formatting normal
python log entries as JSON.

Add RequestIdMiddleware to MIDDLEWARE_CLASSES

This enables adding or tracking a X-Request-Id per
request that can be used in logging.

https://github.com/dabapps/django-log-request-id

various tweaks of log levels up to DEBUG-ish levels

[chouseknecht]

Pushed a commit that sets the config via a function. Seems a little more concise to me, but if you hate it, feel free to revert. Fixed the lint issue too.

Tried it out locally by setting DJANGO_SETTINGS_MODULE=galaxy.settings.production and giving GALAXY_SECRET_KEY a value. Logging comes out as a JSON object with request_id populated.

Wasn't sure if there is some magic to getting SQL to appear in the log, other than turning on logging for django.db.backend. That's OK though. I don't think we want all the SQL logged. In a production setting, I think it will be way too verbose.

Overall LGTM.

[chouseknecht]

Back port to release/3.0.8

[alikins]

@chouseknecht

Wasn't sure if there is some magic to getting SQL to appear in the log, other than turning on logging for django.db.backend. That's OK though. I don't think we want all the SQL logged. In a production setting, I think it will be way too verbose.

No other magic required to log it. The previous versions of this branch had changes to improve the output and added a handler, formatter, and filters for SQL. The handler was just to log to a separate file (useful for localhost dev work, but not useful in openshift). The custom formatter just makes the output easier to read. The custom filter was used to ignore the large number of queries made by celery it uses to poll for new tasks.

[alikins]

I don't think we want all the SQL logged. In a production setting, I think it will be way too verbose.

Definitely too verbose in production.

One thing we could investigate for the future would be allowing log levels to be increased for particular authenticated users. That could include SQL logging. We did this for the RHSM server (candlepin) and it was very useful for tracking down problems that only showed up for some users.

[alikins]

I think I left an unneeded ']' in there that can be removed.

[alikins]

@chouseknecht

Pushed a commit that sets the config via a function. Seems a little more concise to me, but if you hate it, feel free to revert

My preference is the explicit data style, since you usually end up needing the granularity, but if the method works, thats okay for now.

[alikins]

LGTM