Suggestion for using stdlib BoundLogger with third-party apps.

[gilbsgilbs]

Hi there,

Here are some suggestions to improve stdlib. It allows to use the stdlib.BoundLogger, even when defining a root handler (to pass third-party apps logs through structlog's pipeline). Not sure my approach is really logical and elegant though (maybe closer to a POC) and suggestions/improvements are welcome.

• Add a stdlib.Handler class that helps forward third-party apps logs through structlog. May fit to most use cases. However, can lead to recursion issues if you persist on using stdlib.LogFactory. To overcome this, I did some extra-work (see second bullet-point).

  Typical use-case: output third-party apps + structlog logs to stdout.

import logging
import sys

import structlog


root_logger = logging.getLogger()
handler = structlog.stdlib.Handler()
root_logger.addHandler(handler)
root_logger.setLevel(logging.INFO)

structlog.configure(
    processors=[
        structlog.stdlib.add_log_level,
        structlog.stdlib.PositionalArgumentsFormatter(),
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
        # structlog.processors.JSONRenderer(),
        structlog.dev.ConsoleRenderer(),
    ],
    context_class=dict,
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True,
)

logging.getLogger('foo').warning("logging!")
logging.getLogger('foo.bar').warning("logging again!")

structlog.get_logger().warning("we can also log from structlog!")

• Add a stdlib._HandlerWrapper class and a stdlib.wrap_handler function that allows to output structlog log through logging modules handlers (this required some more changes in stdlib.LoggerFactory to work properly). This wrapper prevents structlog from recursing.

Typical use-case: output third-party apps through structlog, and even output everything to vanilla handlers.

import io
import logging
import sys

import structlog


root_logger = logging.getLogger()
s = io.StringIO()
handler = structlog.stdlib.wrap_handler(logging.StreamHandler(s))()
root_logger.addHandler(handler)
root_logger.setLevel(logging.INFO)

structlog.configure(
    processors=[
        structlog.stdlib.add_log_level,
        structlog.stdlib.PositionalArgumentsFormatter(),
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
        # structlog.processors.JSONRenderer(),
        structlog.dev.ConsoleRenderer(),
    ],
    context_class=dict,
    wrapper_class=structlog.stdlib.BoundLogger,
    logger_factory=structlog.stdlib.LoggerFactory(handlers=root_logger.handlers),
    cache_logger_on_first_use=True,
)

logging.getLogger('foo').warning("logging!")
logging.getLogger('foo.bar').warning("logging again!")

structlog.get_logger().warning("we can also log from structlog!")

print(s.getvalue())  # Everything was logged

Thus, this allows to use stdlib.filter_by_level and probably some other stuff I did not think of.

Cheers.

[codecov-io]

Current coverage is 100% (diff: 100%)

Merging #95 into master will not change coverage

@@           master   #95   diff @@
===================================
  Files          13    13          
  Lines         739   771    +32   
  Methods         0     0          
  Messages        0     0          
  Branches       91    95     +4   
===================================
+ Hits          739   771    +32   
  Misses          0     0          
  Partials        0     0          

Powered by Codecov. Last update e9fdd96...b51b724

[hynek]

Sorry I didn’t get around to look at this before. Do I see it correctly, that it’s basically an attempt to codify the FAQ entry about getting stdlib logs into structlog? It’s kind of funny that there’s an PR open to do vice versa at the same time. :)

[gilbsgilbs]

@hynek

Do I see it correctly, that it’s basically an attempt to codify the FAQ entry about getting stdlib logs into structlog?

Not only. This is an attempt to fix #9 . It allows more things than the doc suggests (e.g. output logs to any stdlib handler ; make possible to use stdlib filters such as structlog.stdlib.filter_by_level). It tries overcomes the infinite-recursion issue by not propagating the handler.

[hynek]

How relevant would you see this with the new approaches that have been introduced in 17.1? ISTM this is a closed chapter but I don’t want to just close the PR without consulting with you.

[gilbsgilbs]

@hynek That is just great :) . Thanks! 👍