Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix color of timestamp of first log entry #73

Closed
wants to merge 1 commit into from
Closed

Fix color of timestamp of first log entry #73

wants to merge 1 commit into from

Conversation

fabianbuechler
Copy link
Contributor

The timestamp of the first log entry appeared in black instead of white without sending RESET_ALL as first element.

The timestamp of the first log entry appeared in black instead of white without sending `RESET_ALL` as first element.
@codecov-io
Copy link

codecov-io commented May 31, 2016

Current coverage is 91.09% (diff: 0.00%)

Merging #73 into master will decrease coverage by 8.90%

@@           master        #73   diff @@
========================================
  Files          13         13          
  Lines         739        696    -43   
  Methods         0          0          
  Messages        0          0          
  Branches       91         87     -4   
========================================
- Hits          739        634   -105   
- Misses          0         58    +58   
- Partials        0          4     +4   

Powered by Codecov. Last update c256057...8342adf

@hynek
Copy link
Owner

hynek commented May 31, 2016

heh you broke every single test :)

@fabianbuechler
Copy link
Contributor Author

Hm,... saw that when I got the coverage report from Codecov.
Thought I could get away with just editing the file online on Github ;)
If I find the time I'll redo the pull-request with some more effort, but for now: is the problem clear?

@hynek
Copy link
Owner

hynek commented May 31, 2016

to be honest not quite. the timestamps are always dimmer white for me…is some kind of formatting from your shell prompt bleeding into this? (that would be completely ok and should be fixed, I’m just trying to understand what’s going on)

@fabianbuechler
Copy link
Contributor Author

Huh, you are right. I just installed the project I'm working on on my Linux box at home with fish shell, and everything works fine.
The problem seems to be related to my work setup with Windows + Git shell + Conemu Terminal. That could be a bit too exotic ;)

I'll try to give some details once I'm back at work tomorrow.

@hynek
Copy link
Owner

hynek commented May 31, 2016

Please do

@fabianbuechler
Copy link
Contributor Author

So, back at work - here are some facts:

I'm using Windows 10, Python 3.4 or 3.5 in a virtualenv, everything is running in a Git-for-Windows Terminal in a ConEmu Window.

When I log something using the ConsoleRenderer, the first timestamp comes out black:
structlog_first_timestamp_black

However, the text is printed just fine, since selecting it reveals it:
structlog_first_timestamp_black_selected

My logging configuration looks like this:

import logging

import structlog


class ProcessorFormatter(logging.Formatter):
    """Custom stdlib logging formatter for structlog ``event_dict`` messages.

    Apply a structlog processor to the ``event_dict`` passed as
    ``LogRecord.msg`` to convert it to loggable format (a string).

    """

    def __init__(self, processor, fmt=None, datefmt=None, style='%'):
        """Keep reference to the ``processor``."""
        super().__init__(fmt=fmt, datefmt=datefmt, style=style)
        self.processor = processor

    def format(self, record):
        """Extract structlog's ``event_dict`` from ``record.msg``.

        Process a copy of ``record.msg`` since the some processors modify the
        ``event_dict`` and the ``LogRecord`` will be used for multiple
        formatting runs.

        """
        if isinstance(record.msg, dict):
            msg_repr = self.processor(
                record._logger, record._name, record.msg.copy())
            return msg_repr
        return record.getMessage()


def event_dict_to_message(logger, name, event_dict):
    """Pass the event_dict to stdlib handler for special formatting."""
    return ((event_dict,), {'extra': {'_logger': logger, '_name': name}})



# Configure structlog stack.
structlog.configure(
    processors=[
        structlog.stdlib.filter_by_level,
        structlog.stdlib.add_logger_name,
        structlog.stdlib.add_log_level,
        structlog.stdlib.PositionalArgumentsFormatter(),
        structlog.processors.TimeStamper(
            fmt='%Y-%m-%dT%H:%M:%S', utc=False),
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
        # Do not include last processor that converts to a string for
        # stdlib since we leave that to the handler's formatter.
        event_dict_to_message,
    ],
    context_class=dict,
    logger_factory=structlog.stdlib.LoggerFactory(),
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True,
)

# Cap console and file logs to user-defined level.
level = logging._nameToLevel.get(self.config['log_level'], logging.INFO)

# Use a StreamHandler with a ConsoleRenderer.
handler_stream = logging.StreamHandler(sys.stdout)
handler_stream.setFormatter(
    ProcessorFormatter(processor=structlog.dev.ConsoleRenderer()))
handler_stream.setLevel(level)

# And use a FileHandler with a JSONRenderer, cap to user-defined level.
handler_file = logging.handlers.TimedRotatingFileHandler(
    self.config['logfile'], encoding='utf-8',
    # Rotate logfiles weekly on Monday, keep last 10 weeks.
    when='W0', atTime=datetime.time(0, 0, 0), backupCount=10)
handler_file.setFormatter(
    ProcessorFormatter(processor=structlog.processors.JSONRenderer()))
handler_file.setLevel(level)

# HWcom logger to debug level.
logger = structlog.get_logger('hwcom')
logger.setLevel(logging.DEBUG)

# Apply the handlers only to the root logger.
logging.root.setLevel(logging.WARNING)
logging.root.addHandler(handler_stream)
logging.root.addHandler(handler_file)

These are the installed packages (pip list):

  • backports.shutil-get-terminal-size (1.0.0)
  • colorama (0.3.7)
  • coverage (4.1)
  • Cython (0.24)
  • decorator (4.0.9)
  • docutils (0.12)
  • flake8 (2.5.4)
  • future (0.15.2)
  • httpie (0.9.3)
  • hwcom (0.3.3, c:\users\bue\projects\hwcom)
  • ipdb (0.10.0)
  • ipython (4.2.0)
  • ipython-genutils (0.1.0)
  • Kivy-Garden (0.1.4)
  • mccabe (0.4.0)
  • pefile (2016.3.28)
  • pep257 (0.7.0)
  • pep8 (1.7.0)
  • pickleshare (0.7.2)
  • pip (8.1.2)
  • py (1.4.31)
  • pyflakes (1.0.0)
  • Pygments (2.1.3)
  • PyInstaller (3.2)
  • pypiwin32 (219)
  • pyreadline (2.1)
  • pytest (2.9.1)
  • requests (2.10.0)
  • restructuredtext-lint (0.14.2)
  • setuptools (21.2.1)
  • simplegeneric (0.8.1)
  • six (1.10.0)
  • structlog (16.0.0)
  • traitlets (4.2.1)
  • voluptuous (0.8.11)
  • Werkzeug (0.11.9)
  • wheel (0.29.0)

Not sure what other information could be relevant. Do you have any additional questions on how to replicate this?

@hynek
Copy link
Owner

hynek commented Jun 2, 2016

Nah, just fix the tests. :)

@fabianbuechler
Copy link
Contributor Author

Ok, I'll try as soon as I find the time.

@insolite
Copy link

@fabianbuechler Solution with ProcessorFormatter looks good for setting different renderers for different loggers using native logging formatters. And it is the only way to do this in structlog that I found (please correct me if I'm wrong). So how about posting pull request with this feature? I can do it for you if you want, cause I really need it in my projects. Also this could be useful along with #78 that I've posted recently.

@fabianbuechler
Copy link
Contributor Author

@insolite I don't really manage to create a pullrequest for this at the moment. Lot's of work with the kids... ;)

Feel free to create a PR with my implementation if you like.

@hynek
Copy link
Owner

hynek commented Apr 24, 2017

FYI ProcessorFormatter has been added via #105

@hynek hynek closed this Apr 24, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants