Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for custom log formats to be defined in stacker.yaml #705

Merged
merged 1 commit into from
Mar 6, 2019
Merged

Allow for custom log formats to be defined in stacker.yaml #705

merged 1 commit into from
Mar 6, 2019

Conversation

russellballestrini
Copy link
Member

@russellballestrini russellballestrini commented Mar 4, 2019

At Remind we run stacker on a large portion of our stacks in stage and production concurrently using continuous delivery. This means our logs are interlaced between stage and prod and are hard to use.

Also since we use continuous delivery we want to put the deployment_id or git commit hash on every log line so that we can aggregate them all up with our log parsing and viewing tools.

This PR makes it possible to set a custom log format from stacker.yaml.

modified:   .gitignore
modified:   stacker/commands/stacker/__init__.py
modified:   stacker/commands/stacker/base.py
modified:   stacker/config/__init__.py
modified:   stacker/logger/__init__.py
new file:   stacker/tests/fixtures/not-basic.env
new file:   stacker/tests/fixtures/vpc-custom-log-format-info.yaml
modified:   stacker/tests/test_stacker.py

Copy link
Contributor

@ejholmes ejholmes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems pretty reasonable to me, and seems pretty common through the python world to be able to do this.

I'll let @phobologic do the final review, but I'm 👍

@@ -0,0 +1,24 @@
log_format_info: "[%(asctime)s] ${environment} custom log format - %(message)s"
Copy link
Contributor

@ejholmes ejholmes Mar 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In stacker.logger.setup_logging we also support color and debug keys. Should we just make this a nested structure?

Example

log_format:
  info: "[%(asctime)s] ${environment} custom log format - %(message)s"
  color: # Some color format
  debug: # Some debug format

@russellballestrini russellballestrini changed the title Allow for custom log formats to be defined in stacks.yaml Allow for custom log formats to be defined in stacker.yaml Mar 5, 2019
Copy link
Member

@phobologic phobologic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me - a few minor comments, and I wonder if we could have tests that actually test that the log formats are changing. Not sure that's super important though.

@@ -68,3 +68,5 @@ dev.yaml
dev.env

tests/fixtures/blueprints/*-result

FakeKey.pem
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this about?

Copy link
Member Author

@russellballestrini russellballestrini Mar 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some other new test added testing for keys (#700), and it doesn't seem to clean up after itself. Since I run tests locally they show up in my repo after running make test.

docs/config.rst Outdated

You may optionally provide a custom log format for stacker to use::

# optional log format overrides.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth putting the defaults here, or at least explaining how we use the python logging module standard format variables (link to here? https://docs.python.org/2.7/library/logging.html#logrecord-attributes)

Docs and config model adjustment.

	modified:   .gitignore
	modified:   docs/config.rst
	modified:   stacker/commands/stacker/__init__.py
	modified:   stacker/commands/stacker/base.py
	modified:   stacker/config/__init__.py
	modified:   stacker/logger/__init__.py
	new file:   stacker/tests/fixtures/not-basic.env
	new file:   stacker/tests/fixtures/vpc-custom-log-format-info.yaml
	modified:   stacker/tests/test_stacker.py
@russellballestrini russellballestrini merged commit 3f97c9f into cloudtools:master Mar 6, 2019
@russellballestrini russellballestrini deleted the rb-custom-log-formats branch March 6, 2019 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants