Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow structure logging JSON to be customized #42486

Closed
philwebb opened this issue Sep 30, 2024 · 6 comments
Closed

Allow structure logging JSON to be customized #42486

philwebb opened this issue Sep 30, 2024 · 6 comments
Assignees
@philwebb
Labels
type: enhancement A general enhancement
Milestone
3.4.0-RC1

Comments

@philwebb
Copy link
Member

From https://x.com/jkuipers/status/1840849120874950716

GYwChGvXgAA7T3a
@philwebb
Copy link
Member Author

Several interesting ideas we could consider:

  • Should you be able to rename Logstash field names? I assumed they were standard, but it seems the Logstash appender allows renames.
  • Should we offer a quick way to include additional fields with static values? Service name is interesting because we do have a way to set that with ECS.
  • Should we offer a general MDC filtering mechanism?
  • Should we offer a general stacktrace post-processing mechanism?

@philwebb philwebb added status: waiting-for-triage An issue we've not yet triaged for: team-meeting An issue we'd like to discuss as a team to make progress labels Sep 30, 2024
@mp911de
Copy link
Member

mp911de commented Oct 1, 2024

You can draw some ideas from https://logging.paluch.biz/ that included stack trace filtering, dynamic MDC inclusion/exclusion and field name handling back from when that project has been active.

@philwebb
Copy link
Member Author

philwebb commented Oct 2, 2024

We discussed this today and we'd like to offer some enhancements if we can. I'm not sure how much we'll be able to get into 3.4 at this point. We might take stacktrace post-processing off the table for now.

@philwebb philwebb added type: enhancement A general enhancement and removed status: waiting-for-triage An issue we've not yet triaged for: team-meeting An issue we'd like to discuss as a team to make progress labels Oct 2, 2024
@philwebb philwebb added this to the 3.x milestone Oct 2, 2024
@philwebb philwebb changed the title Consider logstash enhancements Allow structure logging JSON to be customized Oct 10, 2024
philwebb added a commit that referenced this issue Oct 11, 2024
@philwebb
Add filter, name processor and value processor support to JsonWriter
27c59b8 
Update `JsonWriter` to support filtering and processing of names/values.
This update will allow us to offer better customization options with
structured logging.

See gh-42486
@philwebb philwebb modified the milestones: 3.x, 3.4.0-RC1 Oct 11, 2024
@maxxedev
Copy link

maxxedev commented Nov 4, 2024 •
edited
Loading

  • Renaming @ field like @timestamp to timestamp, for example, is tricky. The property needs to be:
logging.structured.json.rename[@timestamp]=timestamp

Consider documenting a tip

  • Is it possible to customize timestamp value? Can LOG_DATEFORMAT_PATTERN be applied there? Or change logstash default to match LOG_DATEFORMAT_PATTERN default of 3 digits sub-second precision?

@wilkinsona
Copy link
Member

Thanks for the suggestion, @maxxedev. Please open a new issue so that we can consider it.

@philwebb
Copy link
Member Author

philwebb commented Nov 4, 2024

Issue was opened as #42980

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@philwebb philwebb

Labels
type: enhancement A general enhancement
Projects
None yet
Milestone
3.4.0-RC1
Development

No branches or pull requests
4 participants
@philwebb @wilkinsona @mp911de @maxxedev