-
Notifications
You must be signed in to change notification settings - Fork 875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Map log levels to StackDriver's severity #726
Comments
Thanks for the hint. That is not really helping us, though. For a bit of background: we're running GKE and log our logs according to Kubernetes/GKE standard way just to stdout. The heavy lifting is then done by Kubernetes and GKE. I'd like it to keep it that way and not switch internally to use StackDriver logging directly in some of our services. Therefore, it would still be great to have a mapping function to accomplish this and it could also be generic to serve more use-cases. |
@marco-jantke did you solve this issue? I tested pino in Google Cloud Run with the following options:
The log entries was not picked up correctly. Are you sure that stack driver picks up severity as integers or do you need to use the string names instead? (https://cloud.google.com/run/docs/logging#container-logs). The following format seems to work fine: |
Hi @pebo, no I'm not really sure. I was following the documentation, but did not verify. It might be that I understood sth wrong, I'd more rely on your testings. Currently, we filter based on the JSON value of the severity field and not use the stackdriver native way. We didn't see a possibility to output the logs in the format you specified, that's why I opened the issue initially. Thanks for the collaboration! |
It would be nice with an official solution, but here's a node module that outputs logs in a stackdriver friendly format.
Even if we define custom log levels it seems like a level |
We have just landed support for the formatters #775. Would any of you check if they could improve things? |
@mcollina Thanks for the update, this seems like a great feature! With the The new
If a request was made with the trace id header Finally pino v6 will also allow us to drop the extra version field so this is looking really promising! |
The formatters introduced in v6 works great with GCP / StackDriver logging. Example mapping:
|
Would you mind to send us a a bit of docs related to this? It can also be a nice little module tbh. |
Sure. Note that Google has recently rebranded Google Stackdriver Logs to Google Cloud Logging. Not sure where the best documentation can be found but here's a starting point: https://cloud.google.com/run/docs/logging#writing_structured_logs We are using pinojs in Cloud Run services so rather than logging via the Google Cloud logging API we simply write to stdout. The logs are picked up by the google infrastructure which adds some context like the service name, service version and a timestamp. Google Cloud Logging supports structured logging so a pinojs api call like The level must be named The message text payload should be named We are also using a traceId to connect log events belonging to the same HTTP request. The Google Cloud Run infrastructure will set a header |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Just wanted to say thanks for this thread, this is a good solution for me too for running my services in the Google Cloud. |
Is there some action to be taken here? Or can we close this issue? |
Since Google Cloud / StackDriver is pretty popular, I think it would be good to either add this solution to the documentation, or perhaps even provide a built-in way to use |
PRs are welcome. |
I have tried out almost all suggestions here. GKE plus Cloud Logging are not a happy bunch. Everything still shows up as info. I tried the level function to introduce the severity key. Even the severity key does not show up. Anything I log gets swallowed into the ‘jsonPaylod’ attribute. I am not sure what I am doing wrong. Official loggers and even ‘Pino-stackdriver’ requires a credential file which seems a bit much. |
A GDE friend pointed me to https://stackoverflow.com/questions/49283233/json-format-for-stackdriver-logging-in-google-kubernetes-engine. |
yes. That friend also says that there is something funky going on with StackDriver as I had a similar problem last week with a Cloud Function that logged warnings as info. I will ping the Google community and see what happens. |
I finally got it working with the following setup. However, same setup did not seem to work with version 5.x but worked with 6.x version of pino. 🤷♂️
|
pino v5 does not support the formatters.m option. Could you send a PR to add this to our docs? Thanks! |
@mcollina Is the FAQ page the right place for such documentation ? Sorry for the stupid question. I am happy to send a PR |
Yeah, the FAQ is totally the correct place. |
Thanks @rahulballal, @pebo and everybody. |
Just a quick note, when logging an object it won't work, add log formatter like below: const productionPinoConfig: pino.LoggerOptions = {
prettyPrint: false,
formatters: {
level(label: string, number: number) {
return {
severity: PinoLevelToSeverityLookup[label] || PinoLevelToSeverityLookup['info'],
level: number,
}
},
log(payload: any) {
return { message: payload }
},
},
} |
Would you like to send a PR to update the documentation? |
There you go: #878 |
FWIW I've requested that GCP provide a plugin for pino as to offload the config to the plugin, posting in this closed issue as I think it would interest folks in this thread: googleapis/nodejs-logging#875 |
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
We have the requirement to use StackDriver's severity for our logs. That means the logs should contain a field
severity
with some integer value mapping to StackDriver's constants. Is there a canonical way to do this with pino? I couldn't figure out how to do it with the current API without using a custom serializer.If not do you think it's interesting for more library users to build in hooks in order to modify the fields on a central place or can you think of something more usable?
The text was updated successfully, but these errors were encountered: