Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert protocol buffers into JSON objects #1421

Open
wunderlejohn opened this issue May 31, 2023 · 1 comment
Open

Convert protocol buffers into JSON objects #1421

wunderlejohn opened this issue May 31, 2023 · 1 comment
Assignees
Labels
api: logging Issues related to the googleapis/nodejs-logging API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@wunderlejohn
Copy link

Is there any movement here on a basic example of convert protocol buffers into JSON objects? feel like that's a stumbling block many would run into using this client library. I'm currently struggling to get buffer data of type 'type.googleapis.com/google.cloud.audit.AuditLog' into anything that might be usable. Have tried installing 'proto3-json-serializer', 'protobufjs' and 'google-proto-files' and following this documentation without any luck.

Originally posted by @wunderlejohn in #785 (comment)

@product-auto-label product-auto-label bot added the api: logging Issues related to the googleapis/nodejs-logging API. label May 31, 2023
@losalex losalex added type: question Request for information or clarification. Not an issue. priority: p3 Desirable enhancement or fix. May not be included in next release. labels May 31, 2023
@losalex losalex removed their assignment Jul 3, 2023
@cindy-peng cindy-peng added priority: p3 Desirable enhancement or fix. May not be included in next release. and removed priority: p3 Desirable enhancement or fix. May not be included in next release. labels Jan 9, 2024
@mlitvinav
Copy link

mlitvinav commented Apr 3, 2024

Hi @wunderlejohn, i am able to parse the log entries into json as follows. The following code is Bun with TypeScript and i am only looking for audit logs in my case.
I did not manage to make it work with the included protos and had to use the google-proto-files package.

  "devDependencies": {
    "@types/bun": "latest"
  },
  "peerDependencies": {
    "typescript": "^5.0.0"
  },
  "dependencies": {
    "@google-cloud/logging": "^11.0.0",
    "google-proto-files": "^4.2.0"
  }

You will need to login into the right google project via gcloud auth application-default login in order to run this.

import { Logging } from '@google-cloud/logging'
import protofiles from 'google-proto-files'

const logging = new Logging()

async function listLogEntries() {
   const filter = `protoPayload.@type="type.googleapis.com/google.cloud.audit.AuditLog"`;
   const [entries] = await logging.getEntries({
      autoPaginate: false,
      filter,
      pageSize: 10,
      pageToken: undefined,
      orderBy: 'timestamp desc',
   });

   for (const entry of entries) {
      const payload = (entry as any).metadata.payload as string // typemismatch; value is present but invisible
      if (payload == 'protoPayload' && Buffer.isBuffer(entry.metadata[payload]?.value)) {
         const protopath = protofiles.getProtoPath('../google/cloud/audit/audit_log.proto')
         const root = protofiles.loadSync(protopath)
         const type = root.lookupType('google.cloud.audit.AuditLog')
         const value = type.decode(entry.metadata[payload]?.value as Uint8Array).toJSON()
         console.log(value) // the same structured log you see in Google Logging Explorer
      }
   }
}
await listLogEntries().catch(console.error)

@triplequark triplequark added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed type: question Request for information or clarification. Not an issue. labels Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: logging Issues related to the googleapis/nodejs-logging API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

5 participants