Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It doesn't appear possible to list organization or folder audit log entries #399

Closed
quantumew opened this issue Sep 20, 2021 · 3 comments
Closed
Assignees
Labels
api: logging Issues related to the googleapis/python-logging API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@quantumew
Copy link

quantumew commented Sep 20, 2021

It doesn't appear possible to list organization or folder audit log entries. I am hoping to use this SDK to poll all audit logs
for all orgs, folders, and projects, then push them to a stream for consumption by our log aggregation system.

Environment details

  • OS type and version: AWS Lambda
  • Python version: 3.8.10
  • pip version: 21.2.4
  • google-cloud-logging version: 2.6.0

Steps to reproduce

  1. Retrieve logs for an organization or folder
  2. Iterate through pages, it results in a ValueError

Code example

client = Client(project=GCP_ADMIN_PROJECT, _use_grpc=0)
iterator = client.list_entries(
       resource_names=["folders/xxx"],        
       order_by=ASCENDING,
       page_size=200,
)

Stack trace

{
  "errorMessage": "path \"folders/xxxx/logs/cloudaudit.googleapis.com%2Fdata_access\" did not match expected pattern \"\n    projects/            # static prefix\n    (?P<project>[^/]+)   # initial letter, wordchars + hyphen\n    /logs/               # static midfix\n    (?P<name>[^/]+)      # initial letter, wordchars + allowed punc\n\"",
  "errorType": "ValueError",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 32, in lambda_handler\n    run(resource_id)\n",
    "  File \"/var/task/timeout_decorator/timeout_decorator.py\", line 82, in new_function\n    return function(*args, **kwargs)\n",
    "  File \"/var/task/lambda_function.py\", line 49, in run\n    flush_logs(resource_id, bookmarks)\n",
    "  File \"/var/task/lambda_function.py\", line 80, in flush_logs\n    for entry in list(page):\n",
    "  File \"/var/task/google/api_core/page_iterator.py\", line 131, in __next__\n    result = self._item_to_value(self._parent, item)\n",
    "  File \"/var/task/google/cloud/logging_v2/_http.py\", line 496, in _item_to_entry\n    return entry_from_resource(resource, iterator.client, loggers)\n",
    "  File \"/var/task/google/cloud/logging_v2/_helpers.py\", line 87, in entry_from_resource\n    return ProtobufEntry.from_api_repr(resource, client, loggers=loggers)\n",
    "  File \"/var/task/google/cloud/logging_v2/entries.py\", line 167, in from_api_repr\n    logger_name = logger_name_from_path(logger_fullname)\n",
    "  File \"/var/task/google/cloud/logging_v2/entries.py\", line 61, in logger_name_from_path\n    return _name_from_project_path(path, None, _LOGGER_TEMPLATE)\n",
    "  File \"/var/task/google/cloud/_helpers.py\", line 488, in _name_from_project_path\n    raise ValueError(\n"
  ]
}
@product-auto-label product-auto-label bot added the api: logging Issues related to the googleapis/python-logging API. label Sep 20, 2021
@daniel-sanche daniel-sanche added priority: p3 Desirable enhancement or fix. May not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Oct 5, 2021
@daniel-sanche
Copy link
Contributor

Thanks, I'll take a look

@daniel-sanche daniel-sanche added this to the v3.0.0 Breaking milestone Oct 5, 2021
@daniel-sanche
Copy link
Contributor

daniel-sanche commented Oct 5, 2021

Possibly related to #257

@daniel-sanche
Copy link
Contributor

I opened a PR, and this should be fixed in the upcoming v3.0.0 release. In the meantime, note that this may not be a problem if you use grpc, since that uses a different code path:

client = Client(project=GCP_ADMIN_PROJECT, _use_grpc=1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: logging Issues related to the googleapis/python-logging API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

2 participants