Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #15668 to 7.x: Support application default credentials (ADC) for Google Pub/Sub #17790

Merged
merged 2 commits into from
May 4, 2020

Conversation

andrewkroh
Copy link
Member

Cherry-pick of PR #15668 to 7.x branch. Original message:

What does this PR do?

Update the Google Pub/Sub input to support reading Application Default Credentials
(ADC) in addition to the credentials_file and credentials_json config options.

If neither config option is set then it will attempt to search for the default credentials.
Generally this means reading the GOOGLE_APPLICATION_CREDENTIALS environment
variable plus searching a few other well known locations.

The googlecloud module was updates to support all three authentication mechanisms.

Why is it important?

Without this change, Filebeat requires a path to a file with credentials to be provided in its config and is unable to use for example a default service account.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • [ ]

How to test this PR locally

  1. Upload the Filebeat binary to a GCP instance created with default config
  2. Run the binary configured to read from PubSub, but do not provide any credentials
  3. Filebeat should be able to detect that it's on GCP and use ADC (get a token for the default service account for the VM from the metadata server)

Related issues

Use cases

Sending logs to Elastic from PubSub queue

Screenshots

Logs

@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

…stic#15668)

Update the Google Pub/Sub input to support reading Application Default Credentials
(ADC) in addition to the credentials_file and credentials_json config options.

If neither config option is set then it will attempt to search for the default credentials.
Generally this means reading the GOOGLE_APPLICATION_CREDENTIALS environment
variable plus search a few other well known locations.

The googlecloud module was updates to support all three authentication mechanisms.

Co-authored-by: Michal Wasilewski <mwasilewski@gmx.com>
Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
(cherry picked from commit 058de35)
@andrewkroh andrewkroh force-pushed the backport_15668_7.x branch from cb8e67c to 17614aa Compare May 2, 2020 02:38
@andrewkroh andrewkroh requested a review from kaiyan-sheng May 2, 2020 02:39
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview stats

Expand to view the summary

Build stats

Test stats 🧪

Test Results
Failed 0
Passed 1177
Skipped 124
Total 1301

Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewkroh andrewkroh merged commit 22ba00e into elastic:7.x May 4, 2020
@andrewkroh andrewkroh deleted the backport_15668_7.x branch January 14, 2022 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants