Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support publishing new log entries to Pub/Sub topics #1580

Merged
merged 1 commit into from
Aug 28, 2023

Conversation

jalseth
Copy link
Contributor

@jalseth jalseth commented Jul 8, 2023

Summary

Adds initial support publishing new log entries to Pub/Sub topics. Interested parties can subscribe to the topic in order to receive notifications when new entries are added.

This relates to #191. Requires sigstore/protobuf-specs#86 to be merged.

Release Note

  • New features: Added support publishing new log entries to Pub/Sub topics
  • Config changes:
    • Added the rekor_server.new_entry_publisher flag to configure the Pub/Sub topic to send notifications to when a new log entry is added. This flag is optional.
    • Addded the rekor_server.publish_events_protobuf flag to configure whether events are published using the CloudEvents protobuf format.
    • Addded the rekor_server.publish_events_json flag to configure whether events are published using the CloudEvents JSON format.

Documentation

This is a TODO still. I would like feedback on the PR before writing the docs.

@jalseth
Copy link
Contributor Author

jalseth commented Jul 10, 2023

@bobcallaway Please note the build will fail as-is due to dependency on sigstore/protobuf-specs#86 which hasn't been merged yet. All tests pass running locally when using the not-yet-merged dependency: https://gist.github.com/jalseth/ebf866def89661d31fd87c93eed80ed2

Copy link
Contributor

@haydentherapper haydentherapper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic work on this!

cmd/rekor-server/app/root.go Outdated Show resolved Hide resolved
pkg/api/api.go Show resolved Hide resolved
cmd/rekor-server/app/root.go Outdated Show resolved Hide resolved
pkg/api/entries.go Show resolved Hide resolved
pkg/api/entries.go Show resolved Hide resolved
pkg/api/entries.go Outdated Show resolved Hide resolved
pkg/events/types.go Show resolved Hide resolved
pkg/events/newentry/new_entry.go Show resolved Hide resolved
pkg/tle/tle.go Outdated Show resolved Hide resolved
pkg/tle/tle.go Show resolved Hide resolved
pkg/tle/tle.go Outdated Show resolved Hide resolved
pkg/events/newentry/new_entry.go Show resolved Hide resolved
pkg/api/api.go Show resolved Hide resolved
@jalseth jalseth force-pushed the use-ce-proto branch 3 times, most recently from 15c34c1 to 7624f62 Compare July 30, 2023 23:13
@jalseth jalseth force-pushed the use-ce-proto branch 4 times, most recently from f83b1b3 to 7a161df Compare August 19, 2023 21:55
@codecov
Copy link

codecov bot commented Aug 21, 2023

Codecov Report

Merging #1580 (789ebf8) into main (ab09135) will decrease coverage by 0.10%.
Report is 2 commits behind head on main.
The diff coverage is 58.82%.

@@            Coverage Diff             @@
##             main    #1580      +/-   ##
==========================================
- Coverage   67.04%   66.94%   -0.10%     
==========================================
  Files          83       88       +5     
  Lines        8490     8804     +314     
==========================================
+ Hits         5692     5894     +202     
- Misses       2120     2213      +93     
- Partials      678      697      +19     
Flag Coverage Δ
e2etests 48.95% <54.48%> (+1.13%) ⬆️
unittests 47.61% <27.01%> (-0.43%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
cmd/rekor-server/app/serve.go 85.24% <ø> (-0.47%) ⬇️
pkg/api/metrics.go 100.00% <ø> (ø)
pkg/events/events.go 27.08% <27.08%> (ø)
pkg/events/types.go 48.57% <48.57%> (ø)
pkg/pubsub/publisher.go 50.00% <50.00%> (ø)
pkg/api/api.go 70.79% <62.50%> (-1.37%) ⬇️
pkg/api/entries.go 65.75% <74.00%> (+0.88%) ⬆️
pkg/pubsub/gcp/publisher.go 79.51% <79.51%> (ø)
pkg/tle/tle.go 93.54% <85.71%> (-2.75%) ⬇️
cmd/rekor-server/app/root.go 60.60% <100.00%> (+1.65%) ⬆️
... and 1 more

... and 5 files with indirect coverage changes

Copy link
Member

@bobcallaway bobcallaway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm overall, a couple questions

cmd/rekor-server/app/root.go Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
pkg/api/entries.go Outdated Show resolved Hide resolved
pkg/pubsub/gcp/publisher.go Show resolved Hide resolved
pkg/tle/tle.go Show resolved Hide resolved
Copy link
Contributor

@haydentherapper haydentherapper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with just a few nits

go.mod Outdated Show resolved Hide resolved
pkg/api/entries.go Show resolved Hide resolved
pkg/api/entries.go Outdated Show resolved Hide resolved
pkg/events/newentry/new_entry.go Show resolved Hide resolved
pkg/events/newentry/new_entry_test.go Show resolved Hide resolved
pkg/events/types.go Show resolved Hide resolved
@jalseth jalseth force-pushed the use-ce-proto branch 4 times, most recently from 771eccc to ffa4537 Compare August 25, 2023 15:47
Adds initial support publishing new log entries to Pub/Sub topics. Interested
parties can subscribe to the topic in order to receive notifications when new
entries are added.

Signed-off-by: James Alseth <james@jalseth.me>
@bobcallaway bobcallaway merged commit a49cd04 into sigstore:main Aug 28, 2023
14 checks passed
@github-actions github-actions bot added this to the v1.2.2 milestone Aug 28, 2023
@jalseth jalseth deleted the use-ce-proto branch December 28, 2023 07:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants