-
Notifications
You must be signed in to change notification settings - Fork 187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement S3 bucket source #146
Conversation
} | ||
|
||
// download bucket content | ||
for object := range s3Client.ListObjects(ctxTimeout, bucket.Spec.BucketName, minio.ListObjectsOptions{Recursive: true}) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the ignores should already be taken into account here, as otherwise the revision will differ on any added file and/or change, while we want to ignore those.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Discussed this a bit further: in a future iteration we would like to switch to an approach where the .sourceignore
file is looked up first in the bucket, and combined with the .spec.ignore
if it exists.
We can then use the merged ignore patterns to efficiently download the files we want to include whilst calculating the revision checksum at the same time, as done in https://github.com/fluxcd/source-controller/blob/v0.0.16/controllers/storage.go#L178-L211
d5c9794
to
6f8c381
Compare
Changes:
Bucket
type to APIBucket
CRD and RBACBucketReconciler
using minio go clientFix: #145
Proposal: fluxcd/flux2#242