Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add Azure Blob Provisioner extension #1536

Conversation

paullatzelsperger
Copy link
Contributor

WHAT

This PR backports the upstream provision-blob extension that allows for creating blob containers and temporary SAS tokens.
Also adds a End-To-End test for cloud transfers, because up until now, we only had tests that solely used the dataplane.

Note that the S3 Provisioner was not backported, because the are some inconsistencies with the endpoint override. For some SDK interactions, the endpoint override can be specified dynamically, for others it is based on a configuration value. That may not be usable in a practical environment.

Technically, this would have meant:

  • significant changes to the AWS SDK client instantiation are necessary (cf this PR)
  • the S3 provisioner itself needs some work to tunnel through the override

WHY

Allowing the creation of temporary SAS tokens when using Azure Blob Storage as data destination.

FURTHER NOTES

  • Once this PR is merged, it should be cherry-picked into the bugfix/0.7.5 branch
  • The Azure Blob provisioner was copied into the edc-extensions/backport directory

Closes # <-- insert Issue number if one exists

@paullatzelsperger paullatzelsperger added bug Something isn't working critical All issues that will definitely be in the milestone they are planned for labels Sep 6, 2024
@paullatzelsperger paullatzelsperger self-assigned this Sep 6, 2024
@paullatzelsperger
Copy link
Contributor Author

@rafaelmag110 FYI

Copy link

sonarqubecloud bot commented Sep 6, 2024

@paullatzelsperger paullatzelsperger merged commit 3f66ebd into eclipse-tractusx:main Sep 6, 2024
33 checks passed
@paullatzelsperger paullatzelsperger deleted the fix/az_s3_provision_extension branch September 6, 2024 10:50
paullatzelsperger added a commit to paullatzelsperger/tractusx-edc that referenced this pull request Sep 6, 2024
* add dedicated cloud e2e test module

* added test without container

* add s3 provision and e2e test (wip)

* backport/copy upstream AZ Blob Provisioner

* add s3 destination credentials

* fix lic header format

* avoid NPE in the generator

* DEPENDENCIES

* re-enable FCC crawling in tests
paullatzelsperger added a commit that referenced this pull request Sep 16, 2024
* fix: add Azure Blob Provisioner extension (#1536)

* add dedicated cloud e2e test module

* added test without container

* add s3 provision and e2e test (wip)

* backport/copy upstream AZ Blob Provisioner

* add s3 destination credentials

* fix lic header format

* avoid NPE in the generator

* DEPENDENCIES

* re-enable FCC crawling in tests

* DEPENDENCIES

* add documentation

* add JSON annotations

* fix tests
ndr-brt pushed a commit that referenced this pull request Sep 24, 2024
* fix: add Azure Blob Provisioner extension (#1536)

* add dedicated cloud e2e test module

* added test without container

* add s3 provision and e2e test (wip)

* backport/copy upstream AZ Blob Provisioner

* add s3 destination credentials

* fix lic header format

* avoid NPE in the generator

* DEPENDENCIES

* re-enable FCC crawling in tests

* DEPENDENCIES

* add documentation

* add JSON annotations

* fix tests

* Remove Transfer DataPlane Signaling Dependencies redundancy.

* Updated DEPENDENCIES.

---------

Co-authored-by: Paul Latzelsperger <43503240+paullatzelsperger@users.noreply.github.com>
Co-authored-by: Paul Latzelsperger <paul.latzelsperger@beardyinc.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working critical All issues that will definitely be in the milestone they are planned for
Projects
Status: Merged
Development

Successfully merging this pull request may close these issues.

2 participants