Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check content-type in importer to warn against unexpected kubevirt imports #2909

Merged
merged 2 commits into from
Nov 9, 2023
Merged

Check content-type in importer to warn against unexpected kubevirt imports #2909

merged 2 commits into from
Nov 9, 2023

Conversation

alromeros
Copy link
Collaborator

@alromeros alromeros commented Sep 26, 2023 •
edited
Loading

What this PR does / why we need it:

In some network environments, proxy servers or firewalls may block access to certain external resources. Although we'd expect them to return a 4xx (Forbidden/Unauthorized) error code, some of these servers return an HTML error page and a 200 error code. Currently, CDI checks if the content is compressed or a qcow2 image. If neither of these are detected it assumes it is a raw disk image.

Even though these servers are misbehaving we can provide a better experience. When importing an kubevirt disk image using a DataVolume, we can also check the HTTP content-type header. If the media type is reported as 'text/html' (or text/*) we can assume that the server has sent a notice page instead of the requested content.

This pull request adds a log message to warn against the described behavior.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Release note:

Check content-type to warn against unexpected imports

@kubevirt-bot kubevirt-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note Denotes a PR that will be considered when it comes time to generate release notes. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Sep 26, 2023
@alromeros alromeros changed the title [WIP] Check content-type in importer to avoid unexpected imports [WIP] Check content-type in importer to avoid unexpected kubevirt imports Sep 26, 2023
@alromeros alromeros changed the title [WIP] Check content-type in importer to avoid unexpected kubevirt imports Check content-type in importer to avoid unexpected kubevirt imports Sep 26, 2023
@kubevirt-bot kubevirt-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 26, 2023
@alromeros alromeros changed the title Check content-type in importer to avoid unexpected kubevirt imports [WIP] Check content-type in importer to avoid unexpected kubevirt imports Sep 26, 2023
@kubevirt-bot kubevirt-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 26, 2023
@akalenyu
Copy link
Collaborator

So IIUC the tests fail because our local test servers return text/* content type?

@alromeros
Copy link
Collaborator Author

So IIUC the tests fail because our local test servers return text/* content type?

That's it, still have to check if it's our proxy who's behaving badly or just something we should allow (maybe just avoid text/html type).

@alromeros alromeros changed the title [WIP] Check content-type in importer to avoid unexpected kubevirt imports Check content-type in importer to warn against unexpected kubevirt imports Oct 18, 2023
@kubevirt-bot kubevirt-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 18, 2023
@alromeros
Copy link
Collaborator Author

alromeros commented Oct 18, 2023 •
edited
Loading

Following some offline discussions, I've been toying with the idea of having a Prometheus metric to get important import information into the PVC or DataVolume (like this specific case). However, I think this might be outside the scope of the PR so we can merge this and groom the idea a bit further. Rejecting the import seems overkill, I think CDI is functioning as expected right now.

akalenyu
akalenyu reviewed Oct 18, 2023
View reviewed changes
Copy link
Collaborator

@akalenyu akalenyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

pkg/importer/http-datasource.go Outdated
nbdkitSocket = "/tmp/nbdkit.sock"
defaultUserAgent = "cdi-golang-importer"
httpContentType = "Content-Type"
httpContentLenght = "Content-Length"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lenght typo

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, fixed!

@kubevirt-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akalenyu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubevirt-bot kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 18, 2023
@alromeros
Check content-type before importing to warn against unexpected conten…
5a3de5a 
…t imports

This commit adds a check in the http-datasource format reader to trigger a warning when importing unexpected content-types.

Signed-off-by: Alvaro Romero <alromero@redhat.com>
@alromeros
Copy link
Collaborator Author

/retest-required

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Nov 1, 2023
@alromeros
Copy link
Collaborator Author

/test pull-containerized-data-importer-e2e-nfs

@alromeros
Copy link
Collaborator Author

/hold
Let's see if I can improve the testing a little

@kubevirt-bot kubevirt-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 2, 2023
@kubevirt-bot kubevirt-bot removed the lgtm Indicates that a PR is ready to be merged. label Nov 2, 2023
@alromeros
Copy link
Collaborator Author

/unhold

@kubevirt-bot kubevirt-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 2, 2023
@alromeros
Copy link
Collaborator Author

/test pull-cdi-linter

@alromeros
Test import with unexpected content-type
4fe9f7d 
This commit adds a test to cover a import with unexpected content-type. It also updates the badserver to allow mocking this behavior.

Signed-off-by: Alvaro Romero <alromero@redhat.com>
@alromeros
Copy link
Collaborator Author

/retest-required

@alromeros alromeros requested a review from awels November 8, 2023 08:48
@awels
Copy link
Member

awels commented Nov 8, 2023

/lgtm

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Nov 8, 2023
@awels
Copy link
Member

awels commented Nov 8, 2023

/cherrypick release-v1.58

@kubevirt-bot
Copy link
Contributor

@awels: once the present PR merges, I will cherry-pick it on top of release-v1.58 in a new PR and assign it to you.

In response to this:

/cherrypick release-v1.58

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kubevirt-bot kubevirt-bot merged commit 56fb051 into kubevirt:main Nov 9, 2023
10 of 18 checks passed
@kubevirt-bot
Copy link
Contributor

@awels: new pull request created: #2975

In response to this:

/cherrypick release-v1.58

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kubevirt-bot kubevirt-bot mentioned this pull request Nov 9, 2023
Merged
@alromeros alromeros mentioned this pull request Mar 8, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@akalenyu akalenyu akalenyu left review comments

@aglitke aglitke Awaiting requested review from aglitke

@maya-r maya-r Awaiting requested review from maya-r

@awels awels Awaiting requested review from awels

Assignees

@awels awels

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@alromeros @akalenyu @kubevirt-bot @awels