-
Notifications
You must be signed in to change notification settings - Fork 185
Fix base64 padding for kube config #79
Fix base64 padding for kube config #79
Conversation
Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA. It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
Codecov Report
@@ Coverage Diff @@
## master #79 +/- ##
==========================================
+ Coverage 92.11% 92.13% +0.02%
==========================================
Files 13 13
Lines 1217 1234 +17
==========================================
+ Hits 1121 1137 +16
- Misses 96 97 +1
Continue to review full report at Codecov.
|
@bpicolo When could we get this fix? We need to run with python 3.6.5 |
/assign @roycaihw . (as recommended by mrsbot) |
if PY3: | ||
jwt_attributes = json.loads( | ||
base64.b64decode(parts[1]).decode('utf-8') | ||
base64.b64decode(parts[1] + padding).decode('utf-8') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
JWT requires base64.urlsafe_b64decode
instead of base64.b64decode
(and for encoding as well), as +
and /
are not url safe
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be still worthwhile to do safe decode here?
config/kube_config_test.py
Outdated
@@ -87,11 +91,13 @@ def _raise_exception(st): | |||
|
|||
TEST_OIDC_TOKEN = "test-oidc-token" | |||
TEST_OIDC_INFO = "{\"name\": \"test\"}" | |||
TEST_OIDC_BASE = _base64(TEST_OIDC_TOKEN) + "." + _base64(TEST_OIDC_INFO) | |||
TEST_OIDC_BASE = _unpadded_base64( | |||
TEST_OIDC_TOKEN) + "." + _unpadded_base64(TEST_OIDC_INFO) | |||
TEST_OIDC_LOGIN = TEST_OIDC_BASE + "." + TEST_CLIENT_CERT_BASE64 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The signature (TEST_CLIENT_CERT_BASE64
) should also be url safe (base64url encoded and padding omitted).
@@ -257,13 +257,15 @@ def _load_oid_token(self, provider): | |||
if len(parts) != 3: # Not a valid JWT |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add a check to make sure the JWT doesn't contain url reserved characters (=
, +
and /
)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for sending this PR :) I left some comments.
After more reading, I'm thinking if the previous padding implementation (padding with According to base64 encoding and JWS doc, base64 (also apply to base64url) padding can only have three cases:
Having three "=" padding characters is illegal. According to https://gist.github.com/perrygeo/ee7c65bb1541ff6ac770 and python binascii_a2b_base64 implementation, extra paddings after previous quad are ignored. Therefore for a valid base64-encoded string with padding stripped, adding "==" would guarantee it can be safely decoded, and for an illegal string (e.g. "A===" being "A" when stripped) adding "==" would raise error. In Python 3.7 the implementation distinguishes illegal string ("A==") from incorrect padding ("AQ="): https://github.com/python/cpython/blob/3.7/Modules/binascii.c#L513-L521. Prior 3.7 the error messages are the same "Incorrect padding". #65 had this issue because we didn't implement padding for python 3. The padding implementation for python 2 was correct (probably intended?) but it didn't distinguish illegal string from incorrect padding. I suggest we could add illegal base64url string check like https://tools.ietf.org/html/rfc7515#appendix-C cc @ltamaster who implemented oidc auth |
@roycaihw fair - I didn't notice the addendum in the JWS RFC. What's the optimal path forward? |
@bpicolo I think your fix is fine for both Python 2 and 3. I'd suggest that we can add a check and raise error if Please also fix the decoding and testcases around oidc id-token (JWT) to use base64url-encoding/decoding. |
@roycaihw Had the first chance/attention span in, clearly quite a while, to address the things we talked about long ago, because I get emails about this on occasion from people referencing it, heh. The actionables I tweaked:
Hopefully this is good to go, though a potentially useful follow up might be noisier auth errors |
Hi @bpicolo - |
@micw523 Done. Where is code coverage report now? Last one I see in this thread ran in July on an entirely different branch |
I think the bot refreshes its comment every once a while - it should refresh itself. Otherwise, the CI says your coverage is 92%. |
Ahhh I see that now, gotcha. Side note - these tests are super tough to get running locally (And the latest kubernetes/ seems to have a dependency glitch with adal so I had to edit the ./run_tox script to target a specific branch locally), and because they run in a bash subprocess they're even harder to debug. It's probably worth adding some documentation to help people get started here. That said, looks like coveragebot is happy now? |
Are you running into import problems with adal? I think part of the reason might be that you opened this PR a long time ago and the python repo has ditched adal as an absolute requirement. It was a recent change. Fast forwarding to the master branch might help? I agree on the difficulty on running the tests. The documentation for this library is quite sparse in its current state. The bot has indeed updated itself. |
config/kube_config.py
Outdated
|
||
if any(char in token for char in reserved_characters): | ||
# Invalid jwt, as it contains url-unsafe chars | ||
return None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like that in the original code a single return
is used. Is it possible if we make the return statements consistent?
if PY3: | ||
jwt_attributes = json.loads( | ||
base64.b64decode(parts[1]).decode('utf-8') | ||
base64.b64decode(parts[1] + padding).decode('utf-8') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be still worthwhile to do safe decode here?
@micw523 cleaned up the return. re: safe decode (can't reply to that comment), we're breaking above if there are url-unsafe characters, so it should be safe as is. In theory, this is ready, but I no longer have a setup locally to be able to try it live. Apologies for the delay here - pretty far out of my mind at this point, heh. Not sure what's up with those new test errors ... doesn't seem code related? |
Just pinging this thread to see if there are updates. This bug still blocks our development without manually editing the offending line. Is this PR just awaiting lgtm? |
Sorry for the delay in review. I will do another pass this week |
Any news on this? |
my apologies.. I was busy with k/k 1.15 freeze. It looks like my comments were addressed a long time ago :)
Fair enough. I re-ran the travis CI and the /lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bpicolo, roycaihw The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Just installed released 10.0.0 fresh from pip and tested against the Pacific Research Platform k8s cluster using python3 and it works now! Thank You! |
…stream-release-9.0-base64-patch Automated cherry pick of #79
…stream-release-8.0-base64-patch Automated cherry pick of #79
Fixes #65 (pulled in / updated another open PR to get this fix moving + tweaked tests)
JWT spec requires tokens be de-padded base64 (so that they're URLsafe), but they need to be re-padded to properly decode.