Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for encoding URL-safe base64 #53

Closed
wants to merge 9 commits into from

Conversation

jessetane
Copy link
Contributor

There are a bunch of modules on npm that do this already, but it seems like most of them depend on another module (sometimes buffer itself) and then add regex replace or similar, or include lots of features I don't want. Since we already do decoding here, I figure adding encoding just makes things symmetrical and so doesn't feel too out of place.

Note that performance remains the same with this approach but there is a tiny memory penalty (the extra alphabet itself should only be another ~128 bytes?). Ideas for other approaches or references to alternate modules welcome.

@waitingsong

This comment has been minimized.

Copy link
Collaborator

@feross feross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If performance is not affected, which it sounds like it's not, then I'm happy to merge this improvement. See inline comments.

@@ -50,11 +53,12 @@ function _byteLength (b64, validLen, placeHoldersLen) {
return ((validLen + placeHoldersLen) * 3 / 4) - placeHoldersLen
}

function toByteArray (b64) {
function toByteArray (b64, alphabet) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does decoding need to take an alphabet? You set revLookups.url = revLookups._ anyway, so this seems like it adds nothing. Am I missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i felt like keeping the external api symmetrical was valuable, i think i prefer it this way but happy to change if you like.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I agree that symmetry is nice, but I think that's outweighed by having a parameter that has no effect. Can you change it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. how much do you want to remove? the entire revlookups table? as in, go back to having a single variable? if anyone wants to experiment with or add additional alphabets later it will be much more difficult...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

basically, now that i'm looking at it, i can see it's not just the interface, it's the implementation as well. the parameter doesn't have "no effect", it's just that the default alphabet happens to include a hack (for node compatibility) to allow decoding url safe base 64, so we're reusing the default reverse lookup for both alphabets to save a little memory.

so yea we could remove just the parameter from the interface, but then why bother keeping the symmetrical implementation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it also seems like it might be worth at least encouraging user code to be specific about what is being decoded if possible - i'm thinking that reading b64.toByteArray(str, 'url') in my source might save me some confusion down the road

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and just to really beat it to death... from https://tools.ietf.org/html/rfc4648#section-5:

This encoding may be referred to as "base64url".  This encoding
should not be regarded as the same as the "base64" encoding and
should not be referred to as only "base64".  Unless clarified
otherwise, "base64" refers to the base 64 in the previous section.

index.js Outdated Show resolved Hide resolved
@bradisbell
Copy link

Any additional thoughts on merging this? Before I reinvent the wheel, I'd rather switch to your existing implementation.

@jessetane
Copy link
Contributor Author

oops, had forgotten about this. I personally still like having the (totally optional!) alphabet arg, but Feross is right that it should "do something". I just pushed a few commits to try and clear up the difference between real base64url support and the node compat hack where the default alphabet also decodes base64url, see what you think.

jrschumacher added a commit to jrschumacher/buffer that referenced this pull request May 17, 2022
Using string replaceAll since base64 is external base64-js.
Dependency does not yet support base64url beatgammit/base64-js#53
TomMettam pushed a commit to CasperTech/buffer that referenced this pull request Jun 7, 2022
Using string replaceAll since base64 is external base64-js.
Dependency does not yet support base64url beatgammit/base64-js#53
@kwikwag
Copy link

kwikwag commented Dec 17, 2023

I implemented an extra optional opts argument to fromArrayBuffer in my fork. I did so before seeing this pull request, so maybe there are some ideas from here I can include there and vice versa. Anyway, that branch also includes modifications inspired by other forks. I can pick up this pull request, if anyone is still maintaining the package.

jrschumacher added a commit to jrschumacher/buffer that referenced this pull request Jul 22, 2024
Using string replaceAll since base64 is external base64-js.
Dependency does not yet support base64url beatgammit/base64-js#53
@jonkoops
Copy link

Hi @jessetane, thanks for the pull request. We have moved the repository, if you are still interested in making a contribution please open a new pull request there.

@feross feross closed this Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants