Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept Uint8Array in place of Buffer #12

Closed
Gozala opened this issue Jun 8, 2020 · 3 comments · Fixed by #13
Closed

Accept Uint8Array in place of Buffer #12

Gozala opened this issue Jun 8, 2020 · 3 comments · Fixed by #13
Labels
need/triage Needs initial labeling and prioritization

Comments

@Gozala
Copy link
Contributor

Gozala commented Jun 8, 2020

Changes to dag-pb attempted to add support for Uint8Arrays in place of node Buffer to better support case where data is passed across web workers. @achingbrain had a thought that it might be better yet to add support for Uint8Arrays at this layer instead.

Opening issue to track / discuss this here.

@Gozala Gozala added the need/triage Needs initial labeling and prioritization label Jun 8, 2020
@welcome
Copy link

welcome bot commented Jun 8, 2020

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

  • "Priority" labels will show how urgent this is for the team.
  • "Status" labels will show if this is ready to be worked on, blocked, or in progress.
  • "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

@Gozala
Copy link
Contributor Author

Gozala commented Jun 8, 2020

While I'm inclined to remove dependency on Buffer all together here, I do fear that this might break some assumptions in users of this library (72 on npm).

More pragmatic approach would be to just expect Uint8Array instance anywhere instead of Buffer instances, but keep return values to remain `Buffer.

@Gozala
Copy link
Contributor Author

Gozala commented Jun 8, 2020

Is the assumption here that val is either node Buffer or a string ?

var encode = function (val, buffer, offset) {
var oldOffset = offset
var len = bufferLength(val)
varint.encode(len, buffer, offset)
offset += varint.encode.bytes
if (Buffer.isBuffer(val)) val.copy(buffer, offset)
else buffer.write(val, offset, len)
offset += len
encode.bytes = offset - oldOffset
return buffer
}

Without necessarily understanding all the details here, it appears to me that supporting strings here is a bad idea. Because detecting byteLength would require one iteration and actual encoding would perform another iteration. Accepting just Uint8Arrays would avoid this because any strings would have need to be encoded first and byteLength info will be available right away.

This probably out of scope for this issue, but thought I'd still post it here.

achingbrain pushed a commit that referenced this issue Jun 24, 2020
With this change all the `Buffer` inputs were replaced by `Uint8Array` which is backwards compatible since `Buffer` is a subclass.

This does make an assumption that "bytes" encoder used to take `Buffer|string` here:

https://github.com/ipfs/protons/blob/3b5276f052f2e17c2d806d27cd9a88e156588977/src/compile/encodings.js#L30-L43


Fixes #12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need/triage Needs initial labeling and prioritization
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant