This repository has been archived by the owner on Apr 29, 2020. It is now read-only.
refactor: exported file links #13
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously we used `getMany` to get **ALL** of the links for a particular file node before passing them on to the next step in the pipeline. This PR refactors that to get the blocks _individually in parallel_, allowing them to be passed on to the next stage in the pipeline as they arrive. I don't know if this is faster than the previous method. In fact, I can see how it might be slightly slower. However, it does mean that we don't have to wait for **ALL** linked blocks to arrive before they are passed to the next stage in the pipeline and by doing this it will reduce the time to first byte when using `ipfs.cat*Stream` which is good from a UX perspective. License: MIT Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
ghost
assigned 
|
This PR was created while I was looking into ipfs/js-ipfs#1706 (comment) - it's worth having a quick read of the discussion there! |
|
A streaming |
|
I'm going to close this for now - |
|
|
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously we used
getManyto get ALL of the links for a particular file node before passing them on to the next step in the pipeline.This PR refactors that to get the blocks individually in parallel, allowing them to be passed on to the next stage in the pipeline as they arrive.
I don't know if this is faster than the previous method. In fact, I can see how it might be slower. However, it does mean that we don't have to wait for ALL linked blocks to arrive before they are passed to the next stage in the pipeline and by doing this it will reduce the time to first byte when using
ipfs.cat*Streamwhich is good from a UX perspective.A streaming
getManyfrom IPLD would actually be useful here, which could be pushed down to bitswap so that it can make good decisions about what network requests to make.cc @vmx