Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore --raw-leaves when data fits in one block #6940

Closed
achingbrain opened this issue Mar 1, 2020 · 4 comments
Closed

Ignore --raw-leaves when data fits in one block #6940

achingbrain opened this issue Mar 1, 2020 · 4 comments
Labels
kind/enhancement A net-new feature or improvement to an existing feature

Comments

@achingbrain
Copy link
Member

Following on from ipfs-inactive/js-ipfs-unixfs-importer#44 (comment)

If we are importing a file, the returned CID should resolve to a DAG that represents a UnixFS entry. That entry should allow for metadata such as modes, mtimes, etc.

If the --raw-leaves flag has been set, and the data fits in one block, the --raw-leaves flag should be ignored and the data inlined* into the root dag-pb node.

Current behaviour is to return a CID that resolves to a raw node, eg a buffer - this cannot have metadata set on it.

js-IPFS has implemented the change to only output UnixFS things from an import, this issue is to track the go-IPFS version.

cc: @Stebalien @hsanjuan


* = js-ipfs-unixfs-importer has a reduceSingleLeafToSelf option which allows the user to control whether or not to inline the file data into the root dag-pb node, it's not currently exposed by the ipfs.add command though doing that might be something to entertain. Maybe with a shorter name..

@achingbrain achingbrain added the kind/enhancement A net-new feature or improvement to an existing feature label Mar 1, 2020
achingbrain added a commit to ipfs/interop that referenced this issue Mar 1, 2020
Disabled temporarily until ipfs/kubo#6940
is implemented in go-IPFS.
@Stebalien
Copy link
Member

This change was never agreed on. Raw leaves are still valid files, they just can't carry metadata. If there's no metadata, there's no reason not to save some space and use a raw leaf.

@achingbrain
Copy link
Member Author

Ah, I think I misread your comment then.

For now, I'd ignore raw leaves when we have metadata and everything fits into a single file.

I guess the distinction is to only do this when we have metadata, not all the time.

achingbrain added a commit to ipfs/js-ipfs-unixfs that referenced this issue Mar 2, 2020
Following further conversation in ipfs/kubo#6940
it seems for small files it would be better to only ignore the `--raw-leaves`
flag when metadata is actually present.
@Stebalien
Copy link
Member

Yeah, I know it's just a few bytes, but I'd like to be as efficient as possible.

(and, as usual, I'd like to avoid changing hashes unless we have a pressing reason to do so)

achingbrain added a commit to ipfs/js-ipfs-unixfs that referenced this issue Mar 3, 2020
#44)

Following further conversation in ipfs/kubo#6940
it seems for small files it would be better to only ignore the `--raw-leaves`
flag when metadata is actually present.
@achingbrain
Copy link
Member Author

Ok, this can probably be closed then as the work will be done as part of #6920 which will need to pass the interface and interop tests to be considered done, which include tests for this case so it won't be missed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement A net-new feature or improvement to an existing feature
Projects
None yet
Development

No branches or pull requests

2 participants