js-ipfs Not finding file that is hosted by 4 peers

[obelisk-dev]

Not able to find files I was able to find using jsIpfs. I have confirmed I can find these using go-IPFS and through public gateways. (this has worked in the past with previous versions of JS-IPFS)

• 0.34.4
• browser
• chrome

Type: Question

Steps to reproduce the error:

  let scriptURL = "https://unpkg.com/ipfs@0.34.4/dist/index.min.js";
            $.getScript(scriptURL, async ()=>{

                const repoPath = 'ipfs-repo'
                
                global.ipfs = new Ipfs({ 

                    start: true,
                    repo: 'ipfs-testing',
                    config: {
                        Bootstrap: [
                        "/dns4/ams-1.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLer265NRgSp2LA3dPaeykiS1J6DifTC88f5uVQKNAd",
                        "/dns4/sfo-1.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLju6m7xTh3DuokvT3886QRYqxAzb1kShaanJgW36yx",
                        "/dns4/lon-1.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLMeWqB7YGVLJN3pNLQpmmEk35v6wYtsMGLzSr5QBU3",
                        "/dns4/sfo-2.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLnSGccFuZQJzRadHn95W2CrSFmZuTdDWP8HXaHca9z",
                        "/dns4/sfo-3.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLPppuBtQSGwKDZT2M73ULpjvfd3aZ6ha4oFGL1KrGM",
                        "/dns4/sgp-1.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLSafTMBsPKadTEgaXctDQVcqN88CNLHXMkTNwMKPnu",
                        "/dns4/nyc-1.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLueR4xBeUbY9WZ9xGUUxunbKWcrNFTDAadQJmocnWm",
                        "/dns4/nyc-2.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64",
                        "/ip4/104.131.131.82/tcp/4001/ipfs/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"

                        ]
                    }
                
                });

                global.ipfs.on('ready', async () => { 

                    console.log(await ipfs.swarm.addrs())

                    let hash = 'QmS8hKtny73KoTh8cL2xLoLHjMDjkG9WYF3AHAdd9PHkxo';
                    global.ipfs.get(hash).then(files => {
                        console.log(files)
                    })
             }); 
  }); 

[obelisk-dev]

Was there changes recently that would cause issues finding files that are only hosted through go-ipfs?

[cretz]

While I can't help with the problem, do you happen to know the last version where this worked? Was it 0.34.3?

[obelisk-dev]

I believe so. Are you able to find QmS8hKtny73KoTh8cL2xLoLHjMDjkG9WYF3AHAdd9PHkxo with js-ipfs?

[cretz]

I'm not in an easy position to check, sorry.

[alanshaw]

@ethericsol if you add the preload nodes too your bootstrap config are you able to find the content?

js-ipfs/src/core/runtime/config-browser.js

Lines 26 to 27 in e6e749d

	'/dns4/node0.preload.ipfs.io/tcp/443/wss/ipfs/QmZMxNdpMkewiVZLMRxaNxUeZpDUb34pWjZ1kZvsd16Zic',
	'/dns4/node1.preload.ipfs.io/tcp/443/wss/ipfs/Qmbut9Ywz9YEDrz8ySBSgWyJk41Uvm2QJPhwDJzJyGFsD6'

[obelisk-dev]

Hey @alanshaw. thanks for taking a look. No that didn't seem to to it. Not sure what has changed recently that I'm no longer able to add and cat files using JS-ipfs like I was in previous weeks.

In the current state of jsIpfs, should an example like this work? https://codepen.io/Heish/pen/rPvvOp

[alanshaw]

In the current state of jsIpfs, should an example like this work? https://codepen.io/Heish/pen/rPvvOp

Not without the preload nodes in the bootstrap.

Separately, there's an issue with the gateway go-ipfs nodes right now that is in the process of being resolved which is the likely cause of you not being able to get hold of your data from them.

[obelisk-dev]

So in this example I am trying to pull a small file that is accessible on the gateway, with the correct bootstrap config. Is this also caused by the issue listed above, even though the file is accessible on the gateway?

https://ipfs.io/ipfs/QmRTPMX1c7hQnymBNyCeAWQTbm8t6KA4vcwVNFFpErKpjR

[alanshaw]

That looks like it should work, let me take a closer look.

[bgits]

This issue might be related: #1880 (comment)

[bgits]

I created a fiddle that replicates the problem: https://jsfiddle.net/bgvianyc/edkgqxcs/

[obelisk-dev]

Hey @alanshaw , have you heard any update on this being fixed? Seems like it affecting quite a few people.

[comntr]

What's the last working version of js-ipfs? I've checked everything from 0.26 to 0.34, but none worked.

[alanshaw]

@comntr this is an issue with the preload nodes JS IPFS uses, not with JS IPFS itself. This is what I’m seeing
ipfs/go-bitswap#68

[bgits]

@comntr this is an issue with the preload nodes JS IPFS uses, not with JS IPFS itself. This is what I’m seeing
ipfs/go-bitswap#68

Is there an alternative configuration using a different set of nodes that will currently work?

[comntr]

What are the "preload" nodes and why are they necessary to run js-ipfs? Isn't it meant to be p2p and resilient to failure of some nodes?

[obelisk-dev]

As far as I understand some features like DHT routing aren't implemented in js-ipfs yet. Right now most of the functionality relies proper communication with the preload nodes, which is currently not happening correctly. I maybe wrong but this is what I have gathered so far.

[ultraschuppi]

@alanshaw thanks for your effort on ipfs/go-bitswap#68 - does it maybe help to remove the preload-nodes completely from bootstrap?

[alanshaw]

does it maybe help to remove the preload-nodes completely from bootstrap?

No, the preload nodes are required for you to be able to load content from nodes that you do not have a direct connection to.

[alanshaw]

What are the "preload" nodes and why are they necessary to run js-ipfs?

They aren't necessary to run JS IPFS but they are necessary for JS IPFS to find content from nodes that it is not connected to directly.

JS IPFS doesn't have a DHT yet (it's coming soon!), so the only way it can get hold of content is to ask peers it already knows about for the content. If the peers it knows about don't have the content then it won't be able to find it (unless one or more of those peers is setup as a relay).

The preload nodes are just some of the bootstrap nodes that all IPFS instances connect to by default. When you add a file to JS IPFS, it sends a request to the preload nodes to tell them to get that content.

The preload nodes either a) already have the content b) know of a node they're directly connected to what has it or c) find a node that does have it via the DHT. Once they find the content they store it themselves.

The same thing happens when you try and get a file - JS IPFS sends a request to the preload nodes to tell them to get that content. Once they have it, they can provide it to JS IPFS because they're directly connected to it.

[ultraschuppi]

does it maybe help to remove the preload-nodes completely from bootstrap?

No, the preload nodes are required for you to be able to load content from nodes that you do not have a direct connection to.

lets say all my clients are connected to one peer i control - would this work? could be a temporary (centralized) workaround

[ghost]

Js-ipfs seems to have resumed work and can now be uploaded successfully.

[ghost]

Now it's ok!

[bgits]

Js-ipfs seems to have resumed work and can now be uploaded successfully.

The issue is not with uploading, it's with retrieving content that is not cached locally. Using this fiddle it still seems to not work: https://jsfiddle.net/bgvianyc/edkgqxcs/

[obelisk-dev]

@bgits agreed. The original codepen i posted https://codepen.io/Heish/pen/MLBZvb is still unable to find a file that is accessible through the public gateway.

https://ipfs.io/ipfs/QmRTPMX1c7hQnymBNyCeAWQTbm8t6KA4vcwVNFFpErKpjR

[alanshaw]

@ethericsol you had some errors in your fiddle - I fixed it up and it is now working for me https://jsfiddle.net/ujn8zbg7/

[bgits]

@ethericsol you had some errors in your fiddle - I fixed it up and it is now working for me https://jsfiddle.net/ujn8zbg7/

I'm trying to understand the difference. Was passing in the arrow function the requirement to make it work?