Gateway files very slow to load

[eefahy]

Files, large and small, have been loading very slowly on the gateway. Webserver metrics show an occasional correlation between slow loads and high outbound bandwidth. CPU, memory, and load stats are stable during long wait times. The issue appears to be at the IPFS level.

In response, the gateway has been rolled back to v0.4.15 which noticeably lessened CPU load and average wait times. We will be monitoring and analyzing performance over the next few days.

We apologize if this slow down have effected your use of the gateway and welcome your comments.

[momack2]

Some discuss issues re using the gateway to find content on personal nodes:

• https://discuss.ipfs.io/t/added-a-file-but-its-not-accessible-on-gateway/4667/21
• https://discuss.ipfs.io/t/problem-acessing-my-own-files-through-the-public-gateway/4563/6
• https://discuss.ipfs.io/t/unable-to-retrieve-files-via-gateway-ipfs-io/503/28

[scout]

Update: The gateway is now running a binary that contains

libp2p/go-libp2p-kad-dht#237 (DHT: adaptive dial queue for staging dials) + follow-up refinements
libp2p/go-libp2p-swarm#102

At the moment there is not an authoritative branch that represents all the changes due to the time it would take to resolve all gx required changes for such a thing.

We are seeing better load times however the issue is not yet resolved. Work continues....

[x5engine]

please in plain English what' does this mean? why some files are not loading? even from my node?

[mkg20001]

@x5engine I'm guessing there are a few bugs in IPFS that make the DHT and other parts of libp2p function only very slowly, the fixes for those have been implemented in the version of IPFS that the gateway runs, yet it was too much effort to publicly ship them at that time. Now they're merged and most likely shipped so just upgrading to the latest IPFS version should fix some of those issues that impact loading times.
But there are no guarantees and I'm guessing it's still a bit slow.

[askender]

@jbenet If I open ipfs companion chrome extension and local ipfs node, and it will never load the gateway url https://bafy**.ipfs.infura-ipfs.io/ (got context deadline exceeded after a long time) (never loaded unless I pinned the resource myself maybe). Other gateway are the same situation.
If I close this chrome extention, this url will be loaded soon.
extension options will not fix this.

If I use Brave browser, this url can be open quickly. And there is a open with ipfs button in address bar, and ipfs://bafy*** will not be openned ever. The setting doesn't fix this.
(I know I can import/pin this url to ipfs node by right click in the page, or I run ipfs add myself)

That's to say, if we choose to load ipfs resource from local, it will never timeout and auto retry the gateway.

[askender]

@jbenet If I open ipfs companion chrome extension and local ipfs node, and it takes very long time to load the gateway url https://bafy**.ipfs.infura-ipfs.io/ (never loaded unless I pinned the resource myself). Other gateway are the same situation. If I close this chrome extention, this url will be loaded soon. extension options will not fix this.

If I use Brave browser, this url can be open quickly. And there is a open with ipfs button in address bar, and ipfs://bafy*** will not be openned ever. The setting doesn't fix this. (I know I can import/pin this url to ipfs node by right click in the page, or I run ipfs add myself)

That's to say, if we choose to load ipfs resource from local, it will never timeout and auto retry the gateway.

Maybe someone want to retry this case, here is a url you can try yourself: https://bafybeiae5c2ip2xxd5vvbnteu2sxntnmvgqqyt3we2s3hzbbcfifdsz3ya.ipfs.infura-ipfs.io/
A real case of our ipfs&filecoin related open-source & open-data project. https://ccnft-test.dweb-lab.com/ We are building a open content community dapp.

[askender]

By the way, it seems it's a dead repo that havn't got any commit since 27 Apr 2020?
Sounds a lot like we don't use this repo anymore (and it should be archived). Does it true?
@gmas Thanks.
I will re-ask the question in other and right repo. ipfs/ipfs-companion#1042