Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breadcrumb context menu slow to appear when user has many local pins #1618

Closed
lidel opened this issue Sep 10, 2020 · 2 comments
Closed

Breadcrumb context menu slow to appear when user has many local pins #1618

lidel opened this issue Sep 10, 2020 · 2 comments
Labels
effort/days Estimated to take multiple days, but less than a week exp/intermediate Prior experience is likely helpful kind/bug A bug in existing code (including security flaws) kind/maintenance Work required to avoid breaking changes or harm to project's status quo need/analysis Needs further analysis before proceeding need/maintainers-input Needs input from the current maintainer(s) P2 Medium: Good to have, but can wait until someone steps up
Milestone

Comments

@lidel
Copy link
Member

lidel commented Sep 10, 2020

Something to be aware of: the more Pins user has, the slower pin.ls is.
I have around 1k of pins and it takes 10 seconds to do pin check against all pinned blocks via URL executed by my browser:

$ time curl -X POST 'http://127.0.0.1:5001/api/v0/pin/ls?paths=bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi&stream=true&arg=bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi'
{"Message":"path 'bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi' is not pinned","Code":0,"Type":"error"}
curl -X POST   0.00s user 0.00s system 0% cpu 10.774 total

This is result of inefficiencies in go-ipfs 0.6 (pinstore needs revamp, I use flatfs on slow spinning disk) but illustrates that if we block UI on pin status check, it may take 10 seconds or more.

I suggest we are cognizant of this possibility and audit our UI, decouppling "realtime" elements (breadcrumbs context menu – #1599) from pin ls for snappier interface.

For example, when pinning service integration lands (ipfs/ipfs-gui#91), we could have static action in context menus to open "pin status modal" and fetch pin status (local+remote) only there.

@lidel lidel added kind/bug A bug in existing code (including security flaws) exp/intermediate Prior experience is likely helpful P2 Medium: Good to have, but can wait until someone steps up kind/maintenance Work required to avoid breaking changes or harm to project's status quo need/analysis Needs further analysis before proceeding need/maintainers-input Needs input from the current maintainer(s) effort/days Estimated to take multiple days, but less than a week labels Sep 10, 2020
@lidel
Copy link
Member Author

lidel commented Sep 10, 2020

I believe its slow because you do pin ls which by default works against all pinned blocks (including indirect – ipfs/kubo#5556), and from the perspective of GUI we only care about the root CID which was pinned recursively.

If you add type: recursive filter to pin ls, the check is instant (0.007s instead if >10s):

$ time curl -X POST 'http://127.0.0.1:5001/api/v0/pin/ls?paths=bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi&stream=true&arg=bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi&type=recursive'
{"Message":"path 'bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi' is not pinned","Code":0,"Type":"error"}
curl -X POST   0.00s user 0.00s system 90% cpu 0.007 total

@rafaelramalho19 Mind opening a PR with a fix for context menus (both breadcrumbs and ones in file list), so we can include it in v2.11?

@lidel lidel added this to the v2.11 milestone Sep 10, 2020
@lidel
Copy link
Member Author

lidel commented Sep 10, 2020

Fixed by #1619

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/days Estimated to take multiple days, but less than a week exp/intermediate Prior experience is likely helpful kind/bug A bug in existing code (including security flaws) kind/maintenance Work required to avoid breaking changes or harm to project's status quo need/analysis Needs further analysis before proceeding need/maintainers-input Needs input from the current maintainer(s) P2 Medium: Good to have, but can wait until someone steps up
Projects
None yet
Development

No branches or pull requests

1 participant