Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipfs add with onlyhash, memery high usage #6744

Closed
hcg1314 opened this issue Nov 4, 2019 · 5 comments · Fixed by #6747
Closed

ipfs add with onlyhash, memery high usage #6744

hcg1314 opened this issue Nov 4, 2019 · 5 comments · Fixed by #6747
Assignees
Labels
kind/bug A bug in existing code (including security flaws)

Comments

@hcg1314
Copy link
Contributor

hcg1314 commented Nov 4, 2019

Version information:

go-ipfs version: 0.4.22-
Repo version: 7
System version: amd64/linux
Golang version: go1.12.7

Description:

The VM has 8GB of RAM.
My application will generate a lots of calls ipfs add -n with a speed 40 per seonds.
Firstly, the ipfs daemon just cost little memery. however, one hours later, out of memery.
If stop my application, the ipfs will return to normal after a few hours.

i want to know, is there a way to reduce recovery time? thanks!

here is the data collected by bin/collect-profiles.sh

@hcg1314 hcg1314 added the kind/bug A bug in existing code (including security flaws) label Nov 4, 2019
@hcg1314
Copy link
Contributor Author

hcg1314 commented Nov 4, 2019

when i add defer nilnode.Peerstore.Close() in core/coreapi/unixfs.go,the MEM% grow up slower

@Stebalien
Copy link
Member

Stebalien commented Nov 4, 2019

@hcg1314 what if you add defer nilnode.Close()? #6744

@hcg1314
Copy link
Contributor Author

hcg1314 commented Nov 5, 2019

@hcg1314 what if you add defer nilnode.Close()? #6744

If add defer nilnode.Close(), seems like can not release Peerstore.
I have tested it in this way:

I add log.Info("I'm stopped!") in vendor/github.com/libp2p/go-libp2p-peerstore/pstoremem/addr_book.go, function: func (mab *memoryAddrBook) background() {}
when the ipfs add -n xxx is over, i'm stopped! appears.
But, if i add defer nilnode.Close(), i'm stopped! does not appear.

@Stebalien
Copy link
Member

What if you add both? I'm trying to figure out "the MEM% grow up slower".

There's a bug in go-ipfs 0.4.22 where we don't close the peerstore when the node is closed. This has been fixed in master.

@hcg1314
Copy link
Contributor Author

hcg1314 commented Nov 11, 2019

'master' branch has fix this issue

@hcg1314 hcg1314 closed this as completed Nov 11, 2019
pontiyaraja pushed a commit to pontiyaraja/go-ipfs that referenced this issue Dec 27, 2019
ralendor pushed a commit to ralendor/go-ipfs that referenced this issue Jun 6, 2020
ralendor pushed a commit to ralendor/go-ipfs that referenced this issue Jun 6, 2020
ralendor pushed a commit to ralendor/go-ipfs that referenced this issue Jun 6, 2020
ralendor pushed a commit to ralendor/go-ipfs that referenced this issue Jun 8, 2020
ralendor pushed a commit to ralendor/go-ipfs that referenced this issue Jun 8, 2020
ralendor pushed a commit to ralendor/go-ipfs that referenced this issue Jun 8, 2020
ralendor pushed a commit to ralendor/go-ipfs that referenced this issue Jun 8, 2020
ralendor pushed a commit to ralendor/go-ipfs that referenced this issue Jun 8, 2020
ralendor pushed a commit to ralendor/go-ipfs that referenced this issue Jun 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A bug in existing code (including security flaws)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants