NodeBB Hangs, importing 1M+, no logs, no crash, process still running but not responding, MEM 200MB RAM , CPU 100% #61

akhoury · 2014-10-07T18:53:24Z

It could be a data problem, the 1M+ number may not be relevant, but I was not able to reproduce with smaller DBs using the same -smf exporter.

startup command NODE_ENV=development node app
start importing 62k users, 83 categories, 73k topics, 1M+ posts
during the import there are not no obvious memory leaks, around 200MB consistent
CPU 100% most of the time, but that's normal
A little while after the posts start importing, NodeBB stops responding, doesn't crash, doesn't log anything out, that's all i see
after that happens
curl localhost:4567 without a timeout does not return
node process is still running 100% CPU, 120MB memory, and still listening on :4567

I do not know why there is a node bcrypt process running, post creation does not call it or get near it.

The text was updated successfully, but these errors were encountered:

akhoury · 2014-10-07T18:54:32Z

TODO:

could it be process.on("uncaughtException") ?
Need to add more logging
Need to isolate the issue with minimal test case, currently reproduction takes over 50min each time :/
maybe re-implement timemachine to isolate the data

barisusakli · 2014-10-07T18:57:23Z

The bcrypt proc is fine it gets created on app start and used whenever someone tries to login or register.

Does it still hang when you try to only import the first 700k posts? or the last 700k posts?

barisusakli · 2014-10-07T18:58:18Z

Also which plugins are installed and active? db search might be a problem.

akhoury · 2014-10-07T19:30:16Z

Does it still hang when you try to only import the first 700k posts? or the last 700k posts?

I havent tried that yet.. I am trying to find which "posts batch" is causing the issue and Im gonna skip it.

Also which plugins are installed and active? db search might be a problem.

nothing, well just markdown is activated. DB search is not active.

akhoury · 2014-10-07T21:23:14Z

what's interesting that it always happens at the same exact post _pid

Added more details of the post before getting created

barisusakli · 2014-10-07T21:38:32Z

What is the data for that post?

akhoury · 2014-10-07T22:27:25Z

actually it's a batch of 10 posts, so it could any one these guys:

{
    "511038": {
        "_pid": 511038,
        "_tid": 48737,
        "_timestamp": 1187119419000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "____88888888:::::8::::::::::::::::88888888888888888888<br />________888_8:::888888:::::::::::::::::::::::::88888888888___888 ___________88:::::88888888::::::m::::::::::::::88888888888____8 _________888888888888888888:M::::::::::::::::8888888888888 ________88888888888888888888:::::::::::::::::M88888888888888 ________8888888888888888888888:::::::::::::M8888888888888888 _________8888888888888888888888::::::::::M888888888888888888 ________8888888888888888::88888:::::::::M88888888888888888888 ______88888888888888888:::88888::::::::M888888888888888___8888 _____88888888888888888:::88888::::::M:::::;o*M*o;888888888____88 ____88888888888888888:::8888:::::::M::::::::::::::::::88888888____8 ___88888888888888888::::88:::::::::M:;:::::::::::::::::::888888888 __8888888888888888888:::8:::::::::M::::aAa::::::::::::M8888888888_______ 8 __88___8888888888::88::::8::::::::M::::::::::::::::::::888888888888888_8 888 _88__88888888888:::8::::::::::::::M:::::::::::;::::::::88:88888888888888 888 _8__8888888888888::::::::::::::::M::::@@@@::::::::8w8888888888888888 __88888888888:888:::::::::::::::M:::::::@a@:::::::M8i888888888888888 _8888888888::::88:::::::::::::::M888:::::::::::::::::M88z888888888888888 88 8888888888:::::8::::::::::::::::M88888::::::::::::MM88888888888888888888 8 888888888:::::8::::::::::::::::M8888888MAmAMVMM88*88888888___88888888 888888_M::::::::::::::::::::::M888888888:::::::::MM8888888888888___88888 88 8888___M:::::::::::::::::::::M88888888888:::::::MM88888888888888____8888 8 _888___M::::::::::::::::::::M8888888888888M:::::mM888888888888____888 __888__M:::::::::::::::::::M8888:8888888888888:::m::Mm8888_8888___888 ___88__M:::::::::::::::::::8888:8888888888888888:::::::::Mm8___8888___88 8 ___88__M::::::::::::::::8888M::88888::888888888888::::::::::Mm8888____88&#38;#160;<br />___8___MM:::::::::::::8888M::::8888:::::888888888888::::::::::::Mm8_____ 4 _______8M::::::::::::8888M:::::::888::::::::88:::8888888::::::::::::::Mm_____2 ______88MM:::::::::8888M::::::::::88:::::::::8:::::888888:::::::::M::::: :M _____8888M:::::::::888MM::::::::::::8::::::::::::M::::8888:::::::::::M:: ::M ____88888M:::::::::88:M:::::::::::::::8:::::::::::::M:::8888:::::::::::: M::M ___88_888MM::::::888:M:::::::::::::::::::::::::::::::M:8888::::::::::::: :M: ___8_88888M::::::88::M:::::::::::::::::::::::::::::::::MM:88:::::::::::: :::::M _____88888M::::::88::M:::::::::::::::::*88*::::::::::::M:88::::::::::::: :::::::M ____888888M::::::88::M:::::::::::::::88@@88::::::::::M::88:::::::::::::: :::::::M ____888888MM::::88::MM:::::::::::::88@@88:::::::::::M::::8:::::::::::::: :::::::*8 ____88888__M::::::8::MM:::::::::::::::*88*:::::::::::::M:::::::::::::::: ::::::::::::88@<br />____8888___MM:::::::::MM::::::::::::::::::::::::::::::MM:::::::::::::::: :::::::::::88@@<br />_____888____M:::::::::::MM:::::::::::::::::::::::::::MM::M:::::::::::::: ::::::::::::888/ _____888____MM:::::::::::MMM:::::::::::::::::::::MM::::MM::::::::::::::: ::::::::::MM ______88_____M:::::::::::::MMMM::::::::::::MMMM::::::::MM::::::::::::::: :::::MMM _______88____MM:::::::::::::::MMMMMMMMMMM:::::::::::::MMM:::::::MMMMMM ________88____MM:::::::::::::::::::MMMMM::::::::::::::::::::MMMMMMMMMMII<br />_________88___8MM::::::::::::::::::::::::::::::::::::::::::::::::::MMMMM MMM __________8___88MM:::::::::::::::::::::::::::::::::M:::M:::::::::::MMM ______________888MM:::::::::::::::::::::::::::MM::::::::MM::::::MM _____________88888MM::::::::::::::::::::::MMM:::::::::mM:::::MM<br /><br />damn. it fucks up on this forum.......oh well, you get the idea.",
        "_uid": 7915,
        "_approved": 1
    },
    "511042": {
        "_pid": 511042,
        "_tid": 48737,
        "_timestamp": 1187120036000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "<br /><br />[img]http://img161.imageshack.us/img161/2999/k1un4.gif[/img]<br /><br />Kanna says:<br />This forum isn&#039;t properly formatted for ASCII art.<br /><br />[img width=446 height=480]http://img65.imageshack.us/img65/2581/92315342ie3.jpg[/img]",
        "_uid": 21621,
        "_approved": 1
    },
    "511052": {
        "_pid": 511052,
        "_tid": 48737,
        "_timestamp": 1187120648000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "NO U!<br /><br />...screwed up pasting :P",
        "_uid": 14,
        "_approved": 1
    },
    "511098": {
        "_pid": 511098,
        "_tid": 48737,
        "_timestamp": 1187125217000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "[me=Thrawn]wakes up.[/me]<br /><br /><br />[quote author=SpliTT link=topic=48737.msg510942#msg510942 date=1187110309]<br />this is really messed up for Jurassic park <br /><br />http://www.bloody-disgusting.com/news/9638<br />[/quote]<br /><br />O.o Yeah. The best [i]Jurassic Park[/i], by far, is the [url=http://www.michaelcrichton.net/books-jurassicpark.html]book[/url].",
        "_uid": 13151,
        "_approved": 1
    },
    "511105": {
        "_pid": 511105,
        "_tid": 48737,
        "_timestamp": 1187125576000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "Whoa what huh.",
        "_uid": 1733,
        "_approved": 1
    },
    "511115": {
        "_pid": 511115,
        "_tid": 48737,
        "_timestamp": 1187126328000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "Credibility and exchange.",
        "_uid": 23843,
        "_approved": 1
    },
    "511123": {
        "_pid": 511123,
        "_tid": 48737,
        "_timestamp": 1187127214000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "[img]http://img161.imageshack.us/img161/750/lorider1un4.gif[/img]<br />[img]http://img65.imageshack.us/img65/2180/4625772ay9.jpg[/img]<br />[img]http://www.tshirthell.com/shirts/products/a729/a729_thumb.jpg[/img]",
        "_uid": 21621,
        "_approved": 1
    },
    "511151": {
        "_pid": 511151,
        "_tid": 48737,
        "_timestamp": 1187130092000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "Where&#039;s the Mr.T-shaped beef?!",
        "_uid": 23843,
        "_approved": 1
    },
    "511206": {
        "_pid": 511206,
        "_tid": 48737,
        "_timestamp": 1187132459000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "saw simpsons movie today.... shit was alright &quot;sequel&quot;!!",
        "_uid": 12238,
        "_approved": 1
    },
    "511211": {
        "_pid": 511211,
        "_tid": 48737,
        "_timestamp": 1187132632000,
        "_subject": "Re: fREE XBOX AND PS3??? CLICK HERE",
        "_content": "[quote author=UrzaTheOtaku link=topic=48737.msg511123#msg511123 date=1187127214]<br />*shit*<br />[/quote]<br /><br />Stop posting shit.<br /><br />it&#039;s not funny.",
        "_uid": 26079,
        "_approved": 1
    }
}

barisusakli · 2014-10-08T00:05:56Z

Haha well that was easy I tried posting the first one on my dev env and it locked up the forum.

barisusakli · 2014-10-08T00:13:06Z

If I disable markdown I was able to post.

barisusakli · 2014-10-08T00:31:42Z

You are using Posts.create to import right there is no reason for that function to call postTools.parse, i will refactor that so the importer can at least work while we take a look at why md plugin breaks with that certain input.

barisusakli · 2014-10-08T00:37:42Z

NodeBB/NodeBB@712bd39 should be better and much faster to import as well. Let me know.

barisusakli · 2014-10-08T01:54:19Z

markedjs/marked#497

akhoury · 2014-10-08T02:12:16Z

thanks.

akhoury · 2014-10-08T02:25:58Z

closing as this is upstream. Thanks @barisusakli

akhoury added the bug label Oct 7, 2014

akhoury mentioned this issue Oct 7, 2014

Segmentation fault: 11, running out memory :/ #51

Closed

akhoury closed this as completed Oct 8, 2014

albertshaw mentioned this issue Oct 28, 2014

要不要考虑换一个markdown解析库 cnodejs/nodeclub#440

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NodeBB Hangs, importing 1M+, no logs, no crash, process still running but not responding, MEM 200MB RAM , CPU 100% #61

NodeBB Hangs, importing 1M+, no logs, no crash, process still running but not responding, MEM 200MB RAM , CPU 100% #61

akhoury commented Oct 7, 2014

akhoury commented Oct 7, 2014

barisusakli commented Oct 7, 2014

barisusakli commented Oct 7, 2014

akhoury commented Oct 7, 2014

akhoury commented Oct 7, 2014

barisusakli commented Oct 7, 2014

akhoury commented Oct 7, 2014

barisusakli commented Oct 8, 2014

barisusakli commented Oct 8, 2014

barisusakli commented Oct 8, 2014

barisusakli commented Oct 8, 2014

barisusakli commented Oct 8, 2014

akhoury commented Oct 8, 2014

akhoury commented Oct 8, 2014

NodeBB Hangs, importing 1M+, no logs, no crash, process still running but not responding, MEM 200MB RAM , CPU 100% #61

NodeBB Hangs, importing 1M+, no logs, no crash, process still running but not responding, MEM 200MB RAM , CPU 100% #61

Comments

akhoury commented Oct 7, 2014

akhoury commented Oct 7, 2014

barisusakli commented Oct 7, 2014

barisusakli commented Oct 7, 2014

akhoury commented Oct 7, 2014

akhoury commented Oct 7, 2014

barisusakli commented Oct 7, 2014

akhoury commented Oct 7, 2014

barisusakli commented Oct 8, 2014

barisusakli commented Oct 8, 2014

barisusakli commented Oct 8, 2014

barisusakli commented Oct 8, 2014

barisusakli commented Oct 8, 2014

akhoury commented Oct 8, 2014

akhoury commented Oct 8, 2014