Excessive CPU usage

[Couby]

Hi,

I'm running a Wekan dedicated CT on a proxmox server.
With 512MiB RAM and 1 CPU as ressources for the CT.

I notice the CT CPU load is nearly 100% permanently.
For the server it represents 50% of its load (56% -> 6% when shutting down the Wekan CT).

It works but I don't think it is a normal load for the server, and it seems to slow down my other services.

Is it related to my installation or can it be confirmed by some other wekan admins ?

Regards.

[bevinhex]

Got the same issue here, First got #245 issue, solved by following the solution in the thread, But after solving the error, CPU still takes up 100%

[robert-scheck]

Same issue here, the excessive CPU usage seems to start when using Wekan the first time via web interface (nearly no CPU usage between starting and first usage).

Hardware resources do not seem to make a considerable difference when looking @Couby's post, affected environment here has 2x 2.8 GHz vCPU and 4 GB RAM, while Wekan simply consumes one vCPU with 100% CPU usage and 45.3% of memory (according to top and ps aux).

[xet7]

@maulal

Do you have idea how to fix this?

[maulal]

Hi, it's certainly a matter of indexes, i need to know the version of wekan and mongodb your working on?

Don't hesitate to check if you have the indexes related to #837 in your database.

[maulal]

@bevinhex and @robert-scheck ? any news ?

[bevinhex]

I also saw some logs about some node modules is installing something that is irrelivant from 'baidu', so could not trust it much (maybe I'm being parenoid)
end up using https://taskboard.matthewross.me/ instead

[robert-scheck]

@maulal, the affected environment here has Wekan 0.11.1-rc1 and MongoDB 2.6.12

[maulal]

@robert-scheck ok, in this version, some indexes were missing. Take the last released version (v0.16), it should be good.

[alexkornitzer]

Our system is running at 100% CPU all the time on CentOS 7: Wekan 0.16 (Release Build) and MongoDB 2.6.12. Are there any logs we can provide to help diagnose the issue?

[maulal]

@alexkornitzer
At least let's say the cpu consumption for each processes (during 5min).
In mongodb by default, slow queries (>100ms) are logged, you can check that point.

[alexkornitzer]

I believe that the issue is due to Wekan or its node libraries and not mongo. If I restart the wekan service CPU is under 1%, but as soon as a client connects to the service the CPU jumps to 100% where it stays even once the client has disconnected. Our number of entries within the DB is small and I have even tried with a clean database.

We are running the service behind an nginx SSL reverse proxy so I am not sure if this could be the issue. I also get a websockets error reported in firefox, could it be that on a failed websocket connection it tries to reconnect endlessly? Similar to this? node-apn/node-apn#180

Running tcpdump I can see that even on a disconnect from the client the server continues to send a packet every 10 seconds to the client.

[alexkornitzer]

Found the source of my issue, now using meteor's node.

/opt/meteor/meteor node ...

Source: https://forums.meteor.com/t/meteor-bundle-on-production-100-cpu/30837/40

[denysvitali]

We noticed the same thing on our server. 100% of CPU.

Seriously... WTF?!!!!

[alexkornitzer]

Yeah it's a meteor thing, not sure why their framework is not node version agnostic.

…

On 14 Apr 2017, at 13:24, Denys Vitali ***@***.***> wrote: We noticed the same thing on our server. 100% of CPU. Seriously... WTF?!!!! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[xavierpriour]

Hi, same problem here with the official docker recommended in the doc (wekanteam/wekan:meteor-1.4).
It uses node v6.10.2, whereas meteor 1.4 supports node 4.8 only.

I'll be happy to try and submit a PR, if you can just point me to the part of the repo where the docker builds are defined.

[xet7]

Node version has been changed yesterday:
https://github.com/wekan/wekan/wiki/Install-and-Update#install-manually-from-source
https://github.com/wekan/wekan/blob/devel/Dockerfile
https://github.com/wekan/wekan/blob/master/Dockerfile

A moment ago I merged Dockerfile fixes:
#1016

Also see other changes:
https://github.com/wekan/wekan/blob/devel/CHANGELOG.md

If you notice there's changes needed elsewhere, please change it or add comment to this issue.

[xet7]

There will probably be more Dockerfile fixes in near future to get it working properly.

Installing from source works correctly though.

[stephenmoloney]

I didn't see this issue. I wasn't aware that meteor only supports node v4. Hopefully, the latest version change fixes it. The latest build includes node v4.8.1

[xavierpriour]

I confirm issue is solved, afaik you can close the issue, as well as https://github.com/wekan/wekan-mongodb/issues/2

Thanks again @xet7

[Madko]

Same here, how to get the fix ?

[xet7]

@Madko

For example Github Wiki platforms page, docker-compose wekan-mongodb. What did you use to install wekan?

[alexkornitzer]

@Madko

If you are using the release bundle or source, then I find that the easiest way is to install the matching meteor version and run the meteor node: /opt/meteor/meteor node <PATH_TO_WEKAN>/build/bundle/main.js

[Madko]

@xet7 it's a manual install on CentOS 7.

@alexkornitzer will try to find out more about meteor

[Madko]

Thanks, it works with meteor (was a bit "fun" to install with their curl | sh)

[alexkornitzer]

Yeah downgrading node is not a good solution so those are the steps I took on our Centos machine. Has been running great since.

…

On 8 Jun 2017, at 13:43, Madko ***@***.***> wrote: Thanks, it works with meteor (was a bit "fun" to install with their curl | sh) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[mfshiu]

Use this package in Wekan is VERY important:

mixmax:smart-disconnect

Meteor connection cost is very expensive, if your MongoDB contains large data, your active users are over several hundreds, you must use the package to auto disconnect user when they leave their focus on Wekan over 1 minute. I read the article by Eliezer Steinbock, then make my Wekan CPU ultra down.

[xet7]

mixmax:smart-connect was removed from Wekan at Wekan v3.03.

From that fix details:

"Remove mixmax:smart-disconnect, previously it did disconnect Wekan when browser tab was not active, but because now users are working on multiple boards at different browser tabs and switching all time time, there was constant loading, so now after removing all tabs keeps active. This can increase server CPU usage. Thanks to xet7."

Additionally, removing mixmax:smart-disconnects fixes data loss. Data loss happened, when someone changes to other tab, first tab of Wekan did not have time to sync those changes to server, and then when browser was closed, changed data was lost forever.