In building develop branch, make -j16 is too demanding for an 8-core i9 CPU with a 16GB memory

[b1aafulei]

eos/scripts/.build_vars

Line 43 in 44e216b

export CPU_CORES=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || sysctl -n hw.ncpu)

eos/scripts/helpers/general.sh

Line 75 in 44e216b

export JOBS=${JOBS:-$(( MEM_GIG > CPU_CORES ? CPU_CORES : MEM_GIG ))}

The current JOBS as in the make -j${JOBS} in the build script is determined by the code listed above.

In short, JOBS is the lower of

1. the number of logical CPU cores
2. the size of main memory in GB

However, in building the current develop branch, this configuration is too demanding, at least for my case. My Macbook Pro has an 8-core i9-9880H CPU with a 16GB memory, so it counts as having 16 logical cores, and thus JOBS=16.

Now the eosio_build.sh consistently freezes my Mac (for four times) when it arrives at

[ 77%] Building CXX object libraries/chain/CMakeFiles/eosio_chain.dir/webassembly/privileged.cpp.o

I am attaching a screenshot of the Activity Monitor at the freezing moment.

Workaround

The workaround is to terminate the build script before it comes to the compilation, go to the build folder, and manually run make -j8.

Suggestion

Consider changing

eos/scripts/.build_vars

Line 43 in 44e216b

export CPU_CORES=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || sysctl -n hw.ncpu)

into

 export CPU_CORES=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || sysctl -n hw.physicalcpu) 

which counts the number of physical CPU cores.

Or maybe other ways to avoid the freezing in the compilation.

[dixia]

I second this. The memory consumption is too much that ends up to freeze the compilation

Ideally if the script could limits the memory usage of each job, then we would be able to fully use the CPU power (i.e. ncpu) without letting clang eat up all ram.

[TimothyBanks]

Changes have been merged from the PRs referenced in this issue.

[b1aafulei]

Changes have been merged from the PRs referenced in this issue.

Thanks for the fix. The developer branch is fine now, but it seems the release/2.1.x branch still experience the problem.