Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 7047 optional jemalloc #7424

Merged
merged 29 commits into from
Nov 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
5affb81
Implementing Issue #7047 - Optionally load jemalloc
amsmota May 14, 2024
24bf03e
Implementing Issue #7047 - Optionally load jemalloc
amsmota May 14, 2024
cf81e23
Implementing Issue #7047 - Optionally load jemalloc: fixes after review
amsmota May 18, 2024
0db033a
Implementing Issue #7047 - Optionally load jemalloc: added entry to C…
amsmota Jun 14, 2024
a8803b7
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Aug 4, 2024
acdd021
Merge branch 'issue-7047-optional-jemalloc' of https://github.com/Cit…
amsmota Aug 4, 2024
8b28c00
Changes after review
amsmota Aug 13, 2024
2ba735c
Added env var check in unix script
amsmota Aug 17, 2024
0bf5046
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 10, 2024
bd1f176
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 11, 2024
27cdddc
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 15, 2024
83e6a1c
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 16, 2024
653d1a8
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 18, 2024
6a38c53
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 20, 2024
e29a361
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 20, 2024
03e0b07
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 21, 2024
9c31091
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 22, 2024
a5328c1
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 23, 2024
a86d523
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 23, 2024
b7206eb
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 24, 2024
0af4668
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 24, 2024
1cd9e44
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Sep 24, 2024
b8f46ce
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Oct 7, 2024
192a3a3
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Oct 30, 2024
ffbccbb
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Nov 3, 2024
0bdfb0f
Improved code and script, build and tested
amsmota Nov 3, 2024
7abe153
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Nov 6, 2024
3260cf9
Improved code and script, build and tested
amsmota Nov 6, 2024
c9cccc5
Merge branch 'hyperledger:main' into issue-7047-optional-jemalloc
amsmota Nov 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ This release version has been deprecated release due to CI bug
- Remove long-deprecated `perm*whitelist*` methods [#7401](https://github.com/hyperledger/besu/pull/7401)

### Additions and Improvements
- Allow optional loading of `jemalloc` (if installed) by setting the environment variable `BESU_USING_JEMALLOC` to true/false. It that env is not set at all it will behave as if it is set to `true`
amsmota marked this conversation as resolved.
Show resolved Hide resolved
- Expose set finalized/safe block in plugin api BlockchainService. These method can be used by plugins to set finalized/safe block for a PoA network (such as QBFT, IBFT and Clique).[#7382](https://github.com/hyperledger/besu/pull/7382)
- In process RPC service [#7395](https://github.com/hyperledger/besu/pull/7395)
- Added support for tracing private transactions using `priv_traceTransaction` API. [#6161](https://github.com/hyperledger/besu/pull/6161)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -433,14 +433,18 @@ public String build() {
private void detectJemalloc(final List<String> lines) {
Optional.ofNullable(Objects.isNull(environment) ? null : environment.get("BESU_USING_JEMALLOC"))
amsmota marked this conversation as resolved.
Show resolved Hide resolved
.ifPresentOrElse(
t -> {
jemallocEnabled -> {
try {
final String version = PlatformDetector.getJemalloc();
lines.add("jemalloc: " + version);
if (Boolean.parseBoolean(jemallocEnabled)) {
final String version = PlatformDetector.getJemalloc();
lines.add("jemalloc: " + version);
} else {
logger.warn(
"besu_using_jemalloc is present but is not set to true, jemalloc library not loaded");
}
} catch (final Throwable throwable) {
logger.warn(
"BESU_USING_JEMALLOC is present but we failed to load jemalloc library to get the version",
throwable);
"besu_using_jemalloc is present but we failed to load jemalloc library to get the version");
}
},
() -> {
Expand Down
24 changes: 12 additions & 12 deletions besu/src/main/scripts/unixStartScript.txt
Original file line number Diff line number Diff line change
Expand Up @@ -182,19 +182,19 @@ APP_ARGS=`save "\$@"`
# Collect all arguments for the java command, following the shell quoting and substitution rules
eval set -- \$DEFAULT_JVM_OPTS \$JAVA_OPTS \$${optsEnvironmentVar} <% if ( appNameSystemProperty ) { %>"\"-D${appNameSystemProperty}=\$APP_BASE_NAME\"" <% } %>-classpath "\"\$CLASSPATH\"" <% if ( mainClassName.startsWith('--module ') ) { %>--module-path "\"\$MODULE_PATH\"" <% } %>${mainClassName} "\$APP_ARGS"

unset BESU_USING_JEMALLOC
if [ "\$darwin" = "false" -a "\$msys" = "false" ]; then
# check if jemalloc is available
TEST_JEMALLOC=\$(LD_PRELOAD=libjemalloc.so sh -c true 2>&1)

# if jemalloc is available the output is empty, otherwise the output has an error line
if [ -z "\$TEST_JEMALLOC" ]; then
export LD_PRELOAD=libjemalloc.so
export BESU_USING_JEMALLOC=true
else
# jemalloc not available, as fallback limit malloc to 2 arenas
export MALLOC_ARENA_MAX=2
fi
if [ "\$BESU_USING_JEMALLOC" != "FALSE" -a "\$BESU_USING_JEMALLOC" != "false" ]; then
amsmota marked this conversation as resolved.
Show resolved Hide resolved
# check if jemalloc is available
TEST_JEMALLOC=\$(LD_PRELOAD=libjemalloc.so sh -c true 2>&1)

# if jemalloc is available the output is empty, otherwise the output has an error line
if [ -z "\$TEST_JEMALLOC" ]; then
export LD_PRELOAD=libjemalloc.so
else
# jemalloc not available, as fallback limit malloc to 2 arenas
export MALLOC_ARENA_MAX=2
fi
fi
fi

exec "\$JAVACMD" "\$@"
12 changes: 8 additions & 4 deletions besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
import static org.mockito.ArgumentMatchers.contains;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.ArgumentMatchers.isNotNull;
import static org.mockito.Mockito.argThat;
import static org.mockito.Mockito.atLeast;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
Expand Down Expand Up @@ -2412,13 +2413,16 @@ public void nativeLibrariesAreEnabledByDefault() {
@Test
public void logsWarningWhenFailToLoadJemalloc() {
assumeTrue(PlatformDetector.getOSType().equals("linux"));
setEnvironmentVariable("BESU_USING_JEMALLOC", "true");
setEnvironmentVariable("BESU_USING_JEMALLOC", "false");
parseCommand();
verify(mockLogger)
.warn(
eq(
"BESU_USING_JEMALLOC is present but we failed to load jemalloc library to get the version"),
any(Throwable.class));
argThat(
arg ->
arg.equals(
"besu_using_jemalloc is present but is not set to true, jemalloc library not loaded")
|| arg.equals(
"besu_using_jemalloc is present but we failed to load jemalloc library to get the version")));
}

@Test
Expand Down
Loading