From 6b3ce8bba0a00356169e39af7e80225e96c8f822 Mon Sep 17 00:00:00 2001 From: pm47 Date: Wed, 16 Dec 2020 21:32:59 +0100 Subject: [PATCH 1/3] add cluster doc, and packaging improvements We now use a regular release zip, and use it in the awseb bundle. The default logback configuration now logs to a file, like `eclair-node`, and we use a dedicated logback configuration for AWS beanstalk. By default the front reads the node secret key from the node seed in the standard `.eclair` directory. --- docs/Cluster.md | 177 +++++++++ eclair-front/modules/assembly.xml | 36 +- eclair-front/modules/awseb.xml | 46 +++ eclair-front/modules/awseb/logback_eb.xml | 45 +++ eclair-front/modules/awseb/run.sh | 10 +- eclair-front/pom.xml | 5 +- .../src/main/resources/application.conf | 9 +- .../src/main/resources/eclair-front.sh | 360 ++++++++++++++++++ eclair-front/src/main/resources/logback.xml | 43 ++- .../src/main/scala/fr/acinq/eclair/Boot.scala | 1 + .../scala/fr/acinq/eclair/FrontSetup.scala | 10 +- eclair-node-gui/pom.xml | 5 - eclair-node/pom.xml | 5 - 13 files changed, 689 insertions(+), 63 deletions(-) create mode 100644 docs/Cluster.md create mode 100644 eclair-front/modules/awseb.xml create mode 100644 eclair-front/modules/awseb/logback_eb.xml create mode 100644 eclair-front/src/main/resources/eclair-front.sh diff --git a/docs/Cluster.md b/docs/Cluster.md new file mode 100644 index 0000000000..f27a15b5d1 --- /dev/null +++ b/docs/Cluster.md @@ -0,0 +1,177 @@ +## How to _clusterize_ your Eclair node + +Eclair allows you to scale up one _logical_ lightning node across multiple servers. + +Front servers take care of routing table related gossip and syncing requests from peers, which is cpu/bandwidth intensive. The backend server can focus on core channel management. So, BOLT 1&7 messages are handled in the frontend, while BOLT 2 messages go through and are processed in the backend. + +Front servers are stateless, they can be stopped/killed at will. The node will remain operational and reachable as long as there is at least one `frontend` available. + +``` + +---+ +-----------+ + | | | +-------+ | + | |-----|-| | | + P -----| L |-----|-| FRONT |-|---, + U | O |-----|-| | | \ + B ---| A | | +-------+ | \ + L | D | | | \ + I | | | +-------+ | \ +------+ + C -----| B |-----|-| | | `| | + | A |-----|-| FRONT |-|---------| BACK |<-- channels management + N | L |-----|-| | | ,| | relay logic + E ----| A | | +-------+ | / +------+ + T | N | | | / + W | C | | +-------+ | / + O | E |-----|-| | | / + R -----| R |-----|-| FRONT |-|---' + K | |-----|-| |<------- connection management + | | | +-------+ | routing table sync + +---+ +-----------+ +``` + +The goal is to offload your node from connection and routing table management: +- incoming connections +- outgoing connections +- gossip queries + pings +- incoming gossip aggregation +- outgoing gossip dispatch (rebroadcast) + +### Prerequisite + +You already have a node up and running in a standalone setup (with Bitcoin Core properly configured, etc.). + +You know what your `node id` is. + +In the following, what we previously called `eclair-node` will be called `backend`. It is to be launched, configured and backed-up exactly like in a standalone setup. + +### Minimal/Demo setup + +Use this if you want to experiment with the cluster mode on a single local server. + +Set the following values in `.eclair/eclair.conf`: +``` +akka.actor.provider = cluster +akka.extensions = ["akka.cluster.pubsub.DistributedPubSub"] + +// replace this with your node id +// if you don't know what your node id is, you should probably stop right here +eclair.front.pub = 03............... +``` + +Start the `backend`: +```shell +$ ./eclair-node.sh +``` + +Then run an instance of `frontend`: +```shell +$ ./eclair-front.sh -Dakka.remote.artery.canonical.port=25521 -Declair.server.port=9736 +``` + +NB: we override the ports, otherwise they would conflict since in this example everything runs on the same server. You can run multiple `frontend`s on the same server, just make sure to change the ports. + +### Production setup + +In production you should: +- run multiple `frontend`s +- run one app per server +- enable `tcp-tls` to encrypted between members of the cluster with your own generated certificate (see below) +- use a load balancer to hide all your `frontend` servers under the same ip address +- set firewall rules to disable lightning connections (port 9735) on your `backend` server, so all connections go through the `frontend` +- enable [monitoring](Monitoring.md) +- on AWS, use AWS Secrets Manager (see [AWS deployment](#aws-deployment)) + +#### Enable encrypted communication for the cluster + +We use a self-signed certificate, which offers a good compromise. More advanced options are available, see [akka doc](https://doc.akka.io/docs/akka/current/remoting-artery.html#remote-security). +> Have a single set of keys and a single certificate for all nodes and disable hostname checking +> - The single set of keys and the single certificate is distributed to all nodes. The certificate can be self-signed as it is distributed both as a certificate for authentication but also as the trusted certificate. +> - If the keys/certificate are lost, someone else can connect to your cluster. +> - Adding nodes to the cluster is simple as the key material can be deployed / distributed to the new node. + +Generate a self-signed certificate (set a strong password): +```shell +$ keytool -genkeypair -v \ + -keystore akka-cluster-tls.jks \ + -dname "O=ACME, C=FR" \ + -keypass:env \ + -storepass:env \ + -keyalg RSA \ + -keysize 4096 \ + -validity 9999 +``` + +Copy the resulting certificate to your `.eclair` directory: +```shell +$ cp akka-cluster-tls.jks ~/.eclair +``` + +Add this to your `eclair.conf`: +``` +AKKA_TLS_PASSWORD= +akka.remote.artery.transport = "tls-tcp" +``` + +### AWS Deployment + +For convenience, we provide a prebuilt AWS Beanstalk bundle for the `frontend` (choose a WebServer environment type, and Java platform). + +You can run it as-is for testing. + +#### TLS encryption + +If you intend to use it in production, you need to enable encryption with your own certificate: +1. Follow the procedure above to generate your `akka-tls.jks` + +2. We recommend forking the project and build your own bundle: +```shell +$ git clone git@github.com:ACINQ/eclair.git +$ vi eclair-core/src/main/reference.conf # set akka.remote.artery.transport = "tls-tcp" +$ cp akka-cluster-tls.jks eclair-front/modules/awseb/ # copy the file you generated +$ vi eclair-front/modules/awseb.xml # uncomment the relevant parts +$ mvn package -DskipTests +``` +Alternatively, you can also edit the existing bundle and manually add the `akka-cluster-tls.jks` file to the root of the zip archive. You will also need to set `akka.remote.artery.transport=tls-tcp` at runtime. + +#### Private key + +In production, we highly recommend using AWS Secrets manager to provide the node private key. This is done by setting `eclair.front.priv-key-provider=aws-sm`. Default secret name is "node-priv-key", but it is configurable with `eclair.front.aws-sm.aws-sm.priv-key-name` + +#### Configuration + +We recommend using Beanstalk environment variables for `AKKA_TLS_PASSWORD`, `BACKEND_IP`, and `NODE_PUB_KEY`. Other configuration keys should be set in the `AKKA_CONF` environment variable, semicolon separated. Example: +- `AKKA_CONF`: `eclair.enable-kamon=true; akka.remote.artery.transport=tls-tcp; eclair.front.priv-key-provider=aws-sm...` +- `AKKA_TLS_PASSWORD`: `xxxxxxxx` +- `BACKEND_IP`: `1.2.3.4` +- `NODE_PUB_KEY`: `03933884aaf1d6b108397e5efe5c86bcf2d8ca8d2f700eda99db9214fc2712b134` + +Port `25520` needs to be open, within the Beanstalk security group, and between the Beanstalk security group and your `backend` node. + +### Tor + +We recommend running your Tor hidden service on a separate server, and use `eclair.tor.targets` to redirect clearnet(*) connections to your `frontend` servers. + +(*) Clearnet for Tor, but BOLT 8 encrypted. + +Here is the resulting architecture: + +``` + +---+ +-----------+ + | | | +-------+ | + | |-----|-| | | + P -----| L |-----|-| FRONT |-|---, + U | O |-----|-| | | \ + B ---| A | | +-------+ | \ + L | D | | | \ + I | | | +-------+ | \ +------+ + C -------| B |-----|-| | | `| | + | A |-----|-| FRONT |-|---------| BACK |<-- channels management + N | L |-----|-| | | ,| | relay logic + E ----| A | | +-------+ | / +------+ + T | N | | | / + W +-------+ | C | | +-------+ | / + O | | | E |-----|-| | | / + R ---| Tor |------| R |-----|-| FRONT |-|---' + K | | | |-----|-| |<------- connection management + +-------+ | | | +-------+ | routing table sync + +---+ +-----------+ +``` diff --git a/eclair-front/modules/assembly.xml b/eclair-front/modules/assembly.xml index 36aec98d26..bfd1c74966 100644 --- a/eclair-front/modules/assembly.xml +++ b/eclair-front/modules/assembly.xml @@ -1,11 +1,10 @@ - ${git.commit.id.abbrev}-awseb_bundle + bin zip - false lib @@ -22,39 +21,14 @@ LICENSE* - - ${project.basedir}/modules/awseb - . - - Procfile - - unix - - - - - ${project.basedir}/modules/awseb - . + + ${project.basedir}/src/main/resources + bin - run.sh + eclair-front.sh 0755 unix - - ${project.build.directory} - . - - application.jar - - \ No newline at end of file diff --git a/eclair-front/modules/awseb.xml b/eclair-front/modules/awseb.xml new file mode 100644 index 0000000000..339acc9c9d --- /dev/null +++ b/eclair-front/modules/awseb.xml @@ -0,0 +1,46 @@ + + ${git.commit.id.abbrev}-awseb_bundle + + zip + + false + + + ${project.build.directory} + . + + eclair-front-*-bin.zip + + + + ${project.basedir}/modules/awseb + . + + Procfile + logback_eb.xml + + unix + + + + + ${project.basedir}/modules/awseb + . + + run.sh + + 0755 + unix + + + \ No newline at end of file diff --git a/eclair-front/modules/awseb/logback_eb.xml b/eclair-front/modules/awseb/logback_eb.xml new file mode 100644 index 0000000000..243bc1519a --- /dev/null +++ b/eclair-front/modules/awseb/logback_eb.xml @@ -0,0 +1,45 @@ + + + + + + + + + + + return formattedMessage.contains("connected to /10.") || + (formattedMessage.contains("connection closed") && !mdc.containsKey("nodeId")) || + (formattedMessage.contains("transport died") && !mdc.containsKey("nodeId")) || + (formattedMessage.contains("stopping") && !mdc.containsKey("nodeId")); + + + NEUTRAL + DENY + + System.out + false + + ${HOSTNAME} %d %-5level %logger{24}%X{category}%X{nodeId}%X{channelId}%X{paymentHash}%.-11X{parentPaymentId}%.-11X{paymentId} - %msg%ex{12}%n + + + + + + + + \ No newline at end of file diff --git a/eclair-front/modules/awseb/run.sh b/eclair-front/modules/awseb/run.sh index f6c34109ad..e3d7d8ecc1 100644 --- a/eclair-front/modules/awseb/run.sh +++ b/eclair-front/modules/awseb/run.sh @@ -1,3 +1,11 @@ export LOCAL_IP=$(curl -s 169.254.169.254/latest/meta-data/local-ipv4) export HOSTNAME=$(hostname) -exec java -javaagent:lib/kanela-agent-1.0.5.jar -jar application.jar \ No newline at end of file + +# make the eclair home directory +mkdir -p /home/webapp/.eclair + +# if provided, copy the certificate to the proper directory +[[ -e akka-cluster-tls.jks ]] && cp -v akka-cluster-tls.jks /home/webapp/.eclair/ + +unzip -o eclair-front-*-bin.zip +exec ./eclair-front-*/bin/eclair-front.sh -with-kanela -Dlogback.configurationFile=logback_eb.xml diff --git a/eclair-front/pom.xml b/eclair-front/pom.xml index e5aafa1f4a..bc9a825dd9 100644 --- a/eclair-front/pom.xml +++ b/eclair-front/pom.xml @@ -35,12 +35,9 @@ org.apache.maven.plugins maven-jar-plugin - application true - lib/ - fr.acinq.eclair.Boot true @@ -55,8 +52,10 @@ maven-assembly-plugin 3.2.0 + ${project.name}-${project.version}-${git.commit.id.abbrev} modules/assembly.xml + modules/awseb.xml diff --git a/eclair-front/src/main/resources/application.conf b/eclair-front/src/main/resources/application.conf index 53eee4a505..2af09a5920 100644 --- a/eclair-front/src/main/resources/application.conf +++ b/eclair-front/src/main/resources/application.conf @@ -3,12 +3,15 @@ eclair { front { // To be overriden with the same key as the backend, so that the front has the same nodeid - priv-key-provider = "aws-sm" // aws-sm (AWS Secrets Manager) or env (environment variable) + priv-key-provider = "seed" // aws-sm (AWS Secrets Manager) or env (environment variable) or seed (seed.dat) priv-key = ${?NODE_PRIV_KEY} // used if priv-key-provider = env aws-sm.priv-key-name = "node-priv-key" // used if priv-key-provider = aws-sm // As a security measure, we also require the pub key, which will be matched against the priv key to make sure the // front is really using the expected key pub = ${NODE_PUB_KEY} + + backend.ip = 127.0.0.1 + backend.ip = ${?BACKEND_IP} } } @@ -34,7 +37,7 @@ akka { cluster { shutdown-after-unsuccessful-join-seed-nodes = 10s # front won't start if back is offline roles = [frontend] - seed-nodes = ["akka://eclair-node@"${BACKEND_IP}":25520"] + seed-nodes = ["akka://eclair-node@"${eclair.front.backend.ip}":25520"] } coordinated-shutdown.terminate-actor-system = on coordinated-shutdown.exit-jvm = on @@ -45,7 +48,7 @@ akka { } kamon { - environment.host = ${HOSTNAME} + environment.host = ${?HOSTNAME} instrumentation.akka { filters { actors { diff --git a/eclair-front/src/main/resources/eclair-front.sh b/eclair-front/src/main/resources/eclair-front.sh new file mode 100644 index 0000000000..2c180f54d8 --- /dev/null +++ b/eclair-front/src/main/resources/eclair-front.sh @@ -0,0 +1,360 @@ +#!/usr/bin/env bash + +# Copyright (c) 2012, Joshua Suereth +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: +# +# Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. +# Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +### ------------------------------- ### +### Helper methods for BASH scripts ### +### ------------------------------- ### + +die() { + echo "$@" 1>&2 + exit 1 +} + +realpath () { +( + TARGET_FILE="$1" + CHECK_CYGWIN="$2" + + cd "$(dirname "$TARGET_FILE")" + TARGET_FILE=$(basename "$TARGET_FILE") + + COUNT=0 + while [ -L "$TARGET_FILE" -a $COUNT -lt 100 ] + do + TARGET_FILE=$(readlink "$TARGET_FILE") + cd "$(dirname "$TARGET_FILE")" + TARGET_FILE=$(basename "$TARGET_FILE") + COUNT=$(($COUNT + 1)) + done + + if [ "$TARGET_FILE" == "." -o "$TARGET_FILE" == ".." ]; then + cd "$TARGET_FILE" + TARGET_FILEPATH= + else + TARGET_FILEPATH=/$TARGET_FILE + fi + + # make sure we grab the actual windows path, instead of cygwin's path. + if [[ "x$CHECK_CYGWIN" == "x" ]]; then + echo "$(pwd -P)/$TARGET_FILE" + else + echo $(cygwinpath "$(pwd -P)/$TARGET_FILE") + fi +) +} + +# TODO - Do we need to detect msys? + +# Uses uname to detect if we're in the odd cygwin environment. +is_cygwin() { + local os=$(uname -s) + case "$os" in + CYGWIN*) return 0 ;; + *) return 1 ;; + esac +} + +# This can fix cygwin style /cygdrive paths so we get the +# windows style paths. +cygwinpath() { + local file="$1" + if is_cygwin; then + echo $(cygpath -w $file) + else + echo $file + fi +} + +# Make something URI friendly +make_url() { + url="$1" + local nospaces=${url// /%20} + if is_cygwin; then + echo "/${nospaces//\\//}" + else + echo "$nospaces" + fi +} + +# This crazy function reads in a vanilla "linux" classpath string (only : are separators, and all /), +# and returns a classpath with windows style paths, and ; separators. +fixCygwinClasspath() { + OLDIFS=$IFS + IFS=":" + read -a classpath_members <<< "$1" + declare -a fixed_members + IFS=$OLDIFS + for i in "${!classpath_members[@]}" + do + fixed_members[i]=$(realpath "${classpath_members[i]}" "fix") + done + IFS=";" + echo "${fixed_members[*]}" + IFS=$OLDIFS +} + +# Fix the classpath we use for cygwin. +fix_classpath() { + cp="$1" + if is_cygwin; then + echo "$(fixCygwinClasspath "$cp")" + else + echo "$cp" + fi +} +# Detect if we should use JAVA_HOME or just try PATH. +get_java_cmd() { + # High-priority override for Jlink images + if [[ -n "$bundled_jvm" ]]; then + echo "$bundled_jvm/bin/java" + elif [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then + echo "$JAVA_HOME/bin/java" + else + echo "java" + fi +} + +echoerr () { + echo 1>&2 "$@" +} +vlog () { + [[ $verbose || $debug ]] && echoerr "$@" +} +dlog () { + [[ $debug ]] && echoerr "$@" +} +execRunner () { + # print the arguments one to a line, quoting any containing spaces + [[ $verbose || $debug ]] && echo "# Executing command line:" && { + for arg; do + if printf "%s\n" "$arg" | grep -q ' '; then + printf "\"%s\"\n" "$arg" + else + printf "%s\n" "$arg" + fi + done + echo "" + } + + # we use "exec" here for our pids to be accurate. + exec "$@" +} +addJava () { + dlog "[addJava] arg = '$1'" + java_args+=( "$1" ) +} +addApp () { + dlog "[addApp] arg = '$1'" + app_commands+=( "$1" ) +} +addResidual () { + dlog "[residual] arg = '$1'" + residual_args+=( "$1" ) +} +addDebugger () { + addJava "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=$1" +} + +addKanelaAgent () { + addJava "-javaagent:$lib_dir/kanela-agent-1.0.5.jar" +} + +require_arg () { + local type="$1" + local opt="$2" + local arg="$3" + if [[ -z "$arg" ]] || [[ "${arg:0:1}" == "-" ]]; then + die "$opt requires <$type> argument" + fi +} +is_function_defined() { + declare -f "$1" > /dev/null +} + +# Attempt to detect if the script is running via a GUI or not +# TODO - Determine where/how we use this generically +detect_terminal_for_ui() { + [[ ! -t 0 ]] && [[ "${#residual_args}" == "0" ]] && { + echo "true" + } + # SPECIAL TEST FOR MAC + [[ "$(uname)" == "Darwin" ]] && [[ "$HOME" == "$PWD" ]] && [[ "${#residual_args}" == "0" ]] && { + echo "true" + } +} + +# Processes incoming arguments and places them in appropriate global variables. called by the run method. +process_args () { + local no_more_snp_opts=0 + while [[ $# -gt 0 ]]; do + case "$1" in + --) shift && no_more_snp_opts=1 && break ;; + -h|-help) usage; exit 1 ;; + -v|-verbose) verbose=1 && shift ;; + -d|-debug) debug=1 && shift ;; + -no-version-check) no_version_check=1 && shift ;; + -mem) echo "!! WARNING !! -mem option is ignored. Please use -J-Xmx and -J-Xms" && shift 2 ;; + -with-kanela) addKanelaAgent && shift ;; + -jvm-debug) require_arg port "$1" "$2" && addDebugger $2 && shift 2 ;; + -main) custom_mainclass="$2" && shift 2 ;; + -java-home) require_arg path "$1" "$2" && jre=`eval echo $2` && java_cmd="$jre/bin/java" && shift 2 ;; + -D*|-agentlib*|-XX*) addJava "$1" && shift ;; + -J*) addJava "${1:2}" && shift ;; + *) addResidual "$1" && shift ;; + esac + done + + if [[ no_more_snp_opts ]]; then + while [[ $# -gt 0 ]]; do + addResidual "$1" && shift + done + fi + + is_function_defined process_my_args && { + myargs=("${residual_args[@]}") + residual_args=() + process_my_args "${myargs[@]}" + } +} + +# Actually runs the script. +run() { + # TODO - check for sane environment + + # process the combined args, then reset "$@" to the residuals + process_args "$@" + set -- "${residual_args[@]}" + argumentCount=$# + + #check for jline terminal fixes on cygwin + if is_cygwin; then + stty -icanon min 1 -echo > /dev/null 2>&1 + addJava "-Djline.terminal=jline.UnixTerminal" + fi + + # check java version + if [[ ! $no_version_check ]]; then + java_version_check + fi + + if [ -n "$custom_mainclass" ]; then + mainclass=("$custom_mainclass") + else + mainclass=("$app_mainclass") + fi + + # Now we check to see if there are any java opts on the environment. These get listed first, with the script able to override them. + if [[ "$JAVA_OPTS" != "" ]]; then + java_opts="${JAVA_OPTS}" + fi + + execRunner "$java_cmd" \ + ${java_opts[@]} \ + "${java_args[@]}" \ + -cp "$(fix_classpath "$app_classpath")" \ + "$mainclass" \ + "${app_commands[@]}" \ + "${residual_args[@]}" + + local exit_code=$? + if is_cygwin; then + stty icanon echo > /dev/null 2>&1 + fi + exit $exit_code +} + +# Loads a configuration file full of default command line options for this script. +loadConfigFile() { + cat "$1" | sed $'/^\#/d;s/\r$//' +} + +# Now check to see if it's a good enough version +java_version_check() { + readonly java_version=$("$java_cmd" -version 2>&1 | awk -F '"' '/version/ {print $2}') + if [[ "$java_version" == "" ]]; then + echo + echo No java installations was detected. + echo Please go to https://adoptopenjdk.net/?variant=openjdk11&jvmVariant=hotspot and download + echo + exit 1 + else + local major=$(echo "$java_version" | cut -d'.' -f1) + if [[ "$major" -eq "1" ]]; then + local major=$(echo "$java_version" | cut -d'.' -f2) + fi + if [[ "$major" -lt "8" ]]; then + echo + echo The java installation you have is not up to date, eclair-front requires + echo at least version 1.8+ \(version 11 recommended\) but you have version $java_version + echo + echo Please go to 'https://adoptopenjdk.net/?variant=openjdk11&jvmVariant=hotspot' and download + echo a valid Java Runtime and install before running eclair-front. + echo + exit 1 + fi + fi +} + +### ------------------------------- ### +### Start of customized settings ### +### ------------------------------- ### +usage() { + cat < Define a custom main class + -jvm-debug Turn on JVM debugging, open at the given port. + -with-kanela Turn on kanela, akka's monitoring agent + + # java version (default: java from PATH, currently $(java -version 2>&1 | grep version)) + -java-home alternate JAVA_HOME + + # jvm options and output control + JAVA_OPTS environment variable, if unset uses "$java_opts" + -Dkey=val pass -Dkey=val directly to the java runtime + -J-X pass option -X directly to the java runtime + (-J is stripped) + + # special option + -- To stop parsing built-in commands from the rest of the command-line. + e.g.) enabling debug and sending -d as app argument + \$ ./start-script -d -- -d + +In the case of duplicated or conflicting options, basically the order above +shows precedence: JAVA_OPTS lowest, command line options highest except "--". +EOM +} + +### ------------------------------- ### +### Main script ### +### ------------------------------- ### + +declare -a residual_args +declare -a java_args +declare -a app_commands +declare -r real_script_path="$(realpath "$0")" +declare -r app_home="$(dirname $(dirname "$real_script_path"))" # two levels of dirname to go from ../bin/script.sh to ../ +declare -r lib_dir=("$app_home/lib") +declare -r app_mainclass=("fr.acinq.eclair.Boot") +declare -r app_entrypoint=$(ls $lib_dir | grep eclair-front) # TODO: improve this +declare -r app_classpath=("$lib_dir:$lib_dir/$app_entrypoint") + +# java_cmd is overrode in process_args when -java-home is used +declare java_cmd=$(get_java_cmd) + +# if configuration files exist, prepend their contents to $@ so it can be processed by this runner +[[ -f "$script_conf_file" ]] && set -- $(loadConfigFile "$script_conf_file") "$@" + +run "$@" diff --git a/eclair-front/src/main/resources/logback.xml b/eclair-front/src/main/resources/logback.xml index fd539b69e0..395f1dad75 100644 --- a/eclair-front/src/main/resources/logback.xml +++ b/eclair-front/src/main/resources/logback.xml @@ -18,27 +18,42 @@ - - - - - return formattedMessage.contains("connected to /10.") || - (formattedMessage.contains("connection closed") && !mdc.containsKey("nodeId")) || - (formattedMessage.contains("transport died") && !mdc.containsKey("nodeId")); - - - NEUTRAL - DENY - System.out false - ${HOSTNAME} %d %-5level %logger{24}%X{category}%X{nodeId}%X{channelId}%X{paymentHash}%.-11X{parentPaymentId}%.-11X{paymentId} - %msg%ex{12}%n + %yellow(${HOSTNAME} %d) %highlight(%-5level) %replace(%logger{24}){'\$.*',''}%X{category}%X{nodeId}%X{channelId}%X{paymentHash}%.-11X{parentPaymentId}%.-11X{paymentId} - %msg%ex{12}%n + + ${eclair.datadir:-${user.home}/.eclair}/eclair.log + + + ${eclair.datadir:-${user.home}/.eclair}/eclair-front.%d{yyyy-MM-dd}.log + + 90 + 5GB + + + %d %-5level %replace(%logger{24}){'\$.*',''}%X{category}%X{nodeId}%X{channelId}%X{paymentHash}%.-11X{parentPaymentId}%.-11X{paymentId} - %msg%ex{24}%n + + + + + + + + %d %-5level %replace(%logger{24}){'\$.*',''}%X{category}%X{nodeId}%X{channelId}%X{paymentHash}%.-11X{parentPaymentId}%.-11X{paymentId} - %msg%ex{24}%n + + + + + + + + - + \ No newline at end of file diff --git a/eclair-front/src/main/scala/fr/acinq/eclair/Boot.scala b/eclair-front/src/main/scala/fr/acinq/eclair/Boot.scala index 7c2d635400..b783a3b0b4 100644 --- a/eclair-front/src/main/scala/fr/acinq/eclair/Boot.scala +++ b/eclair-front/src/main/scala/fr/acinq/eclair/Boot.scala @@ -32,6 +32,7 @@ object Boot extends App with Logging { val config = ConfigFactory.parseString( Option(System.getenv("AKKA_CONF")).getOrElse("").replace(";", "\n"), ConfigParseOptions.defaults().setSyntax(ConfigSyntax.PROPERTIES)) + .withFallback(ConfigFactory.parseFile(new File(datadir, "eclair.conf"))) .withFallback(ConfigFactory.load()) // the actor system name needs to be the same for all members of the cluster diff --git a/eclair-front/src/main/scala/fr/acinq/eclair/FrontSetup.scala b/eclair-front/src/main/scala/fr/acinq/eclair/FrontSetup.scala index 29f7783299..5497374c3f 100644 --- a/eclair-front/src/main/scala/fr/acinq/eclair/FrontSetup.scala +++ b/eclair-front/src/main/scala/fr/acinq/eclair/FrontSetup.scala @@ -18,7 +18,6 @@ package fr.acinq.eclair import java.io.File import java.net.InetSocketAddress - import akka.Done import akka.actor.{ActorSystem, Address, Props, RootActorPath, SupervisorStrategy} import akka.pattern.ask @@ -27,12 +26,14 @@ import com.amazonaws.services.secretsmanager.AWSSecretsManagerClient import com.amazonaws.services.secretsmanager.model.GetSecretValueRequest import fr.acinq.bitcoin.Crypto.{PrivateKey, PublicKey} import fr.acinq.eclair.crypto.Noise.KeyPair +import fr.acinq.eclair.crypto.keymanager.LocalNodeKeyManager import fr.acinq.eclair.io.Switchboard.{GetRouterPeerConf, RouterPeerConf} import fr.acinq.eclair.io.{ClientSpawner, Server} import fr.acinq.eclair.router.FrontRouter import grizzled.slf4j.Logging import scodec.bits.ByteVector +import java.nio.file.Files import scala.concurrent.duration._ import scala.concurrent.{ExecutionContext, Future, Promise} @@ -65,6 +66,13 @@ class FrontSetup(datadir: File)(implicit system: ActorSystem) extends Logging { sm.shutdown() } case "env" => ByteVector.fromValidHex(config.getString("front.priv-key")) + case "seed" => + // demo in single-server setup + val chain = config.getString("chain") + val nodeSeedFilename: String = "node_seed.dat" + val seedPath = new File(datadir, nodeSeedFilename) + val nodeSeed = ByteVector(Files.readAllBytes(seedPath.toPath)) + new LocalNodeKeyManager(nodeSeed, NodeParams.hashFromChain(chain)).nodeKey.privateKey.value.bytes } val keyPair = KeyPair(pub, priv) require(PrivateKey(priv).publicKey == PublicKey(pub), "priv/pub keys mismatch") diff --git a/eclair-node-gui/pom.xml b/eclair-node-gui/pom.xml index 0f6ca5bff9..39833b76af 100644 --- a/eclair-node-gui/pom.xml +++ b/eclair-node-gui/pom.xml @@ -56,11 +56,6 @@ modules/assembly.xml - - - fr.acinq.eclair.JavafxBoot - - diff --git a/eclair-node/pom.xml b/eclair-node/pom.xml index 5cb6ce00b5..ddf24e0da5 100644 --- a/eclair-node/pom.xml +++ b/eclair-node/pom.xml @@ -56,11 +56,6 @@ modules/assembly.xml - - - fr.acinq.eclair.Boot - - From c2f62de106cce9e774ef7d3c569e80039a71b463 Mon Sep 17 00:00:00 2001 From: pm47 Date: Thu, 17 Dec 2020 16:59:16 +0100 Subject: [PATCH 2/3] fixup! add cluster doc, and packaging improvements --- docs/Cluster.md | 6 +++--- eclair-front/src/main/resources/application.conf | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/Cluster.md b/docs/Cluster.md index f27a15b5d1..2ad0d746ea 100644 --- a/docs/Cluster.md +++ b/docs/Cluster.md @@ -74,7 +74,7 @@ NB: we override the ports, otherwise they would conflict since in this example e In production you should: - run multiple `frontend`s - run one app per server -- enable `tcp-tls` to encrypted between members of the cluster with your own generated certificate (see below) +- enable `tcp-tls` to encrypt communications between members of the cluster with your own generated certificate (see below) - use a load balancer to hide all your `frontend` servers under the same ip address - set firewall rules to disable lightning connections (port 9735) on your `backend` server, so all connections go through the `frontend` - enable [monitoring](Monitoring.md) @@ -122,7 +122,7 @@ You can run it as-is for testing. If you intend to use it in production, you need to enable encryption with your own certificate: 1. Follow the procedure above to generate your `akka-tls.jks` -2. We recommend forking the project and build your own bundle: +2. We recommend forking the project and building your own bundle: ```shell $ git clone git@github.com:ACINQ/eclair.git $ vi eclair-core/src/main/reference.conf # set akka.remote.artery.transport = "tls-tcp" @@ -134,7 +134,7 @@ Alternatively, you can also edit the existing bundle and manually add the `akka- #### Private key -In production, we highly recommend using AWS Secrets manager to provide the node private key. This is done by setting `eclair.front.priv-key-provider=aws-sm`. Default secret name is "node-priv-key", but it is configurable with `eclair.front.aws-sm.aws-sm.priv-key-name` +In production, we highly recommend using AWS Secrets manager to provide the node private key. This is done by setting `eclair.front.priv-key-provider=aws-sm`. Default secret name is "node-priv-key", but it is configurable with `eclair.front.aws-sm.priv-key-name` #### Configuration diff --git a/eclair-front/src/main/resources/application.conf b/eclair-front/src/main/resources/application.conf index 2af09a5920..1e5d3f95dd 100644 --- a/eclair-front/src/main/resources/application.conf +++ b/eclair-front/src/main/resources/application.conf @@ -8,7 +8,7 @@ eclair { aws-sm.priv-key-name = "node-priv-key" // used if priv-key-provider = aws-sm // As a security measure, we also require the pub key, which will be matched against the priv key to make sure the // front is really using the expected key - pub = ${NODE_PUB_KEY} + pub = ${?NODE_PUB_KEY} backend.ip = 127.0.0.1 backend.ip = ${?BACKEND_IP} From aeb494785f996bbdd666813fe2e2e81099d7ca1d Mon Sep 17 00:00:00 2001 From: pm47 Date: Fri, 18 Dec 2020 11:19:28 +0100 Subject: [PATCH 3/3] make system props take precedence over eclair.conf --- eclair-front/src/main/scala/fr/acinq/eclair/Boot.scala | 1 + 1 file changed, 1 insertion(+) diff --git a/eclair-front/src/main/scala/fr/acinq/eclair/Boot.scala b/eclair-front/src/main/scala/fr/acinq/eclair/Boot.scala index b783a3b0b4..27f11d5eed 100644 --- a/eclair-front/src/main/scala/fr/acinq/eclair/Boot.scala +++ b/eclair-front/src/main/scala/fr/acinq/eclair/Boot.scala @@ -32,6 +32,7 @@ object Boot extends App with Logging { val config = ConfigFactory.parseString( Option(System.getenv("AKKA_CONF")).getOrElse("").replace(";", "\n"), ConfigParseOptions.defaults().setSyntax(ConfigSyntax.PROPERTIES)) + .withFallback(ConfigFactory.parseProperties(System.getProperties)) .withFallback(ConfigFactory.parseFile(new File(datadir, "eclair.conf"))) .withFallback(ConfigFactory.load())