make mobile fails

[madhms]

Background

Building Android & IOS libraries fails. I cloned lnd into the GOPATH src like so:

.
└── go/
    └── src/
        └── lnd

Output:

zzz@zzzs-MacBook-Air lnd % sudo make android
 Re-creating vendor directory.
rm -r vendor/; GO111MODULE=on go mod vendor
 Creating mobile RPC from protos.
cd ./lnrpc; COMPILE_MOBILE=1 ./gen_protos_docker.sh
Building protobuf compiler docker image...
[+] Building 1.1s (8/8) FINISHED                                                                                            
 => [internal] load build definition from Dockerfile                                                                   0.0s
 => => transferring dockerfile: 37B                                                                                    0.0s
 => [internal] load .dockerignore                                                                                      0.0s
 => => transferring context: 2B                                                                                        0.0s
 => [internal] load metadata for docker.io/library/golang:1.15.7-buster                                                0.9s
 => [1/4] FROM docker.io/library/golang:1.15.7-buster@sha256:83b4fd9355ba9ff2d0c78cdf6fc485f739a2834c27e3160a5026faf6  0.0s
 => CACHED [2/4] RUN apt-get update && apt-get install -y   git   protobuf-compiler='3.6.1*'   clang-format='1:7.0*'   0.0s
 => CACHED [3/4] RUN cd /tmp   && mkdir -p /tmp/build/.cache   && mkdir -p /tmp/build/.modcache   && export GO111MODU  0.0s
 => CACHED [4/4] WORKDIR /build                                                                                        0.0s
 => exporting to image                                                                                                 0.0s
 => => exporting layers                                                                                                0.0s
 => => writing image sha256:bb6413874d970f72b903734a110fc77ee5858bf1a82ac96bc03579fb82bf5616                           0.0s
 => => naming to docker.io/library/lnd-protobuf-builder                                                                0.0s
Compiling and formatting *.proto files...
/build/lnrpc /build
Generating root gRPC server protos
Generating protos from rpc.proto, into .
Generating protos from walletunlocker.proto, into .
Generating protos from autopilotrpc/autopilot.proto, into autopilotrpc
Generating protos from chainrpc/chainnotifier.proto, into chainrpc
Generating protos from invoicesrpc/invoices.proto, into invoicesrpc
Generating protos from lnclipb/lncli.proto, into lnclipb
Generating protos from routerrpc/router.proto, into routerrpc
Generating protos from signrpc/signer.proto, into signrpc
Generating protos from verrpc/verrpc.proto, into verrpc
Generating protos from walletrpc/walletkit.proto, into walletrpc
Generating protos from watchtowerrpc/watchtower.proto, into watchtowerrpc
Generating protos from wtclientrpc/wtclient.proto, into wtclientrpc
/build
/build/mobile /build
Using plugin /go/bin/falafel v0.7.1
Generating mobile protos from rpc.proto
Generating mobile protos from walletunlocker.proto
Generating mobile protos from ../lnrpc/autopilotrpc/autopilot.proto, with build tag autopilotrpc
Generating mobile protos from ../lnrpc/chainrpc/chainnotifier.proto, with build tag chainrpc
Generating mobile protos from ../lnrpc/invoicesrpc/invoices.proto, with build tag invoicesrpc
Generating mobile protos from ../lnrpc/lnclipb/lncli.proto, with build tag lnclipb
Generating mobile protos from ../lnrpc/routerrpc/router.proto, with build tag routerrpc
Generating mobile protos from ../lnrpc/signrpc/signer.proto, with build tag signrpc
Generating mobile protos from ../lnrpc/verrpc/verrpc.proto, with build tag verrpc
Generating mobile protos from ../lnrpc/walletrpc/walletkit.proto, with build tag walletrpc
Generating mobile protos from ../lnrpc/watchtowerrpc/watchtower.proto, with build tag watchtowerrpc
Generating mobile protos from ../lnrpc/wtclientrpc/wtclient.proto, with build tag wtclientrpc
/build
 Building Android library (/Users/zzz/go/src/github.com/lightningnetwork/lnd/mobile/build/android/Lndmobile.aar).
mkdir -p /Users/zzz/go/src/github.com/lightningnetwork/lnd/mobile/build/android
GO111MODULE=off /Users/zzz/go/bin/gomobile bind -target=android -tags="mobile dev autopilotrpc" -ldflags " -s -w -X github.com/lightningnetwork/lnd/build.Commit=v0.12.0-beta-177-g56b61078-dirty -X github.com/lightningnetwork/lnd/build.CommitHash=56b61078c5b2be007d318673a5f3b40c6346883a -X github.com/lightningnetwork/lnd/build.GoVersion=go1.16 -X github.com/lightningnetwork/lnd/build.RawTags=" -v -o /Users/zzz/go/src/github.com/lightningnetwork/lnd/mobile/build/android/Lndmobile.aar github.com/lightningnetwork/lnd/mobile
no exported names in the package "github.com/lightningnetwork/lnd/mobile"
no exported names in the package "github.com/lightningnetwork/lnd/mobile"
no exported names in the package "github.com/lightningnetwork/lnd/mobile"
no exported names in the package "github.com/lightningnetwork/lnd/mobile"
/Users/zzz/go/bin/gomobile: /Users/zzz/go/bin/gobind -lang=go,java -outdir=/tmp/gomobile-work-672796928 -tags=mobile,dev,autopilotrpc github.com/lightningnetwork/lnd/mobile failed: exit status 1

make: *** [android] Error 1

Your environment

• lnd v0.12.1-beta
• Darwin zzzs-MacBook-Air.local 20.1.0 Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:11 PDT 2020; root:xnu-7195.50.7~2/RELEASE_X86_64 x86_64

[guggero]

I suspect that the sudo in front of make causes this (somehow mixing up stuff between root's and your own home directories).
If you're using sudo because of Docker, you need to set it up correctly to make it work without sudo (I think it just requires you to add your user to the docker group).

[madhms]

I think that only works for Linux but I couldn't find any information in the docs for macOS. I still tried to add the docker user group and add the root user but doesn't seem to work.

Docker itself can run but during runtime it needs to read files requiring root privileges:

zzz@zzzs-MacBook-Air lnd % make android
 Re-creating vendor directory.
rm -r vendor/; GO111MODULE=on go mod vendor
 Creating mobile RPC from protos.
cd ./lnrpc; COMPILE_MOBILE=1 ./gen_protos_docker.sh
Building protobuf compiler docker image...
[+] Building 1.2s (3/3) FINISHED                                                                                                        
 => [internal] load build definition from Dockerfile                                                                               0.0s
 => => transferring dockerfile: 37B                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                  0.0s
 => => transferring context: 2B                                                                                                    0.0s
 => ERROR [internal] load metadata for docker.io/library/golang:1.15.7-buster                                                      1.1s
------
 > [internal] load metadata for docker.io/library/golang:1.15.7-buster:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: rpc error: code = Unknown desc = open /Users/zzz/.docker/.token_seed: permission denied
make: *** [mobile-rpc] Error 1

[halseth]

Looks like something is wrong with your docker setup. Can you try make rpc?

[madhms]

I fixed it by removing GO111MODULE=off from GOMOBILE_BIN := GO111MODULE=off $(GO_BIN)/gomobile in the Makefile. Probably caused by changes which were added in Go 1.16.

I imported the library into my project but whenever calling the native start method the app crashes.

public static native void start(String var0, Callback var1, Callback var2);

My code to call it:

    public void start(final Promise promise) {
        File appDir = getReactApplicationContext().getFilesDir();
        copyConfig(appDir);

        final String logDir = appDir + "/logs/bitcoin/testnet";
        final String logFile = logDir + "/lnd.log";

        FileInputStream stream = null;
        while (true) {
            try {
                stream = new FileInputStream(logFile);
            } catch (FileNotFoundException e) {
                File dir = new File(logDir);
                dir.mkdirs();
                File f = new File(logFile);
                try {
                    f.createNewFile();
                    continue;
                } catch (IOException e1) {
                    e1.printStackTrace();
                    return;
                }
            }
            break;
        }

        final InputStreamReader istream = new InputStreamReader(stream);
        final BufferedReader buf = new BufferedReader(istream);
        try {
            readToEnd(buf, false);
        } catch (IOException e) {
            e.printStackTrace();
            return;
        }

        logObserver = new FileObserver(logFile) {
            @Override
            public void onEvent(int event, String file) {
                if(event != FileObserver.MODIFY) {
                    return;
                }
                try {
                    readToEnd(buf, true);
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        };
        logObserver.startWatching();
        Log.i("LndNativeModule", "Started watching " + logFile);

        final String args = "--lnddir=" + appDir;
        Log.i("LndNativeModule", "Starting LND with args " + args);


        class UnlockCallback implements Callback {
            @Override
            public void onError(Exception e) {
                Log.i("Lndmobile", "Wallet unlock err: " + e.getMessage());
                Log.d("Lndmobile", "Wallet unlock err: " + e.getMessage());
                promise.reject(e);
            }
            @Override
            public void onResponse(byte[] bytes) {
                Log.i("Lndmobile", "Wallet ready to be unlocked");
                Log.d("Lndmobile", "Wallet ready to be unlocked");
                promise.resolve("unlocked");
            }
        }
        class RPCCallback implements Callback {
            @Override
            public void onError(Exception e) {
                Log.i("Lndmobile", "RPC start err: " + e.getMessage());
                Log.d("Lndmobile", "RPC start err: " + e.getMessage());
            }
            @Override
            public void onResponse(byte[] bytes) {
                Log.i("Lndmobile", "RPC ready for requests");
                Log.d("Lndmobile", "RPC ready for requests");
            }
        }

        Runnable startLnd = new Runnable() {
            @Override
            public void run() {
                Lndmobile.start(args, new UnlockCallback(), new RPCCallback());
            }
        };
        new Thread(startLnd).start();
    }

I tried it on a physical device (arm-v7) and an emulator (arm64).

Crash log on arm-v7:

E/Sensors: handleToDriver handle(0)

That doesn't really help figuring out what's causing the crash since it's caused somewhere in the native code. Is there a way to get more specific information about what actually causes the crash?

[halseth]

@madhms What version of gomobile are you using? And which Android SDK are you targetting (there is a known issue with API level 30: golang/go#40569)

I haven't tried go 1.16 on mobile yet, unfortunately. Can you try using go 1.15 and see if that resolves your issue for now?

[halseth]

Looks like you are experiencing a React Issue: facebook/react-native#24097

I would try to update your React setup, perhaps try to isolate the issue with a smaller native module than LND then file an issue on their repo.

[madhms]

Looks like you are experiencing a React Issue: facebook/react-native#24097

I would try to update your React setup, perhaps try to isolate the issue with a smaller native module than LND then file an issue on their repo.

I created a native Android project but still happening, the error probably just means that React loses connection to the native driver because of the crash.

I tested it on API 22 and 25. For some reason I can't display the gomobile version but that's probably a bug. I also tried someone else's compiled library but it still crashes.

If you can upload your library I could try to see if it's a compilation issue.

[halseth]

@madhms Do you have a lnd conf file set?

My hunch is that lnd fails to start because something is wrong with your config. If this causes you app to shut down, then you might not see what the actual problem is.

I modified the code you posted and ran it with my own compiled library. Had to add a few startup args to get it to start on the emulator:

final String args = "--lnddir=" + appDir + " --bitcoin.active --bitcoin.testnet --bitcoin.node=neutrino --norest --nolisten";

[madhms]

Okay that seems to work. In your article you're stating that these two dependencies are required to generate the data passed to the native functions:

        implementation 'com.google.protobuf:protobuf-java:3.10.0'
        implementation 'com.google.api.grpc:proto-google-common-protos:1.17.0'

They require min SDK 26. Would it be possible to generate the byte array in the Javascript version and pass it to the native function?

[halseth]

@madhms Absolutely! This is done in several apps, you can generate the bytes in any language you want using a protobuf library, then pass them to lnd.

Btw, I realized the reason you didn't see the error before the app crashes is because we hard exit if it happens. I created a PR to resolve this: #5086