Build status-go on windows and macos

[cammellos]

Would it be possible to add a build of status-go on CI on windows? it happened that status-go could not be built on windows, and we only realised when integrating with desktop.

cc @jakubgs

[igor-sirotin]

Probably MacOS build is a good idea too?

[jakubgs]

The way we should build status-go for Windows and MacOS should be informed by how they are built for the status-desktop application, which we can see in the Jenkinsfiles:
https://github.com/status-im/status-desktop/blob/0645ed47123f742faa2e110bc6daf1f2320dedd8/ci/Jenkinsfile.macos#L96-L100
https://github.com/status-im/status-desktop/blob/0645ed47123f742faa2e110bc6daf1f2320dedd8/ci/Jenkinsfile.windows#L81-L85

And the Makefile:
https://github.com/status-im/status-desktop/blob/0645ed47123f742faa2e110bc6daf1f2320dedd8/Makefile#L430-L434

Which appears to be simply building the statusgo-shared-library target from status-go:

status-go/Makefile

Lines 229 to 248 in 551af54

	statusgo-shared-library: ##@cross-compile Build status-go as shared library for current platform
	## cmd/library/README.md explains the magic incantation behind this
	mkdir -p build/bin/statusgo-lib
	go run cmd/library/*.go > build/bin/statusgo-lib/main.go
	@echo "Building shared library..."
	@echo "Tags: $(BUILD_TAGS)"
	$(GOBIN_SHARED_LIB_CFLAGS) $(GOBIN_SHARED_LIB_CGO_LDFLAGS) go build \
	-tags '$(BUILD_TAGS)' \
	$(BUILD_FLAGS) \
	-buildmode=c-shared \
	-o build/bin/libstatus.$(GOBIN_SHARED_LIB_EXT) \
	./build/bin/statusgo-lib
	ifeq ($(detected_OS),Linux)
	cd build/bin && \
	ls -lah . && \
	mv ./libstatus.$(GOBIN_SHARED_LIB_EXT) ./libstatus.$(GOBIN_SHARED_LIB_EXT).0 && \
	ln -s ./libstatus.$(GOBIN_SHARED_LIB_EXT).0 ./libstatus.$(GOBIN_SHARED_LIB_EXT)
	endif
	@echo "Shared library built:"
	@ls -la build/bin/libstatus.*

With one special case for Darwin aarch64 where STATUSGO_MAKE_PARAMS are overridden:
https://github.com/status-im/status-desktop/blob/0645ed47123f742faa2e110bc6daf1f2320dedd8/Makefile#L203

[jakubgs]

Considering we should probably build both statusgo-library and statusgo-shared-libraryb targets for both platforms like we do for Linux we can probably re-use the same Jenkinsfile for other platforms:

status-go/_assets/ci/Jenkinsfile.linux

Lines 52 to 63 in 551af54

	/* Sanity-check C bindings */
	stage('Build Static Lib') {
	steps { script {
	nix.shell('make statusgo-library', pure: false)
	} }
	}
	stage('Build Shared Lib') {
	steps { script {
	nix.shell('make statusgo-shared-library', pure: false)
	} }
	}

The only tricky part I can see is the use of statically defined Jenkins slave labels:

status-go/_assets/ci/Jenkinsfile.linux

Line 5 in 551af54

agent { label 'linux && x86_64 && nix-2.19' }

One way to work around that is to do what we do in nimbus-eth2:

  /* This way we run the same Jenkinsfile on different platforms. */
  agent { label params.AGENT_LABEL }

https://github.com/status-im/nimbus-eth2/blob/cb2c379b373d429c12f22d13a61b6dd4b0cbbcde/ci/Jenkinsfile#L12-L13

The AGENT_LABEL parameter default is generated using a getAgentLabel() function:

    string(
      name: 'AGENT_LABEL',
      description: 'Label for targetted CI slave host: linux/macos',
      defaultValue: params.AGENT_LABEL ?: getAgentLabel(),
    )

https://github.com/status-im/nimbus-eth2/blob/83333658489bc7a5d2533424a20d52f7ffd914d7/ci/Jenkinsfile#L16-L20

Which generates job labels based off of the Jenkins job path tokens:

/* This allows us to use one Jenkinsfile and run
 * jobs on different platforms based on job name. */
def getAgentLabel() {
    if (params.AGENT_LABEL) { return params.AGENT_LABEL }
    /* We extract the name of the job from currentThread because
     * before an agent is picket env is not available. */
    def tokens = Thread.currentThread().getName().split('/')
    def labels = []
    /* Check if the job path contains any of the valid labels. */
    ['linux', 'macos', 'x86_64', 'aarch64', 'arm64'].each {
        if (tokens.contains(it)) { labels.add(it) }
    }
    return labels.join(' && ')
}

https://github.com/status-im/nimbus-eth2/blob/83333658489bc7a5d2533424a20d52f7ffd914d7/ci/Jenkinsfile#L170-L183

This probably means that the path of status-go jobs might need to be modified.

[jakubgs]

Current Jenkins job paths for status-go are like this:

• https://ci.status.im/job/status-go/job/prs/job/android/
• https://ci.status.im/job/status-go/job/prs/job/ios/
• https://ci.status.im/job/status-go/job/prs/job/linux/

And we'll need to change that to take into account architecture, since MacOS builds will probably have to exist both for x86_64 and aarch64. For that reason we'll need the paths to end at the platform, or even better, at just a main package build, so:

• https://ci.status.im/job/status-go/job/prs/job/android/job/x86_64/job/main
• https://ci.status.im/job/status-go/job/prs/job/ios/job/aarch64/job/main
• https://ci.status.im/job/status-go/job/prs/job/linux/job/x86_64/job/main
• https://ci.status.im/job/status-go/job/prs/job/macos/job/aarch64/job/main
• https://ci.status.im/job/status-go/job/prs/job/macos/job/x86_64/job/main
• https://ci.status.im/job/status-go/job/prs/job/windows/job/x86_64/job/main

By including a main name at the end of the path we allow ourselves the space to add more types of builds under given platforms. One example of such an extra build was a new compiler version build for Nim v2.

[jakubgs]

For clarity of purpose for the shared Jenkinsfile we cold also include symlinks like so:

Jeninsfile.windows -> Jenkinsfile.library
Jeninsfile.linux -> Jenkinsfile.library
Jeninsfile.macos -> Jenkinsfile.library

It could also be called Jenkinsfile.desktop I guess.

[jakubgs]

One issue with re-using the Jenkinsfile might be the use of s5cmd to upload builds:

status-go/_assets/ci/Jenkinsfile.linux

Lines 72 to 76 in 551af54

	stage('Upload') {
	steps { script {
	env.PKG_URL = s5cmd.upload(ARTIFACT)
	} }
	}

Which currently gets the s5cmd tool via a Nix Shell:

    nix.shell(
      """
      s5cmd ${flags.join(" ")} ${path} s3://${opts.bucket}/
      """,
      packages: ['s5cmd'],
      pure: false,
    )

https://github.com/status-im/status-jenkins-lib/blob/ab9aa93c13eb42fb524b2f8e54521c73da140c9c/vars/s5cmd.groovy#L36-L42

One difficult solution to this problem might be use of NixOS-WSL image to use Nix on Windows, but that's something that not been researched yet. Might be cool to try though.

Another option would be to modify status-jenkins-lib to allow use of s5cmd from the host system without Nix Shell, which is probably the simpler way, but also not as cool.

You can find an example of how that can be done in s3cmd usage:
https://github.com/status-im/status-jenkins-lib/blob/ab9aa93c13eb42fb524b2f8e54521c73da140c9c/vars/s3.groovy#L40-L47

Or you could simply use s3cmd instead of s5cmd on Windows. But it is slower.

[jakubgs]

The PLATFORM variable should also be dynamically updated:

status-go/_assets/ci/Jenkinsfile.linux

Line 35 in 551af54

PLATFORM = 'linux'

Since it informs the platform column in comments made by github-comment-manager service:
https://github.com/status-im/status-jenkins-lib/blob/ab9aa93c13eb42fb524b2f8e54521c73da140c9c/vars/ghcmgr.groovy#L20

[jakubgs]

Since WSL is problematic due to issues with paths, as well as possibly being equal to cross-compiling from Linux, it might be simpler to just do what we do in Windows Desktop builds and simply override SHELL to avoid using Nix Shell:

	$(MAKE) -C vendor/status-go statusgo-shared-library SHELL=/bin/sh \
		$(STATUSGO_MAKE_PARAMS) $(HANDLE_OUTPUT)

https://github.com/status-im/status-desktop/blob/fa500a3b5b58444764a4c803c8b6458b3aff8cd9/Makefile#L440-L441

Since in the context of a Nix shell within WSL we would simply be using the same tooling as on Linux, even if local system tooling is available from the Windows host system.