Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Builtin Test Audio Module #11

Merged
merged 5 commits into from
Apr 14, 2024
Merged

Conversation

duvallj
Copy link
Collaborator

@duvallj duvallj commented Apr 10, 2024

@MatthD This should also fix #2 ; do you mind testing this locally? I've also cherry-picked your other relevant changes from #10

@nakagawa424
Copy link

linux-x64 build may broken.

#2 (comment)
This check was built by overwriting src/ from ec0ea06 into b5351f1 build environment.

on linux-x64, building ec0ea06 fails:

gn gen ${BINARY_DIR} "--args=${GN_GEN_ARGS}"
ERROR at //build/config/sysroot.gni:64:7: Assertion failed.
      assert(
      ^-----
Missing sysroot (//build/linux/debian_sid_amd64-sysroot). To fix, run: build/linux/sysroot_scripts/install-sysroot.py --arch=amd64
See //build/config/sysroot.gni:65:11: 
          exec_script("//build/dir_exists.py",
          ^-----------------------------------
This is where it was set.
See //build/config/linux/pkg_config.gni:5:1: whence it was imported.
import("//build/config/sysroot.gni")
^----------------------------------
See //BUILD.gn:15:1: whence it was imported.
import("//build/config/linux/pkg_config.gni")
^-------------------------------------------
ninja: build stopped: subcommand failed.
ERR! OMG Process terminated: 1

Reverting part of d4aceff fixes this fail on linux-x64.

diff --git a/scripts/configure-webrtc.sh b/scripts/configure-webrtc.sh
index 6eeb93b..4244ebf 100755
--- a/scripts/configure-webrtc.sh
+++ b/scripts/configure-webrtc.sh
@@ -8,6 +8,14 @@ export PATH=$DEPOT_TOOLS:$PATH
 
 cd ${SOURCE_DIR}
 
+if [ "$TARGET_ARCH" == "arm" ]; then
+  python build/linux/sysroot_scripts/install-sysroot.py --arch=arm
+elif [ "$TARGET_ARCH" == "arm64" ]; then
+  python build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
+else
+  python build/linux/sysroot_scripts/install-sysroot.py --arch=amd64
+fi
+
 # NOTE(mroberts): Running hooks generates this file, but running hooks also
 # takes too long in CI; so do this manually.
 (cd build/util && python lastchange.py -o LASTCHANGE)

1 similar comment
@nakagawa424
Copy link

linux-x64 build may broken.

#2 (comment)
This check was built by overwriting src/ from ec0ea06 into b5351f1 build environment.

on linux-x64, building ec0ea06 fails:

gn gen ${BINARY_DIR} "--args=${GN_GEN_ARGS}"
ERROR at //build/config/sysroot.gni:64:7: Assertion failed.
      assert(
      ^-----
Missing sysroot (//build/linux/debian_sid_amd64-sysroot). To fix, run: build/linux/sysroot_scripts/install-sysroot.py --arch=amd64
See //build/config/sysroot.gni:65:11: 
          exec_script("//build/dir_exists.py",
          ^-----------------------------------
This is where it was set.
See //build/config/linux/pkg_config.gni:5:1: whence it was imported.
import("//build/config/sysroot.gni")
^----------------------------------
See //BUILD.gn:15:1: whence it was imported.
import("//build/config/linux/pkg_config.gni")
^-------------------------------------------
ninja: build stopped: subcommand failed.
ERR! OMG Process terminated: 1

Reverting part of d4aceff fixes this fail on linux-x64.

diff --git a/scripts/configure-webrtc.sh b/scripts/configure-webrtc.sh
index 6eeb93b..4244ebf 100755
--- a/scripts/configure-webrtc.sh
+++ b/scripts/configure-webrtc.sh
@@ -8,6 +8,14 @@ export PATH=$DEPOT_TOOLS:$PATH
 
 cd ${SOURCE_DIR}
 
+if [ "$TARGET_ARCH" == "arm" ]; then
+  python build/linux/sysroot_scripts/install-sysroot.py --arch=arm
+elif [ "$TARGET_ARCH" == "arm64" ]; then
+  python build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
+else
+  python build/linux/sysroot_scripts/install-sysroot.py --arch=amd64
+fi
+
 # NOTE(mroberts): Running hooks generates this file, but running hooks also
 # takes too long in CI; so do this manually.
 (cd build/util && python lastchange.py -o LASTCHANGE)

@MatthD
Copy link

MatthD commented Apr 11, 2024

[ "$TARGET_ARCH" == "arm" ]; then

Everything was working well on darwin but not on linux yes (for build), was debugging at the same time as you !
To be able to approve this PR I wanted to generate a build on linux arm64/x64 and test it inside my docker app which exposed the issue about RTCaudiosink.

@nakagawa424 can you publish a small pr that point to this one with your fix please 🙏

@duvallj
Copy link
Collaborator Author

duvallj commented Apr 11, 2024

@nakagawa424 @MatthD I've updated this PR with the script to download the sysroot on Linux, do you mind testing if it works for you?

@MatthD
Copy link

MatthD commented Apr 11, 2024

@nakagawa424 @MatthD I've updated this PR with the script to download the sysroot on Linux, do you mind testing if it works for you?

I am testing on darwin/linux right now ;) seems not able to compile via npm run build on docker via linux arm 64 I had too pass FROM --platform=linux/amd64 .
So not sure this will work for linux-arm64, will try after

@MatthD
Copy link

MatthD commented Apr 11, 2024

So for linux-x64 it's ok it build
For linux-arm64 it result in issue:

Running cmake-js build
info TOOL Using Ninja generator, because ninja is available.
info CMD BUILD
info RUN [ 'cmake', '--build', '/app/build-linux-arm64', '--config', 'Release' ]
[7/106] Performing update step for 'project_depot_tools'
Previous HEAD position was a7c97f78 Roll recipe dependencies (trivial).
HEAD is now at 4eaefabc Roll recipe dependencies (trivial).
[13/106] Performing configure step for 'project_libwebrtc'
FAILED: external/libwebrtc/stamp/project_libwebrtc-configure /app/build-linux-arm64/external/libwebrtc/stamp/project_libwebrtc-configure
cd /app/build-linux-arm64/external/libwebrtc/build/Release && /usr/bin/cmake -E env BINARY_DIR=/app/build-linux-arm64/external/libwebrtc/build/Release DEPOT_TOOLS=/app/build-linux-arm64/external/depot_tools/src "GN_GEN_ARGS=rtc_build_examples=false rtc_use_x11=false rtc_enable_protobuf=false rtc_include_pulse_audio=false rtc_include_tests=false use_lld=false use_custom_libcxx=false target_cpu=\"arm64\" rtc_build_tools=true is_debug=false" SOURCE_DIR=/app/build-linux-arm64/external/libwebrtc/download/src /app/scripts/configure-webrtc.sh && /usr/bin/cmake -E touch /app/build-linux-arm64/external/libwebrtc/stamp/project_libwebrtc-configure

export PATH=$DEPOT_TOOLS:$PATH

cd ${SOURCE_DIR}

if [ "$TARGET_ARCH" == "arm" ]; then
  python build/linux/sysroot_scripts/install-sysroot.py --arch=arm
elif [ "$TARGET_ARCH" == "arm64" ]; then
  python build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
else
  python build/linux/sysroot_scripts/install-sysroot.py --arch=amd64
fi
Installing Debian sid arm64 root image: /app/build-linux-arm64/external/libwebrtc/download/src/build/linux/debian_sid_arm64-sysroot
Downloading https://commondatastorage.googleapis.com/chrome-linux-sysroot/toolchain/2befe8ce3e88be6080e4fb7e6d412278ea6a7625/debian_sid_arm64_sysroot.tar.xz

# NOTE(mroberts): Running hooks generates this file, but running hooks also
# takes too long in CI; so do this manually.
(cd build/util && python lastchange.py -o LASTCHANGE)
/app/build-linux-arm64/external/libwebrtc/download/src/build/util/lastchange.py:307: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
  lastchange_year = datetime.datetime.utcfromtimestamp(

gn gen ${BINARY_DIR} "--args=${GN_GEN_ARGS}"
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
ninja: build stopped: subcommand failed.
ERR! OMG Process terminated: 1
/app/scripts/build-from-source.js:41
    throw new Error("cmake-js build failed for wrtc");

I am using nikolaik/python-nodejs:python3.12-nodejs20image.. will try one based on ubuntu

UPDATE:

Result is similar with a full ubuntu arm 64 image

python src/tools/clang/scripts/update.py
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Linux_x64/clang-llvmorg-14-init-1002-gb5e470aa-1.tgz .......... Done.

rm -f webrtc

ln -s src webrtc
[21/114] Performing configure step for 'project_libwebrtc'
FAILED: external/libwebrtc/stamp/project_libwebrtc-configure /home/app/build-linux-arm64/external/libwebrtc/stamp/project_libwebrtc-configure
cd /home/app/build-linux-arm64/external/libwebrtc/build/Release && /usr/bin/cmake -E env BINARY_DIR=/home/app/build-linux-arm64/external/libwebrtc/build/Release DEPOT_TOOLS=/home/app/build-linux-arm64/external/depot_tools/src "GN_GEN_ARGS=rtc_build_examples=false rtc_use_x11=false rtc_enable_protobuf=false rtc_include_pulse_audio=false rtc_include_tests=false use_lld=false use_custom_libcxx=false target_cpu=\"arm64\" rtc_build_tools=true is_debug=false" SOURCE_DIR=/home/app/build-linux-arm64/external/libwebrtc/download/src /home/app/scripts/configure-webrtc.sh && /usr/bin/cmake -E touch /home/app/build-linux-arm64/external/libwebrtc/stamp/project_libwebrtc-configure

export PATH=$DEPOT_TOOLS:$PATH

cd ${SOURCE_DIR}

if [ "$TARGET_ARCH" == "arm" ]; then
  python build/linux/sysroot_scripts/install-sysroot.py --arch=arm
elif [ "$TARGET_ARCH" == "arm64" ]; then
  python build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
else
  python build/linux/sysroot_scripts/install-sysroot.py --arch=amd64
fi
Installing Debian sid arm64 root image: /home/app/build-linux-arm64/external/libwebrtc/download/src/build/linux/debian_sid_arm64-sysroot
Downloading https://commondatastorage.googleapis.com/chrome-linux-sysroot/toolchain/2befe8ce3e88be6080e4fb7e6d412278ea6a7625/debian_sid_arm64_sysroot.tar.xz

# NOTE(mroberts): Running hooks generates this file, but running hooks also
# takes too long in CI; so do this manually.
(cd build/util && python lastchange.py -o LASTCHANGE)

gn gen ${BINARY_DIR} "--args=${GN_GEN_ARGS}"
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
ninja: build stopped: subcommand failed.
ERR! OMG Process terminated: 1
/home/app/scripts/build-from-source.js:41
    throw new Error("cmake-js build failed for wrtc");
    ^

Error: cmake-js build failed for wrtc
    at main (/home/app/scripts/build-from-source.js:41:11)
    at Object.<anonymous> (/home/app/scripts/build-from-source.js:50:3)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Module._load (node:internal/modules/cjs/loader:1022:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12)
    at node:internal/main/run_main_module:28:49

Node.js v20.12.2
root@5b265b95b365:/home/app

@MatthD
Copy link

MatthD commented Apr 12, 2024

@nakagawa424
For the moment I was not able to build everythin inside a docker container on arm64.
if someone could help on that it's failing at the step of configuration where the gn tool downloaded is a x64 type and not arm64 ..
This is the Dockerfile I have put inside the project, not working on my mac arm M2

FROM ubuntu:22.04

USER root
WORKDIR /home/app
COPY . .
RUN apt-get update
RUN apt-get -y install curl gnupg g++ gcc make cmake build-essential ninja-build python3 python-is-python3 gcc-arm-linux-gnueabihf  libc6-dev-armhf-cross qemu-user-static qemu-system-i386  qemu-system-arm qemu-system-mips qemu-efi-aarch64 qemu-kvm

RUN apt-get install -y \
apparmor \
automake \
bash-completion \
build-essential \
cmake \
curl \
g++ \
gcc \
git \
iptables \
jq \
libapparmor-dev \
libc6-dev \
libcap-dev \
libsystemd-dev \
libyaml-dev \
mercurial \
net-tools \
parallel \
pkg-config \
golang-go \
iproute2 \
iputils-ping \
vim-common \
vim \
wget \
curl \
--no-install-recommends \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*


# Set environment variables
ENV DEBIAN_FRONTEND=noninteractive

# Install Mantatory tools (curl git python3) and optional tools (vim sudo)
RUN apt-get update && \
    apt-get install -y curl git lsb-release python3 git file vim sudo && \
    rm -rf /var/lib/apt/lists/*

# Configure git for safe.directory
RUN git config --global --add safe.directory /depot_tools

RUN curl -sL https://deb.nodesource.com/setup_20.x  | bash -
RUN apt-get -y install nodejs

RUN node -p "process.arch"

RUN mkdir /app

ENV SKIP_DOWNLOAD=true
# ENV CC=clang

RUN npm install

CMD ["sleep","3600"]

The I just run the image, exect in it and npm run build fail

@duvallj
Copy link
Collaborator Author

duvallj commented Apr 14, 2024

@MatthD The issues with building libwebrtc on Arm Linux are unique to the WebRTC codebase, not to this. I don't think there's much I can do about this unfortunately. I'm going to close the PR for now, if you'd like help setting up a cross-compile environment please see #3 and https://github.com/WonderInventions/node-webrtc/blob/d567298049d8e6efd2a4a4d9ff0d6658476adb9f/toolchains/linux-arm64.toolchain . I do realize the docs are a bit lacking in that regard, sorry about that...

@duvallj duvallj merged commit c1be017 into develop Apr 14, 2024
@MatthD
Copy link

MatthD commented Apr 15, 2024

@MatthD The issues with building libwebrtc on Arm Linux are unique to the WebRTC codebase, not to this. I don't think there's much I can do about this unfortunately. I'm doing to close the PR for now, if you'd like help setting up a cross-compile environment please see #3 and https://github.com/WonderInventions/node-webrtc/blob/d567298049d8e6efd2a4a4d9ff0d6658476adb9f/toolchains/linux-arm64.toolchain . I do realize the docs are a bit lacking in that regard, sorry about that...

hey @duvallj thanks for this ! yes did not realize because the build are present for arm64 on 0.7.2.
So do you think you will be able to publish build for them ? Or do you want me to do it ?

@pr0g
Copy link

pr0g commented May 1, 2024

Today I've been trying to build node-webrtc from source on Linux x64, and I'm hitting the same error as mentioned above (using Amazon Linux 2023 and Node 18)

[13/106] Performing configure step for 'project_libwebrtc'
FAILED: external/libwebrtc/stamp/project_libwebrtc-configure /node-webrtc/build-linux-x64/external/libwebrtc/stamp/project_libwebrtc-configure 
cd /node-webrtc/build-linux-x64/external/libwebrtc/build/Release && /usr/bin/cmake -E env BINARY_DIR=/node-webrtc/build-linux-x64/external/libwebrtc/build/Release DEPOT_TOOLS=/node-webrtc/build-linux-x64/external/depot_tools/src "GN_GEN_ARGS=rtc_build_examples=false rtc_use_x11=false rtc_enable_protobuf=false rtc_include_pulse_audio=false rtc_include_tests=false use_lld=false use_custom_libcxx=false rtc_build_tools=false is_debug=false" SOURCE_DIR=/node-webrtc/build-linux-x64/external/libwebrtc/download/src /node-webrtc/scripts/configure-webrtc.sh && /usr/bin/cmake -E touch /node-webrtc/build-linux-x64/external/libwebrtc/stamp/project_libwebrtc-configure

export PATH=$DEPOT_TOOLS:$PATH

cd ${SOURCE_DIR}

# NOTE(mroberts): Running hooks generates this file, but running hooks also
# takes too long in CI; so do this manually.
(cd build/util && python3 lastchange.py -o LASTCHANGE)

gn gen ${BINARY_DIR} "--args=${GN_GEN_ARGS}"
ERROR at //build/config/sysroot.gni:64:7: Assertion failed.
      assert(
      ^-----
Missing sysroot (//build/linux/debian_sid_amd64-sysroot). To fix, run: build/linux/sysroot_scripts/install-sysroot.py --arch=amd64
See //build/config/sysroot.gni:65:11: 
          exec_script("//build/dir_exists.py",
          ^-----------------------------------
This is where it was set.
See //build/config/linux/pkg_config.gni:5:1: whence it was imported.
import("//build/config/sysroot.gni")
^----------------------------------
See //BUILD.gn:15:1: whence it was imported.
import("//build/config/linux/pkg_config.gni")
^-------------------------------------------
ninja: build stopped: subcommand failed.
ERR! OMG Process terminated: 1
/node-webrtc/scripts/build-from-source.js:41
    throw new Error("cmake-js build failed for wrtc");
    ^

When it says:

To fix, run: build/linux/sysroot_scripts/install-sysroot.py --arch=amd64

I'm not quite sure what it's referring to. I potentially could be missing a dependency that's causing this to fail, if someone could give me a pointer I'd be very grateful!

Thanks!

@duvallj
Copy link
Collaborator Author

duvallj commented May 1, 2024

@pr0g that was my bad, I accidentally reverted the change to fix this in a later commit. I just pushed a series of changes that fix this again and also update to M98; ef6b044 is the commit you can apply if you just want to fix this error.

@pr0g
Copy link

pr0g commented May 2, 2024

Hi @duvallj,

Thank you very much for getting back to me, no worries and thanks for sharing the commit to hopefully fix it.

I tried cherry picking ef6b044 to the develop branch and also tried syncing M98, but unfortunately I get a slightly different, but related error:

[21/114] Performing configure step for 'project_libwebrtc'
FAILED: external/libwebrtc/stamp/project_libwebrtc-configure /node-webrtc/build-linux-x64/external/libwebrtc/stamp/project_libwebrtc-configure 
cd /node-webrtc/build-linux-x64/external/libwebrtc/build/Release && /usr/bin/cmake -E env BINARY_DIR=/node-webrtc/build-linux-x64/external/libwebrtc/build/Release DEPOT_TOOLS=/node-webrtc/build-linux-x64/external/depot_tools/src "GN_GEN_ARGS=rtc_build_examples=false rtc_use_x11=false rtc_enable_protobuf=false rtc_include_pulse_audio=false rtc_include_tests=false use_lld=false use_custom_libcxx=false rtc_build_tools=false is_debug=false" SOURCE_DIR=/node-webrtc/build-linux-x64/external/libwebrtc/download/src /node-webrtc/scripts/configure-webrtc.sh && /usr/bin/cmake -E touch /node-webrtc/build-linux-x64/external/libwebrtc/stamp/project_libwebrtc-configure

export PATH=$DEPOT_TOOLS:$PATH

cd ${SOURCE_DIR}

case "$(uname -s)" in
  Linux*)
    if [ "$TARGET_ARCH" == "arm" ]; then
      python3 build/linux/sysroot_scripts/install-sysroot.py --arch=arm
    elif [ "$TARGET_ARCH" == "arm64" ]; then
      python3 build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
    else
      python3 build/linux/sysroot_scripts/install-sysroot.py --arch=amd64
    fi
esac
Installing Debian sid amd64 root image: /node-webrtc/build-linux-x64/external/libwebrtc/download/src/build/linux/debian_sid_amd64-sysroot
Downloading https://commondatastorage.googleapis.com/chrome-linux-sysroot/toolchain/95051d95804a77144986255f534acb920cee375b/debian_sid_amd64_sysroot.tar.xz

# NOTE(mroberts): Running hooks generates this file, but running hooks also
# takes too long in CI; so do this manually.
(cd build/util && python3 lastchange.py -o LASTCHANGE)

gn gen ${BINARY_DIR} "--args=${GN_GEN_ARGS}"
ERROR at //build_overrides/build.gni:39:13: Script returned non-zero exit code.
  _result = exec_script("//build/mac/should_use_hermetic_xcode.py",
            ^----------
Current dir: /node-webrtc/build-linux-x64/external/libwebrtc/build/Release/
Command: python3 /node-webrtc/build-linux-x64/external/libwebrtc/download/src/build/mac/should_use_hermetic_xcode.py linux
Returned 1.
stderr:

Traceback (most recent call last):
  File "/node-webrtc/build-linux-x64/external/libwebrtc/download/src/build/mac/should_use_hermetic_xcode.py", line 28, in <module>
    import mac_toolchain
  File "/node-webrtc/build-linux-x64/external/libwebrtc/download/src/build/mac/../mac_toolchain.py", line 25, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

See //build/toolchain/toolchain.gni:10:1: whence it was imported.
import("//build_overrides/build.gni")
^-----------------------------------
See //build/config/coverage/coverage.gni:5:1: whence it was imported.
import("//build/toolchain/toolchain.gni")
^---------------------------------------
See //build/config/profiling/profiling.gni:6:1: whence it was imported.
import("//build/config/coverage/coverage.gni")
^--------------------------------------------
See //build/config/sanitizers/sanitizers.gni:9:1: whence it was imported.
import("//build/config/profiling/profiling.gni")
^----------------------------------------------
See //BUILD.gn:16:1: whence it was imported.
import("//build/config/sanitizers/sanitizers.gni")
^------------------------------------------------
ninja: build stopped: subcommand failed.
ERR! OMG Process terminated: 1
/node-webrtc/scripts/build-from-source.js:41
    throw new Error("cmake-js build failed for wrtc");
    ^

Error: cmake-js build failed for wrtc
    at main (/node-webrtc/scripts/build-from-source.js:41:11)
    at Object.<anonymous> (/node-webrtc/scripts/build-from-source.js:50:3)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12)
    at node:internal/main/run_main_module:28:49

Node.js v18.20.2
error Command failed with exit code 1.

It looks like it's trying to pull some dependencies relating to macOS/Darwin (I see Xcode mentioned, which might be something?)

I have a bit of an odd setup trying to build this which might be related, I'm on an M1 Mac, but want to build for Linux x64 (for AWS Lambda). I'm using Docker to launch a container, the Dockerfile looks like this:

FROM amazonlinux:2023
RUN dnf update -y
RUN dnf groupinstall "Development Tools" -y
RUN dnf install ninja-build cmake python3 -y
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
ENV NVM_DIR "/root/.nvm"
RUN . $NVM_DIR/nvm.sh && nvm --version
RUN . $NVM_DIR/nvm.sh && nvm install 18
ENV PATH=$NVM_DIR/versions/node/v18.20.2/bin:$PATH
RUN npm install --global yarn
RUN yarn global add patch-package

And I then do:

docker build --platform linux/amd64 -t <image-name> .
docker run -it --platform linux/amd64 -v /path/to/node-webrtc:/node-webrtc <image-name>

I do a tiny bit of local config in the container to setup my Git credentials, but that's it. Would you still expect this issue to be present? I'm wondering maybe there's another dependency I'm missing?

The good news with this is when running yarn install on a package using @roamhq/node-wrtc in my container, it downloads the correct wrtc-linux-x64 folder to node_modules/@roamhq (on my Mac it downloads wrtc-darwin-arm64 which I don't want when packaging the code to push to AWS Lambda), so it's definitely an x64 arch in the docker container.

Thank you very much for your help, please let me know if there's anything else you'd like me to test or if there's any more information I can provide.

@duvallj
Copy link
Collaborator Author

duvallj commented May 2, 2024

  File "/node-webrtc/build-linux-x64/external/libwebrtc/download/src/build/mac/../mac_toolchain.py", line 25, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Sounds like the python installation is missing setuptools. Not quite sure how that happens since it should be using the depot_tools python, but would you humor me by trying pip install setuptools (or from dnf, inside the docker container) and running the installation again?

I am still worried about it trying to do Mac stuff inside a docker container though...

@pr0g
Copy link

pr0g commented May 2, 2024

How did I miss that?! 🤦 Thank you @duvallj 🙌

I'm on it now! Thanks very much for the suggestion, I'll report back when I have an update 👍

@pr0g
Copy link

pr0g commented May 2, 2024

Okay it got way further! 🥳 Which is fantastic news, unfortunately right at the end of building I hit this error:

...
[3718/4200] CC obj/third_party/opus/opus/cwrs.o
[3719/4200] CC obj/third_party/opus/opus/A2NLSF.o
[3720/4200] CC obj/third_party/opus/opus/celt.o
[3721/4200] LINK ./nasm
FAILED: nasm 
python3 "../../download/src/build/toolchain/gcc_link_wrapper.py" --output="./nasm" -- ../../download/src/third_party/llvm-build/Release+Asserts/bin/clang++ -Wl,--fatal-warnings -Wl,--build-id -fPIC -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -fuse-ld=gold -Wl,--threads -Wl,--thread-count=4 -m64 -no-canonical-prefixes -Wl,-O2 -Wl,--gc-sections -rdynamic --sysroot=../../download/src/build/linux/debian_sid_amd64-sysroot -Wl,-z,defs -Wl,--as-needed -pie -Wl,--disable-new-dtags -Werror -o "./nasm" -Wl,--start-group @"./nasm.rsp"  -Wl,--end-group  -latomic -ldl -lpthread -lrt
clang++: error: invalid linker name in argument '-fuse-ld=gold'
[3722/4200] CC obj/third_party/opus/opus/entcode.o
[3723/4200] CC obj/third_party/opus/opus/celt_lpc.o
[3724/4200] CC obj/third_party/opus/opus/bands.o
[3725/4200] CC obj/third_party/opus/opus/celt_decoder.o
[3726/4200] CC obj/third_party/opus/opus/entdec.o
[3727/4200] CC obj/third_party/opus/opus/decode_frame.o
[3728/4200] CC obj/third_party/opus/opus/entenc.o
[3729/4200] CC obj/third_party/opus/opus/decode_indices.o
[3730/4200] CC obj/third_party/opus/opus/decode_parameters.o
[3731/4200] CC obj/third_party/opus/opus/celt_encoder.o
[3732/4200] CC obj/third_party/pffft/pffft/pffft.o
ninja: build stopped: subcommand failed.
ninja: build stopped: subcommand failed.
ERR! OMG Process terminated: 1
/node-webrtc/scripts/build-from-source.js:41
    throw new Error("cmake-js build failed for wrtc");
    ^

Error: cmake-js build failed for wrtc
    at main (/node-webrtc/scripts/build-from-source.js:41:11)
    at Object.<anonymous> (/node-webrtc/scripts/build-from-source.js:50:3)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12)
    at node:internal/main/run_main_module:28:49

Node.js v18.20.2

I think this is probably caused by me not having nasm as a dependency, I'm going to try and add that now to see if I have any luck.

The other issued seemed to be clang++: error: invalid linker name in argument '-fuse-ld=gold', but I'm not sure if that's related 🤔

I'm going to check if I have nasm, and if not install it and give it another try 👍

@pr0g
Copy link

pr0g commented May 2, 2024

Actually I don't think it was nasm in the end, it seems to be the gold linker, not sure what the package name for that is but I'm trying to find if I can install it now

@pr0g
Copy link

pr0g commented May 2, 2024

Okay good and bad news 😅 (mostly good!)

I've been able to get things to build!! 🥳 The one last change I needed to make was to install ld.gold which isn't available by default on Amazon Linux 2023 😞 It's also not available as a package (see this issue - amazonlinux/amazon-linux-2023#330), but some kind soul has provided a script to download and build it from source (see this gist).

After running this script, and then running yarn build again, I finally got to see the output I'd been waiting for!

...
[105/105] Linking CXX shared library Release/wrtc.node
Built wrtc
Done in 122.14s.

🕺

Huge thanks for your help in getting to this stage, I just need to now upload this executable and hope pray Amazon Linux 2023 loads it okay. I'll report back when I've tested that with a bit of luck 🤞

@MatthD
Copy link

MatthD commented May 4, 2024

You have to note that on my side this PR did broke the audio quality, maybe the owner has fixed it if the dev branch !

I did uploaded those artifact on my side for linux/darwin arm64 and x64 for both

https://www.npmjs.com/package/@avahq/wrtc

Thanks to everyone helping on all this node-webrtc ;)

@duvallj
Copy link
Collaborator Author

duvallj commented May 6, 2024

@MatthD I effectively reverted this PR in e1e0d4d and 5d8f59e

@duvallj duvallj deleted the jack/use-builtin-test-audio-module branch September 3, 2024 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RTCAudioSink ondata event never fires
4 participants