Ubuntu 18.04: <command-line>: error: expected identifier or '(' before numeric constant

[KuiWei004]

Hello, everyone!

When I run "bazel test -c opt ...", I got an error below

e418145f18940c8c0db59ce8308014ec/external/org_gnu_m4/BUILD.bazel:53:10: C++ compilation of rule '@org_gnu_m4//:m4' failed (Exit 1): gcc failed: error executing command

/usr/local/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections -MD -MF bazel-out/host/bin/external/org_gnu_m4/objs/m4/binary-io.d '-frandom-seed=bazel-out/host/bin/external/org_gnu_m4/objs/m4/binary-io.o' -D_IO_ftrylockfile '-D_IO_IN_BACKUP=0x100' -D_GNU_SOURCE '-DO_BINARY=0' -iquote external/org_gnu_m4 -iquote bazel-out/host/bin/external/org_gnu_m4 -iquote external/bazel_tools -iquote bazel-out/host/bin/external/bazel_tools -isystem external/org_gnu_m4/lib -isystem bazel-out/host/bin/external/org_gnu_m4/lib -isystem external/org_gnu_m4/build-aux/snippet -isystem bazel-out/host/bin/external/org_gnu_m4/build-aux/snippet -g0 -w -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/org_gnu_m4/lib/binary-io.c -o bazel-out/host/bin/external/org_gnu_m4/_objs/m4/binary-io.o)

: error: expected identifier or '(' before numeric constant.

I appreciate your help！

[cdleary]

Hi KuiWei0004, can you note the output of gcc --version and your distro cat /etc/issue? We require a sufficiently new GCC since we make use of a bunch of C++17 features and such. Ubuntu 20.04 (which we use for CI) is 9.3.0. We include that Dockerfile (noted in the README) so it's easy to get started with a known working build environment. Thanks!

[KuiWei004]

Thanks, cdleary, my ubuntu version is 18.04 and gcc version is 9.3.0. It is noted that the gcc in my environment is what I update from gcc-8.2.0.

[KuiWei004]

It seemes that this problem is caused by ubuntu 18.04, but I do not find the detail reason.
I have tried xls test on three machines whose OS is ubuntu 18.04, and got the same error mentioned above!

[cdleary]

Thanks for confirming it's an environment issue KuiWei004, I'll see if I can make an 18.04 dockerfile today and see how many issues there are, but for CI testing we're using 20.04, so running/developing out of a docker container could be a good option potentially, 18.04 support will be best-effort.

[KuiWei004]

Thanks for following this issue! It should be noted that the original gcc on ubuntu 18.04 is gcc-7, and this version is old for xls, SO I have updated gcc to gcc-9 by referring to https://linuxize.com/post/how-to-install-gcc-compiler-on-ubuntu-18-04/，then I get <command-line>: error: expected identifier or '(' before numeric constant: . I hope this can help you reproduce this problem.

[cdleary]

Ok, confirmed I was able to repro this in a 18.04 docker sequence, this was the approximate dockerfile:

# NOTE: Ubuntu 18.04 environment support is best-effort only (and generally PRs
# are accepted for build fixes that do not regress 20.04).

FROM ubuntu:18.04

# LABEL about the custom image
LABEL version="0.1"
LABEL description="Docker Image for Building/Testing XLS on Ubuntu 18.04"

# Update package info
RUN apt-get update -y

# Install Bazel
RUN apt-get install -y curl gnupg && \
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg && \
mv bazel.gpg /etc/apt/trusted.gpg.d/ && \
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list && \
apt-get update -y && apt-get install -y bazel

# Install dependencies
RUN apt-get -y install python3 python3-distutils python3-dev libtinfo5

# Install development tools
RUN apt-get install -y git vim

# Get a more modern toolchain (GCC 9)
RUN apt-get install -y software-properties-common
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test
RUN apt-get update
RUN apt-get install -y gcc-9 g++9
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 90 --slave /usr/bin/g++ g++ /usr/bin/g++-9 --slave /usr/bin/gcov gcov /usr/bin/gcov-9

RUN useradd -m xls-developer
USER xls-developer

# Create "python" binary in PATH for Bazel Python environment.
# There is no python3-is-python as there is in 20.04 AFAICT.
RUN mkdir -p ~/opt/bin && \
ln -s $(which python3) ~/opt/bin/python

ENV PATH /home/xls-developer/opt/bin:$PATH

# Clone the project
WORKDIR /home/xls-developer/
RUN git clone https://github.com/google/xls.git

WORKDIR /home/xls-developer/xls

# Build everything (opt)
RUN bazel build -c opt ...

# Test everything (opt)
RUN bazel test -c opt ...

Failing on this command:

(cd /home/xls-developer/.cache/bazel/_bazel_xls-developer/970c5c2433bb6038ab152477a024c421/sandbox/processwrapper-sandbox/36/execroot/com_google_xls && \
  exec env - \
    BAZEL_CXXOPTS='-std=c++17' \
    PATH=/home/xls-developer/opt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/bin/python3 \
    TMPDIR=/tmp \
  /home/xls-developer/.cache/bazel/_bazel_xls-developer/install/f1f06b37c54db45d54451bf6dea565c4/process-wrapper '--timeout=0' '--kill_delay=15' /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections -MD -MF bazel-out/k8-opt/bin/external/org_gnu_m4/_objs/m4/ftello.d '-frandom-seed=bazel-out/k8-opt/bin/external/org_gnu_m4/_objs/m4/ftello.o' -D_IO_ftrylockfile '-D_IO_IN_BACKUP=0x100' -D_GNU_SOURCE '-DO_BINARY=0' -iquote external/org_gnu_m4 -iquote bazel-out/k8-opt/bin/external/org_gnu_m4 -iquote external/bazel_tools -iquote bazel-out/k8-opt/bin/external/bazel_tools -isystem external/org_gnu_m4/lib -isystem bazel-out/k8-opt/bin/external/org_gnu_m4/lib -isystem external/org_gnu_m4/build-aux/snippet -isystem bazel-out/k8-opt/bin/external/org_gnu_m4/build-aux/snippet -Wno-sign-compare -Wno-comment -w -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/org_gnu_m4/lib/ftello.c -o bazel-out/k8-opt/bin/external/org_gnu_m4/_objs/m4/ftello.o)

Identified by bazel build -c opt -s --verbose_failures @org_gnu_m4//:m4

This diff seems to get us past this particular error, something this environment dislikes about the _IO_ftrylockfile define I haven't dug into yet:

diff --git a/dependency_support/org_gnu_m4/bundled.BUILD.bazel b/dependency_support/org_gnu_m4/bundled.BUILD.bazel
index e9690cd..2944b99 100644
--- a/dependency_support/org_gnu_m4/bundled.BUILD.bazel
+++ b/dependency_support/org_gnu_m4/bundled.BUILD.bazel
@@ -91,7 +91,7 @@ cc_binary(
     ],
     output_licenses = ["unencumbered"],
     defines = [
-        "_IO_ftrylockfile",
+        #"_IO_ftrylockfile",
         # Glibc 2.28 made _IO_IN_BACKUP private.  For now, work around this
         # problem by defining it ourselves.  FIXME: Do not rely on glibc
         # internals.

Build at HEAD temporarily has another issue due to LLVM version skew, once we fix that we can see if this fully resolves the issue.

[KuiWei004]

I noticed that the glibc version of ubuntu 18.04 is 2.27. Can this be the reason for the problem?

[cdleary]

@KuiWei004 that could be part of it, I'll see if it's easy to dig into the ftrylockfile difference.

Next wrinkle I hit was that python 3.6.9 doesn't have dataclasses in the stdlib, had to add a pip requirement for the "polyfill" library. Seeing how far we get with that added.

(These little things usually add to up why it's hard to support too many different environments simultaneously except in a best effort fashion, bah. :-)

[cdleary]

Ok with this patch and that dockerfile it passes all tests:

diff --git a/dependency_support/org_gnu_m4/bundled.BUILD.bazel b/dependency_support/org_gnu_m4/bundled.BUILD.bazel
index e9690cd..2944b99 100644
--- a/dependency_support/org_gnu_m4/bundled.BUILD.bazel
+++ b/dependency_support/org_gnu_m4/bundled.BUILD.bazel
@@ -91,7 +91,7 @@ cc_binary(
     ],
     output_licenses = ["unencumbered"],
     defines = [
-        "_IO_ftrylockfile",
+        #"_IO_ftrylockfile",
         # Glibc 2.28 made _IO_IN_BACKUP private.  For now, work around this
         # problem by defining it ourselves.  FIXME: Do not rely on glibc
         # internals.
diff --git a/dependency_support/pip_requirements.txt b/dependency_support/pip_requirements.txt
index ec83a99..c3bbd40 100644
--- a/dependency_support/pip_requirements.txt
+++ b/dependency_support/pip_requirements.txt
@@ -11,3 +11,4 @@ termcolor==1.1.0
 psutil==5.7.0
 portpicker==1.3.1
 pyfakefs==4.0.2
+dataclasses==0.7
diff --git a/xls/dslx/BUILD b/xls/dslx/BUILD
index 1ca6894..35e598c 100644
--- a/xls/dslx/BUILD
+++ b/xls/dslx/BUILD
@@ -267,6 +267,7 @@ py_library(
         "//xls/dslx/python:cpp_concrete_type",
         "//xls/dslx/python:cpp_type_info",
         "@com_google_absl_py//absl/logging",
+       requirement("dataclasses"),
     ],
 )

Will see if I can reconcile those deltas with what 20.04 wants.

[KuiWei004]

I have tried your solution on ubuntu 18.04, it works well !

[cdleary]

I believe could be fixed by 195ed10 -- will make a nightly Ubuntu-18.04 action so we can track breakages (but all still best effort since it's not part of CI).

[cdleary]

We have an Ubuntu 18.04 nightly action: https://github.com/google/xls/actions/workflows/nightly-ubuntu-18.04.yml Closing.