Build failure: python311Packages.cchardet

[davidak]

Steps To Reproduce

Steps to reproduce the behavior:

1. build X

[davidak@gaming:~/code/nixpkgs]$ nix-shell -I . -p python311Packages.cchardet

Build log

this derivation will be built:
  /nix/store/9gykg3l2pbfs6fdbmcs01bf06iyqwj5q-python3.11-cchardet-2.1.7.drv
building '/nix/store/9gykg3l2pbfs6fdbmcs01bf06iyqwj5q-python3.11-cchardet-2.1.7.drv'...
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing setuptools-check-hook
unpacking sources
unpacking source archive /nix/store/4jf9q7rps4nnpjwzizg9n722gv1i52mp-cchardet-2.1.7.tar.gz
source root is cchardet-2.1.7
setting SOURCE_DATE_EPOCH to timestamp 1603786655 of file cchardet-2.1.7/setup.cfg
patching sources
configuring
no configure script, doing nothing
building
Executing setuptoolsBuildPhase
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/cchardet
copying src/cchardet/version.py -> build/lib.linux-x86_64-cpython-311/cchardet
copying src/cchardet/__init__.py -> build/lib.linux-x86_64-cpython-311/cchardet
running build_ext
building 'cchardet._cchardet' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/src
creating build/temp.linux-x86_64-cpython-311/src/cchardet
creating build/temp.linux-x86_64-cpython-311/src/ext
creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet
creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src
creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src/LangModels
g++ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/64vcf3x48sjfckvwzbdsr3a9yjf0jcbq-libxcrypt-4.4.30/include -fPIC -Isrc/ext/uchardet/src -I/nix/store/536aws8kkljng4ggdakk8i1nfdqb4f4x-python3-3.11.3/include/python3.11 -c src/cchardet/_cchardet.cpp -o build/temp.linux-x86_64-cpython-311/src/cchardet/_cchardet.o
src/cchardet/_cchardet.cpp:196:12: fatal error: longintrepr.h: No such file or directory
  196 |   #include "longintrepr.h"
      |            ^~~~~~~~~~~~~~~
compilation terminated.
error: command '/nix/store/y372k7srz9jm7v9sv2cs84g3crz27z1s-gcc-wrapper-11.3.0/bin/g++' failed with exit code 1
error: builder for '/nix/store/9gykg3l2pbfs6fdbmcs01bf06iyqwj5q-python3.11-cchardet-2.1.7.drv' failed with exit code 1;
       last 15 log lines:
       > running build_ext
       > building 'cchardet._cchardet' extension
       > creating build/temp.linux-x86_64-cpython-311
       > creating build/temp.linux-x86_64-cpython-311/src
       > creating build/temp.linux-x86_64-cpython-311/src/cchardet
       > creating build/temp.linux-x86_64-cpython-311/src/ext
       > creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet
       > creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src
       > creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src/LangModels
       > g++ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/64vcf3x48sjfckvwzbdsr3a9yjf0jcbq-libxcrypt-4.4.30/include -fPIC -Isrc/ext/uchardet/src -I/nix/store/536aws8kkljng4ggdakk8i1nfdqb4f4x-python3-3.11.3/include/python3.11 -c src/cchardet/_cchardet.cpp -o build/temp.linux-x86_64-cpython-311/src/cchardet/_cchardet.o
       > src/cchardet/_cchardet.cpp:196:12: fatal error: longintrepr.h: No such file or directory
       >   196 |   #include "longintrepr.h"
       >       |            ^~~~~~~~~~~~~~~
       > compilation terminated.
       > error: command '/nix/store/y372k7srz9jm7v9sv2cs84g3crz27z1s-gcc-wrapper-11.3.0/bin/g++' failed with exit code 1
       For full logs, run 'nix log /nix/store/9gykg3l2pbfs6fdbmcs01bf06iyqwj5q-python3.11-cchardet-2.1.7.drv'.

Additional context

Needed for Nikola and others.

Same issue with python310.

Notify maintainers

@dotlambda @ivan

Metadata

Built from nixpkgs master (c873717)

• system: "x86_64-linux"
• host os: Linux 6.3.5, NixOS, 22.11 (Raccoon), 22.11.4588.93fddcf640c
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.11.1
• channels(root): "home-manager-22.11.tar.gz, nixos-22.11, nixos-hardware, nixos-unstable"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

[kirillrdy]

@davidak greetings ! builds for me

git checkout c873717c6f81082f7cf8302017971520495a75fe
Note: switching to 'c873717c6f81082f7cf8302017971520495a75fe'.

nix-build -A python311Packages.cchardet
/nix/store/0klkk49c1vk8v6lk2jw0xx7ykz12m2ak-python3.11-faust-cchardet-2.1.18

 nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.3.9, NixOS, 23.11 (Tapir), 23.11.20230622.c44a3b0`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.15.1`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

I've noticed that hash is different, maybe something in your overlays ?

can you try
nix build github:nixos/nixpkgs?rev=c873717c6f81082f7cf8302017971520495a75fe#python311Packages.cchardet

[tjni]

Here's a rundown of the issue:

At some point, changes to CPython in python/cpython#79315 must have made their way into Python. This broke any code generated by Cython until around version 0.29.25 (it was fixed in cython/cython#4428). Unfortunately, cchardet 2.1.7 was using Cython 0.29.21.

The cchardet project was not actively maintained at that time, so a fork of it was created and the Cython version was updated to 0.29.30 in faust-streaming/cChardet@e8426f8 and released as faust-cchardet. This was pulled into nixpkgs in d8fe683 and became a drop-in replacement for cchardet in 1bd4b71 and 2d53019.

One way to get this to just work is to pick up all these changes by updating to 23.05. I don't know whether a backport would be possible.

[kirillrdy]

@tjni thank you very much for your summary, I am sure we can sort out a backport if it's necessary.

I'll just add one more bit of info, to clear confusion on my side

Original report claims that Built from nixpkgs master (https://github.com/NixOS/nixpkgs/commit/c873717c6f81082f7cf8302017971520495a75fe)

but from build logs ( in original report )

this derivation will be built:
  /nix/store/9gykg3l2pbfs6fdbmcs01bf06iyqwj5q-python3.11-cchardet-2.1.7.drv

but that commit already contains cchardet-2.1.8

[tjni]

@davidak do you still encounter this failure? I want to do some maintenance on old issues.