zlib v1.2.12 causes java.util.zip.ZipException

[rx80]

The one way i can reliably reproduce this is as follows:

npm install google-closure-compiler
npm exec google-closure-compiler

With previous (working) version of zlib-1.2.11 it runs fine.

Upgrading to zlib-1.2.12 causes this error:

java.util.zip.ZipException: invalid entry CRC (expected 0x4e1f14a4 but got 0xb1e0eb5b)
        at java.util.zip.ZipInputStream.readEnd(ZipInputStream.java:410)
        at java.util.zip.ZipInputStream.read(ZipInputStream.java:199)
        at java.util.zip.ZipInputStream.closeEntry(ZipInputStream.java:143)
        at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:121)
        at com.google.javascript.jscomp.AbstractCommandLineRunner.getBuiltinExterns(AbstractCommandLineRunner.java:481)
        at com.google.javascript.jscomp.CommandLineRunner.createExterns(CommandLineRunner.java:2083)
        at com.google.javascript.jscomp.AbstractCommandLineRunner.doRun(AbstractCommandLineRunner.java:1168)
        at com.google.javascript.jscomp.AbstractCommandLineRunner.run(AbstractCommandLineRunner.java:532)
        at com.google.javascript.jscomp.CommandLineRunner.main(CommandLineRunner.java:2241)

Downgrading to 1.2.11 fixes it.

This was compiled on Gentoo with GCC 11.2.1, no special CFLAGS.

Gentoo bug: https://bugs.gentoo.org/836370

[jpalus]

Experiencing the same issue in different Linux distribution but sadly it seems to be random or at least so far I was not able to create deterministic reproducer.

[jpalus]

So far I was able to trigger error in jar:

java.io.IOException: Push back buffer is full
	at java.base/java.io.PushbackInputStream.unread(PushbackInputStream.java:229)
	at java.base/java.util.zip.ZipInputStream.readEnd(ZipInputStream.java:361)
	at java.base/java.util.zip.ZipInputStream.read(ZipInputStream.java:199)
	at java.base/java.util.zip.ZipInputStream.closeEntry(ZipInputStream.java:143)
	at jdk.jartool/sun.tools.jar.Main.extractFile(Main.java:1456)
	at jdk.jartool/sun.tools.jar.Main.extract(Main.java:1363)
	at jdk.jartool/sun.tools.jar.Main.run(Main.java:409)
	at jdk.jartool/sun.tools.jar.Main.main(Main.java:1680)

Bisected to f8719f5:

commit f8719f5ae5acdc31d3794ddfea8ac963359de41e (HEAD)
Author: Mark Adler <madler@alumni.caltech.edu>
Date:   Tue Dec 11 01:11:38 2018 -0800

    Speed up software CRC-32 computation by a factor of 1.5 to 3.

The problem seems to be in zip creation -- jar with this commit produces zip that cannot be extracted by either 1.2.11 or 1.2.12.

Attaching broken sample (created on linux x86-64).
unoloader.zip

[jpalus]

Note that above issue is only reproducible on x86-64. Also tried x86 32-bit, arm 32-bit and arm 64-bit but all seem to work fine.

[jpalus]

unzip on attached zip reports bad CRC:

$ unzip unoloader.zip 
Archive:  /home/users/jan/unoloader.zip
   creating: META-INF/
  inflating: META-INF/MANIFEST.MF     bad CRC 53958e9f  (should be 48958ad5)
  inflating: module-info.class       
   creating: com/
   creating: com/sun/
   creating: com/sun/star/
   creating: com/sun/star/lib/
   creating: com/sun/star/lib/unoloader/
  inflating: com/sun/star/lib/unoloader/UnoLoader$1.class  
  inflating: com/sun/star/lib/unoloader/UnoLoader.class  
  inflating: com/sun/star/lib/unoloader/UnoClassLoader.class  

[jpalus]

Issue appears to be related to change in treatment of higher order bytes of crc parameter to crc32() function. Given following code:

#include <stdio.h>
#include <zlib.h>

int main() {
	printf("%lx\n", crc32(0xe3d73c3c, "a", 1));
	printf("%lx\n", crc32(0xffffffffe3d73c3c, "a", 1));
	return 0;
}

zlib 1.2.11 prints same result twice:

c73b15f8
c73b15f8

while zlib 1.2.12 gives two different values:

c73b15f8
ffffff383b15f8

This affects java since it uses smaller unsigned int that affects higher order bytes.

[jpalus]

As a workaround applied:

diff --git a/crc32.c b/crc32.c
index a1bdce5..748b7ba 100644
--- a/crc32.c
+++ b/crc32.c
@@ -19,6 +19,7 @@
   MAKECRCH can be #defined to write out crc32.h. A main() routine is also
   produced, so that this one source file can be compiled to an executable.
  */
+#include <limits.h>
 
 #ifdef MAKECRCH
 #  include <stdio.h>
@@ -1065,7 +1066,12 @@ unsigned long ZEXPORT crc32(crc, buf, len)
     const unsigned char FAR *buf;
     uInt len;
 {
+/* if sizeof(unsigned long) > 4 */
+#if ULONG_MAX > 0xffffffffUL
+    return crc32_z(crc & 0xffffffffUL, buf, len);
+#else
     return crc32_z(crc, buf, len);
+#endif
 }
 
 /* ========================================================================= */

[ydroneaud]

is crc32() returning sign extended CRC32 values ?

[madler]

This does not appear to be a CRC output problem, but rather a CRC input problem. It looks like if crc32() is being given an incorrect initial value, it now gives an incorrect result. Whereas, before, the bug in the application was hidden by zlib. I will look into this immediately, since the previous behavior should be retained, despite the bug in the application.

[madler]

Let me know if this commit remedies the issue: ec3df00 .

[jpalus]

@madler confirmed ec3df00 fixes jar.

[madler]

Thank you!