Big-endian test failures on IBM Z #4213

hugovk · 2019-11-19T07:29:46Z

What did you do?

Run tests on IBM Z (s390x) on Travis CI, newly available:

https://blog.travis-ci.com/2019-11-12-multi-cpu-architecture-ibm-power-ibm-z

Like this:

master...hugovk:travis-arch

(The other new architectures, arm64 and ppc64le, both passed when skipping the pyqt5 install.)

What did you expect to happen?

Tests pass.

What actually happened?

7 tests fail.

TestFileJpeg2k.test_16bit_monochrome_j2k_like_tiff
TestFileJpeg2k.test_16bit_monochrome_jp2_like_tiff
TestFilePng.test_sanity
TestFileWebpAnimation.test_write_animation_L
TestFileWebpAnimation.test_write_animation_RGB
TestImageGetExtrema.test_extrema
TestImageGetExtrema.test_true_16

=================================== FAILURES ===================================
______________ TestFileJpeg2k.test_16bit_monochrome_j2k_like_tiff ______________
self = <Tests.test_file_jpeg2k.TestFileJpeg2k testMethod=test_16bit_monochrome_j2k_like_tiff>
    def test_16bit_monochrome_j2k_like_tiff(self):
    
        tiff_16bit = Image.open("Tests/images/16bit.cropped.tif")
        j2k = Image.open("Tests/images/16bit.cropped.j2k")
>       self.assert_image_similar(j2k, tiff_16bit, 1e-3)
Tests/test_file_jpeg2k.py:180: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Tests/helper.py:153: in assert_image_similar
    raise e
Tests/helper.py:139: in assert_image_similar
    self.assertGreaterEqual(
E   AssertionError: 0.001 not greater than or equal to 254.937744140625 :  average pixel value difference 254.9377 > epsilon 0.0010
______________ TestFileJpeg2k.test_16bit_monochrome_jp2_like_tiff ______________
self = <Tests.test_file_jpeg2k.TestFileJpeg2k testMethod=test_16bit_monochrome_jp2_like_tiff>
    def test_16bit_monochrome_jp2_like_tiff(self):
    
        tiff_16bit = Image.open("Tests/images/16bit.cropped.tif")
        jp2 = Image.open("Tests/images/16bit.cropped.jp2")
>       self.assert_image_similar(jp2, tiff_16bit, 1e-3)
Tests/test_file_jpeg2k.py:174: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Tests/helper.py:153: in assert_image_similar
    raise e
Tests/helper.py:139: in assert_image_similar
    self.assertGreaterEqual(
E   AssertionError: 0.001 not greater than or equal to 254.937744140625 :  average pixel value difference 254.9377 > epsilon 0.0010
___________________________ TestFilePng.test_sanity ____________________________
self = <Tests.test_file_png.TestFilePng testMethod=test_sanity>
    def test_sanity(self):
    
        # internal version number
        self.assertRegex(Image.core.zlib_version, r"\d+\.\d+\.\d+(\.\d+)?$")
    
        test_file = self.tempfile("temp.png")
    
        hopper("RGB").save(test_file)
    
        with Image.open(test_file) as im:
            im.load()
            self.assertEqual(im.mode, "RGB")
            self.assertEqual(im.size, (128, 128))
            self.assertEqual(im.format, "PNG")
            self.assertEqual(im.get_format_mimetype(), "image/png")
    
        for mode in ["1", "L", "P", "RGB", "I", "I;16"]:
            im = hopper(mode)
            im.save(test_file)
            reloaded = Image.open(test_file)
            if mode == "I;16":
                reloaded = reloaded.convert(mode)
>           self.assert_image_equal(reloaded, im)
Tests/test_file_png.py:97: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Tests/helper.py:113: in assert_image_equal
    self.fail(msg or "got different content")
E   AssertionError: got different content
----------------------------- Captured stdout call -----------------------------
STREAM b'IHDR' 16 13
STREAM b'IDAT' 41 30416
STREAM b'IHDR' 16 13
STREAM b'IDAT' 41 1867
STREAM b'IHDR' 16 13
STREAM b'IDAT' 41 10582
STREAM b'IHDR' 16 13
STREAM b'PLTE' 41 768
STREAM b'IDAT' 821 8528
STREAM b'IHDR' 16 13
STREAM b'IDAT' 41 30416
STREAM b'IHDR' 16 13
STREAM b'IDAT' 41 13537
STREAM b'IHDR' 16 13
STREAM b'IDAT' 41 13539
------------------------------ Captured log call -------------------------------
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IHDR' 16 13
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IDAT' 41 30416
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IHDR' 16 13
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IDAT' 41 1867
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IHDR' 16 13
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IDAT' 41 10582
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IHDR' 16 13
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'PLTE' 41 768
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IDAT' 821 8528
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IHDR' 16 13
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IDAT' 41 30416
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IHDR' 16 13
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IDAT' 41 13537
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IHDR' 16 13
DEBUG    PIL.PngImagePlugin:PngImagePlugin.py:142 STREAM b'IDAT' 41 13539
_________________ TestFileWebpAnimation.test_write_animation_L _________________
self = <Tests.test_file_webp_animated.TestFileWebpAnimation testMethod=test_write_animation_L>
    def test_write_animation_L(self):
        """
        Convert an animated GIF to animated WebP, then compare the
        frame count, and first and last frames to ensure they're
        visually similar.
        """
    
        with Image.open("Tests/images/iss634.gif") as orig:
            self.assertGreater(orig.n_frames, 1)
    
            temp_file = self.tempfile("temp.webp")
            orig.save(temp_file, save_all=True)
            im = Image.open(temp_file)
            self.assertEqual(im.n_frames, orig.n_frames)
    
            # Compare first and last frames to the original animated GIF
            orig.load()
            im.load()
            self.assert_image_similar(im, orig.convert("RGBA"), 25.0)
            orig.seek(orig.n_frames - 1)
            im.seek(im.n_frames - 1)
            orig.load()
            im.load()
>           self.assert_image_similar(im, orig.convert("RGBA"), 25.0)
Tests/test_file_webp_animated.py:62: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Tests/helper.py:153: in assert_image_similar
    raise e
Tests/helper.py:139: in assert_image_similar
    self.assertGreaterEqual(
E   AssertionError: 25.0 not greater than or equal to 57.66992086630571 :  average pixel value difference 57.6699 > epsilon 25.0000
________________ TestFileWebpAnimation.test_write_animation_RGB ________________
self = <Tests.test_file_webp_animated.TestFileWebpAnimation testMethod=test_write_animation_RGB>
    def test_write_animation_RGB(self):
        """
        Write an animated WebP from RGB frames, and ensure the frames
        are visually similar to the originals.
        """
    
        def check(temp_file):
            im = Image.open(temp_file)
            self.assertEqual(im.n_frames, 2)
    
            # Compare first frame to original
            im.load()
            self.assert_image_equal(im, frame1.convert("RGBA"))
    
            # Compare second frame to original
            im.seek(1)
            im.load()
            self.assert_image_equal(im, frame2.convert("RGBA"))
    
        frame1 = Image.open("Tests/images/anim_frame1.webp")
        frame2 = Image.open("Tests/images/anim_frame2.webp")
    
        temp_file1 = self.tempfile("temp.webp")
        frame1.copy().save(
            temp_file1, save_all=True, append_images=[frame2], lossless=True
        )
>       check(temp_file1)
Tests/test_file_webp_animated.py:90: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Tests/test_file_webp_animated.py:81: in check
    self.assert_image_equal(im, frame2.convert("RGBA"))
Tests/helper.py:113: in assert_image_equal
    self.fail(msg or "got different content")
E   AssertionError: got different content
_______________________ TestImageGetExtrema.test_extrema _______________________
self = <Tests.test_image_getextrema.TestImageGetExtrema testMethod=test_extrema>
    def test_extrema(self):
        def extrema(mode):
            return hopper(mode).getextrema()
    
        self.assertEqual(extrema("1"), (0, 255))
        self.assertEqual(extrema("L"), (0, 255))
        self.assertEqual(extrema("I"), (0, 255))
        self.assertEqual(extrema("F"), (0, 255))
        self.assertEqual(extrema("P"), (0, 225))  # fixed palette
        self.assertEqual(extrema("RGB"), ((0, 255), (0, 255), (0, 255)))
        self.assertEqual(extrema("RGBA"), ((0, 255), (0, 255), (0, 255), (255, 255)))
        self.assertEqual(extrema("CMYK"), ((0, 255), (0, 255), (0, 255), (0, 0)))
>       self.assertEqual(extrema("I;16"), (0, 255))
E       AssertionError: Tuples differ: (0, 65280) != (0, 255)
E       
E       First differing element 1:
E       65280
E       255
E       
E       - (0, 65280)
E       + (0, 255)
Tests/test_image_getextrema.py:19: AssertionError
_______________________ TestImageGetExtrema.test_true_16 _______________________
self = <Tests.test_image_getextrema.TestImageGetExtrema testMethod=test_true_16>
    def test_true_16(self):
        im = Image.open("Tests/images/16_bit_noise.tif")
        self.assertEqual(im.mode, "I;16")
        extrema = im.getextrema()
>       self.assertEqual(extrema, (106, 285))
E       AssertionError: Tuples differ: (513, 64512) != (106, 285)
E       
E       First differing element 0:
E       513
E       106
E       
E       - (513, 64512)
E       + (106, 285)
Tests/test_image_getextrema.py:25: AssertionError

https://travis-ci.org/hugovk/Pillow/jobs/613630151

What are your OS, Python and Pillow versions?

OS: Linux
Python: 3.5-3.8
Pillow: master

It seems that the Travis builds in #4336 were missing LibTiff. I added it while trying to move these jobs to GHA and found at least one new failure: test_file_libtiff.py::TestFileLibTiff::test_strip_ycbcr_jpeg_2x2_sampling. I will edit this when I know the complete list, but the tests take forever running on QEMU.

Edit: The full list is:

FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_strip_ycbcr_jpeg_2x2_sampling
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_strip_ycbcr_jpeg_1x1_sampling
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_tiled_ycbcr_jpeg_1x1_sampling
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_tiled_ycbcr_jpeg_2x2_sampling
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_old_style_jpeg

Edit2: Fixed by #5364.

radarhere · 2021-12-22T20:21:18Z

Looks like the webp failures might actually be a problem with webp itself - I created https://bugs.chromium.org/p/webp/issues/detail?id=548#c3 and it has been accepted.

radarhere · 2021-12-28T20:09:51Z

#4213 (comment) was fixed by #5364
#5901 fixed TestFileJpeg2k.test_16bit_monochrome_j2k_like_tiff, TestFileJpeg2k.test_16bit_monochrome_jp2_like_tiff and TestFilePng.test_sanity.
#4457 fixed TestImageGetExtrema.test_extrema and TestImageGetExtrema.test_true_16

radarhere · 2022-01-23T20:32:34Z

libwebp 1.2.2 has been released, fixing the webp failures. I've created PR #5984 to remove the xfails and resolve this.

hugovk added the Bug Any unexpected behavior, until confirmed feature. label Nov 19, 2019

hugovk added the Big-endian Big-endian processors label Dec 18, 2019

hugovk mentioned this issue Jan 5, 2020

Test on new Travis CPUs #4336

Merged

radarhere mentioned this issue Feb 29, 2020

Fixed endian handling for I;16 getextrema #4457

Merged

This was referenced Mar 14, 2020

test_file_jpeg2k.py::TestFileJpeg2k::test_16bit_monchrome_j2k_like_tiff FAILED on ppc64 #3290

Closed

test_file_webp_animated.py::TestFileWebpAnimation::test_write_animation_L FAILED on ppc64 #3291

Closed

nulano mentioned this issue Nov 3, 2020

The new pricing model for Travis CI #5028

Closed

5 tasks

This was referenced Dec 11, 2020

Move non-x86 tests to GHA #5088

Merged

Mark new test failures on non-x86 as xfail #5091

Merged

This was referenced Dec 21, 2021

Fixed I;16 conversion on big endian #5900

Closed

Improved I;16 operations on big endian #5901

Merged

This was referenced Jan 15, 2022

Install latest libwebp on s390x python-pillow/docker-images#128

Merged

Test suite fails 16 tests on hppa #3927

Closed

radarhere mentioned this issue Jan 23, 2022

libwebp 1.2.2 fixed endian bugs #5984

Merged

hugovk closed this as completed in #5984 Mar 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Big-endian test failures on IBM Z #4213

Big-endian test failures on IBM Z #4213

hugovk commented Nov 19, 2019 •

edited by radarhere

Loading

radarhere commented Dec 30, 2019

hugovk commented Dec 30, 2019

nulano commented Nov 3, 2020 •

edited

Loading

radarhere commented Dec 22, 2021

radarhere commented Dec 28, 2021

radarhere commented Jan 23, 2022

Big-endian test failures on IBM Z #4213

Big-endian test failures on IBM Z #4213

Comments

hugovk commented Nov 19, 2019 • edited by radarhere Loading

What did you do?

What did you expect to happen?

What actually happened?

What are your OS, Python and Pillow versions?

Related

radarhere commented Dec 30, 2019

hugovk commented Dec 30, 2019

nulano commented Nov 3, 2020 • edited Loading

radarhere commented Dec 22, 2021

radarhere commented Dec 28, 2021

radarhere commented Jan 23, 2022

hugovk commented Nov 19, 2019 •

edited by radarhere

Loading

nulano commented Nov 3, 2020 •

edited

Loading