OpenJPEG failing tests on big endian #1204

nijel · 2015-04-22T09:05:50Z

Looks like there are quite some issues within the code when executed on big endian machine, see testsuite failures:

======================================================================
ERROR: TestFileIco.test_save_to_bytes
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_ico.py", line 27, in test_save_to_bytes
    reloaded = Image.open(output)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8992cb90>

======================================================================
ERROR: TestFileJpeg2k.test_16bit_j2k_roundtrips
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 157, in test_16bit_j2k_roundtrips
    im = self.roundtrip(j2k)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff896f2050>

======================================================================
ERROR: TestFileJpeg2k.test_16bit_jp2_roundtrips
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 163, in test_16bit_jp2_roundtrips
    im = self.roundtrip(jp2)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff896f2050>

======================================================================
ERROR: TestFileJpeg2k.test_irreversible_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 83, in test_irreversible_rt
    im = self.roundtrip(test_card, irreversible=True, quality_layers=[20])
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff896f2050>

======================================================================
ERROR: TestFileJpeg2k.test_layers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 107, in test_layers
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff896f2050>

======================================================================
ERROR: TestFileJpeg2k.test_lossless_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 65, in test_lossless_rt
    im = self.roundtrip(test_card)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8afe2770>

======================================================================
ERROR: TestFileJpeg2k.test_lossy_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 69, in test_lossy_rt
    im = self.roundtrip(test_card, quality_layers=[20])
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8afe2770>

======================================================================
ERROR: TestFileJpeg2k.test_prog_qual_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 88, in test_prog_qual_rt
    test_card, quality_layers=[60, 40, 20], progression='LRCP')
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8afe2770>

======================================================================
ERROR: TestFileJpeg2k.test_prog_res_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 92, in test_prog_res_rt
    im = self.roundtrip(test_card, num_resolutions=8, progression='RLCP')
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8afe2770>

======================================================================
ERROR: TestFileJpeg2k.test_tiled_offset_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 79, in test_tiled_offset_rt
    tile_offset=(0, 0), offset=(32, 32))
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8992cb90>

======================================================================
ERROR: TestFileJpeg2k.test_tiled_rt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 73, in test_tiled_rt
    im = self.roundtrip(test_card, tile_size=(128, 128))
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 27, in roundtrip
    im = Image.open(out)
  File "/home/abuild/rpmbuild/BUILDROOT/python-Pillow-2.8.1-37.1.ppc64/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2290, in open
    % (filename if filename else fp))
IOError: cannot identify image file <_io.BytesIO object at 0x3fff8992cb90>

======================================================================
FAIL: TestFileJpeg2k.test_16bit_monchrome_j2k_like_tiff
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 152, in test_16bit_monchrome_j2k_like_tiff
    self.assert_image_similar(j2k, tiff_16bit, 1e-3)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/helper.py", line 105, in assert_image_similar
    ave_diff, epsilon))
AssertionError:  average pixel value difference 250.8379 > epsilon 0.0010

======================================================================
FAIL: TestFileJpeg2k.test_16bit_monchrome_jp2_like_tiff
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/test_file_jpeg2k.py", line 146, in test_16bit_monchrome_jp2_like_tiff
    self.assert_image_similar(jp2, tiff_16bit, 1e-3)
  File "/home/abuild/rpmbuild/BUILD/Pillow-2.8.1/Tests/helper.py", line 105, in assert_image_similar
    ave_diff, epsilon))
AssertionError:  average pixel value difference 250.8379 > epsilon 0.0010

Full build log can be found here:

https://build.opensuse.org/package/live_build_log/devel:languages:python/python-Pillow/openSUSE_Tumbleweed/ppc64

The text was updated successfully, but these errors were encountered:

wiredfool · 2015-04-22T15:01:33Z

I've been testing on big-endian (ubuntu 12.04 PPC), and Jpeg2k has always had issues. I think there's something with the underlying library on that version. What version is is suse testing against?

nijel · 2015-04-22T17:28:48Z

It was openjpeg2-devel-2.1.0

wiredfool · 2015-05-13T17:58:41Z

Just got the PPC fired up here, and I'm seeing similar issues. (32bit, g4 mini on ubuntu 12.04). It's got openjpeg 2.1.0, and most of the jpeg2k bits are failing.

I'm seeing a slightly different error in TestFileIco which at least points to something:

  File "/home/erics/Pillow/Tests/test_file_ico.py", line 27, in test_save_to_bytes
    reloaded = Image.open(output)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/Image.py", line 2266, in open
    im = factory(fp, filename)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/ImageFile.py", line 97, in __init__
    self._open()
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/IcoImagePlugin.py", line 264, in _open
    self.load()
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/IcoImagePlugin.py", line 267, in load
    im = self.ico.getimage(self.size)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/IcoImagePlugin.py", line 150, in getimage
    return self.frame(i)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/Pillow-2.9.0.dev0-py2.7-linux-ppc.egg/PIL/IcoImagePlugin.py", line 160, in frame
    self.buf.seek(header['offset'])
OverflowError: cannot fit 'long' into an index-sized integer

Ico files are little endian, ref #1204

hugovk · 2015-05-29T15:30:39Z

(Partial) Fix #1232 is merged.

hugovk · 2019-02-17T09:42:20Z

Is this still an issue after the fix from #1232?

nijel · 2019-02-17T17:57:05Z

I don't see how ICO fix would relate to OpenJPEG bug...

Testsuites fail on big endian arch Upstream report: python-pillow/Pillow#3798 python-pillow/Pillow#1204

okurz · 2019-06-11T06:23:07Z

The links to the test results have changed:

https://build.opensuse.org/package/live_build_log/devel:languages:python/python-Pillow/openSUSE_Factory_PowerPC/ppc64 shows the always most recent build and test results on ppc64 (BE)
https://build.opensuse.org/package/live_build_log/devel:languages:python/python-Pillow/openSUSE_Factory_zSystems/s390x are the build and test results on s390x

Both show a lot of problems with the self-tests, eventually ending in an "Segmentation fault".

azaghal/pydenticon#10 is another related downstream issue.

radarhere · 2021-12-22T23:32:19Z

The opensuse builds are currently blocked by other packages.

I'm going to say that this is resolved by #5901, since then all tests mentioned here pass on our Big Endian CI job. Please let us know if there are still problems.

wiredfool added a commit to wiredfool/Pillow that referenced this issue May 13, 2015

Ico files are little endian, ref python-pillow#1204

93461e6

wiredfool mentioned this issue May 13, 2015

Ico files are little endian, ref #1204 #1232

Merged

hugovk added a commit that referenced this issue May 29, 2015

Merge pull request #1232 from wiredfool/ico-bigendian

022cebd

Ico files are little endian, ref #1204

aclark4life added the Bug Any unexpected behavior, until confirmed feature. label Jun 28, 2015

aclark4life added this to the Future milestone Jun 28, 2015

wiredfool changed the title ~~Broken on big endian~~ OpenJPEG failing tests on big endian Dec 29, 2015

jayvdb mentioned this issue Apr 10, 2019

test_make_inmemory_image_should_pass fails on s390x and ppc64 sunscrapers/djet#31

Closed

tuan-hoang1 mentioned this issue Apr 23, 2019

Testsuite fails on alpine linux s390x #3798

Closed

tuan-hoang1 added a commit to tuan-hoang1/aports that referenced this issue Apr 23, 2019

main/py-pillow: disable tests on s390x

fd136d9

Testsuites fail on big endian arch Upstream report: python-pillow/Pillow#3798 python-pillow/Pillow#1204

tuan-hoang1 mentioned this issue Apr 23, 2019

main/py-pillow: disable tests on s390x alpinelinux/aports#7217

Closed

algitbot pushed a commit to alpinelinux/aports that referenced this issue Apr 23, 2019

main/py-pillow: disable tests on s390x

b65c2c0

Testsuites fail on big endian arch Upstream report: python-pillow/Pillow#3798 python-pillow/Pillow#1204

okurz mentioned this issue Jun 8, 2019

self-tests fail in test_generate_format and following on big endian architecture with "tile cannot extend outside image" azaghal/pydenticon#10

Open

hugovk mentioned this issue Nov 19, 2019

Big-endian test failures on IBM Z #4213

Closed

7 tasks

radarhere added the Big-endian Big-endian processors label Apr 29, 2021

radarhere mentioned this issue Dec 22, 2021

Improved I;16 operations on big endian #5901

Merged

hugovk closed this as completed in #5901 Dec 28, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OpenJPEG failing tests on big endian #1204

OpenJPEG failing tests on big endian #1204

nijel commented Apr 22, 2015

wiredfool commented Apr 22, 2015

nijel commented Apr 22, 2015

wiredfool commented May 13, 2015

hugovk commented May 29, 2015

hugovk commented Feb 17, 2019

nijel commented Feb 17, 2019

okurz commented Jun 11, 2019

radarhere commented Dec 22, 2021

OpenJPEG failing tests on big endian #1204

OpenJPEG failing tests on big endian #1204

Comments

nijel commented Apr 22, 2015

wiredfool commented Apr 22, 2015

nijel commented Apr 22, 2015

wiredfool commented May 13, 2015

hugovk commented May 29, 2015

hugovk commented Feb 17, 2019

nijel commented Feb 17, 2019

okurz commented Jun 11, 2019

radarhere commented Dec 22, 2021