Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

memory allocation failure (Big memory allocation attempt) on BMP file #1039

Open
youwei1988 opened this issue Oct 25, 2017 · 9 comments
Open

memory allocation failure (Big memory allocation attempt) on BMP file #1039

youwei1988 opened this issue Oct 25, 2017 · 9 comments

Comments

@youwei1988
Copy link

on master and the latest version of openjpeg:

./opj_compress -n 1 -i poc.bmp -o null.j2c

==25686==ERROR: AddressSanitizer failed to allocate 0x836872000 (35274563584) bytes of LargeMmapAllocator (errno: 12)
==25686==Process memory map follows:
0x000000400000-0x00000045c000 /home/youwei/poc/openjpeg/build/bin/opj_compress
0x00000065b000-0x00000065c000 /home/youwei/poc/openjpeg/build/bin/opj_compress
0x00000065c000-0x000000663000 /home/youwei/poc/openjpeg/build/bin/opj_compress
0x00007fff7000-0x00008fff7000
0x00008fff7000-0x02008fff7000
0x02008fff7000-0x10007fff8000
0x600000000000-0x604000000000
0x604000000000-0x604000010000
0x604000010000-0x606000000000
0x606000000000-0x606000010000
0x606000010000-0x616000000000
0x616000000000-0x616000020000
0x616000020000-0x619000000000
0x619000000000-0x619000020000
0x619000020000-0x621000000000
0x621000000000-0x621000020000
0x621000020000-0x640000000000
0x640000000000-0x640000003000
0x7fe726e6f000-0x7fe7aa600000
0x7fe7aa700000-0x7fe7aa800000
0x7fe7aa84c000-0x7fe7acb9e000
0x7fe7acb9e000-0x7fe7acbb7000 /lib/x86_64-linux-gnu/libz.so.1.2.8
0x7fe7acbb7000-0x7fe7acdb6000 /lib/x86_64-linux-gnu/libz.so.1.2.8
0x7fe7acdb6000-0x7fe7acdb7000 /lib/x86_64-linux-gnu/libz.so.1.2.8
0x7fe7acdb7000-0x7fe7acdb8000 /lib/x86_64-linux-gnu/libz.so.1.2.8
0x7fe7acdb8000-0x7fe7acdce000 /lib/x86_64-linux-gnu/libgcc_s.so.1
0x7fe7acdce000-0x7fe7acfcd000 /lib/x86_64-linux-gnu/libgcc_s.so.1
0x7fe7acfcd000-0x7fe7acfce000 /lib/x86_64-linux-gnu/libgcc_s.so.1
0x7fe7acfce000-0x7fe7acfd1000 /lib/x86_64-linux-gnu/libdl-2.23.so
0x7fe7acfd1000-0x7fe7ad1d0000 /lib/x86_64-linux-gnu/libdl-2.23.so
0x7fe7ad1d0000-0x7fe7ad1d1000 /lib/x86_64-linux-gnu/libdl-2.23.so
0x7fe7ad1d1000-0x7fe7ad1d2000 /lib/x86_64-linux-gnu/libdl-2.23.so
0x7fe7ad1d2000-0x7fe7ad1ea000 /lib/x86_64-linux-gnu/libpthread-2.23.so
0x7fe7ad1ea000-0x7fe7ad3e9000 /lib/x86_64-linux-gnu/libpthread-2.23.so
0x7fe7ad3e9000-0x7fe7ad3ea000 /lib/x86_64-linux-gnu/libpthread-2.23.so
0x7fe7ad3ea000-0x7fe7ad3eb000 /lib/x86_64-linux-gnu/libpthread-2.23.so
0x7fe7ad3eb000-0x7fe7ad3ef000
0x7fe7ad3ef000-0x7fe7ad5af000 /lib/x86_64-linux-gnu/libc-2.23.so
0x7fe7ad5af000-0x7fe7ad7af000 /lib/x86_64-linux-gnu/libc-2.23.so
0x7fe7ad7af000-0x7fe7ad7b3000 /lib/x86_64-linux-gnu/libc-2.23.so
0x7fe7ad7b3000-0x7fe7ad7b5000 /lib/x86_64-linux-gnu/libc-2.23.so
0x7fe7ad7b5000-0x7fe7ad7b9000
0x7fe7ad7b9000-0x7fe7ad8c1000 /lib/x86_64-linux-gnu/libm-2.23.so
0x7fe7ad8c1000-0x7fe7adac0000 /lib/x86_64-linux-gnu/libm-2.23.so
0x7fe7adac0000-0x7fe7adac1000 /lib/x86_64-linux-gnu/libm-2.23.so
0x7fe7adac1000-0x7fe7adac2000 /lib/x86_64-linux-gnu/libm-2.23.so
0x7fe7adac2000-0x7fe7adae6000 /lib/x86_64-linux-gnu/libpng12.so.0.54.0
0x7fe7adae6000-0x7fe7adce5000 /lib/x86_64-linux-gnu/libpng12.so.0.54.0
0x7fe7adce5000-0x7fe7adce6000 /lib/x86_64-linux-gnu/libpng12.so.0.54.0
0x7fe7adce6000-0x7fe7adce7000 /lib/x86_64-linux-gnu/libpng12.so.0.54.0
0x7fe7adce7000-0x7fe7adecd000 /home/youwei/poc/openjpeg/build/bin/libopenjp2.so.2.3.0
0x7fe7adecd000-0x7fe7ae0cc000 /home/youwei/poc/openjpeg/build/bin/libopenjp2.so.2.3.0
0x7fe7ae0cc000-0x7fe7ae0ce000 /home/youwei/poc/openjpeg/build/bin/libopenjp2.so.2.3.0
0x7fe7ae0ce000-0x7fe7ae0d9000 /home/youwei/poc/openjpeg/build/bin/libopenjp2.so.2.3.0
0x7fe7ae0d9000-0x7fe7ae1cd000 /usr/lib/x86_64-linux-gnu/libasan.so.2.0.0
0x7fe7ae1cd000-0x7fe7ae3cd000 /usr/lib/x86_64-linux-gnu/libasan.so.2.0.0
0x7fe7ae3cd000-0x7fe7ae3d0000 /usr/lib/x86_64-linux-gnu/libasan.so.2.0.0
0x7fe7ae3d0000-0x7fe7ae3d1000 /usr/lib/x86_64-linux-gnu/libasan.so.2.0.0
0x7fe7ae3d1000-0x7fe7af046000
0x7fe7af046000-0x7fe7af06c000 /lib/x86_64-linux-gnu/ld-2.23.so
0x7fe7af212000-0x7fe7af245000
0x7fe7af24a000-0x7fe7af252000
0x7fe7af253000-0x7fe7af26b000
0x7fe7af26b000-0x7fe7af26c000 /lib/x86_64-linux-gnu/ld-2.23.so
0x7fe7af26c000-0x7fe7af26d000 /lib/x86_64-linux-gnu/ld-2.23.so
0x7fe7af26d000-0x7fe7af26e000
0x7ffc6c564000-0x7ffc6c585000 [stack]
0x7ffc6c5a3000-0x7ffc6c5a5000 [vvar]
0x7ffc6c5a5000-0x7ffc6c5a7000 [vdso]
0xffffffffff600000-0xffffffffff601000 [vsyscall]
==25686==End of process memory map.
==25686==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_posix.cc:121 "(("unable to mmap" && 0)) != (0)" (0x0, 0x0)
#0 0x7fe7ae179631 (/usr/lib/x86_64-linux-gnu/libasan.so.2+0xa0631)
#1 0x7fe7ae17e613 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/usr/lib/x86_64-linux-gnu/libasan.so.2+0xa5613)
#2 0x7fe7ae186641 (/usr/lib/x86_64-linux-gnu/libasan.so.2+0xad641)
#3 0x7fe7ae0fbc0c (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x22c0c)
#4 0x7fe7ae0fcb65 (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x23b65)
#5 0x7fe7ae17203f in __interceptor_posix_memalign (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x9903f)
#6 0x7fe7adeadf90 in opj_aligned_alloc_n /home/youwei/poc/openjpeg/src/lib/openjp2/opj_malloc.c:61
#7 0x7fe7adeadf90 in opj_aligned_malloc /home/youwei/poc/openjpeg/src/lib/openjp2/opj_malloc.c:209
#8 0x7fe7add35790 in opj_image_create /home/youwei/poc/openjpeg/src/lib/openjp2/image.c:77
#9 0x434881 in bmptoimage /home/youwei/poc/openjpeg/src/bin/jp2/convertbmp.c:800
#10 0x409e7a in main /home/youwei/poc/openjpeg/src/bin/jp2/opj_compress.c:1849
#11 0x7fe7ad40f82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#12 0x40f788 in _start (/home/youwei/poc/openjpeg/build/bin/opj_compress+0x40f788)

testcase: (poc.bmp in the zip file)
poc.bmp.zip
@szukw000
Copy link
Contributor

@youwei1988 ,
here is what I get:

bin/opj_compress -n 1 -i poc.bmp -o poc.bmp.j2k

/sources/LIB/IMAGE_FORMATS/OPENJPEG/VERSION-2.3/openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:345: bfSize(2163318)
/sources/LIB/IMAGE_FORMATS/OPENJPEG/VERSION-2.3/openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:356: bfOffBits(0)
Other system than 24 bits/pixels or 8 bits (no RLE coding) is not yet implemented [2]
Unable to load bmp file

The actual filesize is 

ls -l poc.bmp
-rw------- 1 szukw000 szukw000 36 Oct 25 18:24 poc.bmp

And an offset of 0 (zero) is wrong too.

winfried

@youwei1988
Copy link
Author

@szukw000

Did you compile openjpeg with asan available (CFLAGS="-fsanitize=address")?
I have tried to confirm that this flaw can only be detected when compiling with asan available.

@szukw000
Copy link
Contributor

@youwei1988 ,

here is what I get:

cd BUILD
rm -rf *
cmake CFLAGS="-fsanitize=address" ..

bin/opj_compress -i /tmp/poc.bmp -o poc.bmp.j2k

openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:345: bfSize(2163318)
openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:356: bfOffBits(0)
Other system than 24 bits/pixels or 8 bits (no RLE coding) is not yet implemented [2]
Unable to load bmp file

bin/opj_compress -n 1 -i /tmp/poc.bmp -o poc.bmp.j2k

openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:345: bfSize(2163318)
openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:356: bfOffBits(0)
Other system than 24 bits/pixels or 8 bits (no RLE coding) is not yet implemented [2]
Unable to load bmp file

winfried

@youwei1988
Copy link
Author

@szukw000

Thanks for reply.
Could you please let me know the version or commit id that you tested?
I will build the same environment that you use to test the issue.

@szukw000
Copy link
Contributor

@youwei1988 ,
the currently availabe zip file: Latest commit d9f8f7b 9 days ago
winfried

@youwei1988
Copy link
Author

@szukw000

Hi, I have tested the version commit d9f8f7b and can also got the memory allocation failure as I posted before.

I think the reason of different testing results is the compile options you used: cmake CFLAGS="-fsanitize=address" ..
I checked the compiled binary and found that it does not contain ASAN code.

If you use CFLAGS="-fsanitize=address" cmake .. (i.e., set CFLAGS before executing cmake)
then the ASAN code will be enabled, and you will see the memory allocation failure.

Could you please try that?
Thanks.

@szukw000
Copy link
Contributor

szukw000 commented Nov 5, 2017

@youwei1988 ,

here is what I get:

cd BUILD
rm -rf *
CFLAGS="-fsanitize=address" cmake ..

bin/opj_compress -i /tmp/poc.bmp -o poc.bmp.jp2

openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:345: bfSize(2163318)
openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:356: bfOffBits(0)
Other system than 24 bits/pixels or 8 bits (no RLE coding) is not yet implemented [2]
Unable to load bmp file


bin/opj_compress -n 1 -i /tmp/poc.bmp -o poc.bmp.jp2

openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:345: bfSize(2163318)
openjpeg2-2017-10-22/src/bin/jp2/convertbmp.c:356: bfOffBits(0)
Other system than 24 bits/pixels or 8 bits (no RLE coding) is not yet implemented [2]
Unable to load bmp file

winfried

@youwei1988
Copy link
Author

@szukw000
Did you test in the 32 bit environment or 64 bit environment?

@szukw000
Copy link
Contributor

szukw000 commented Nov 5, 2017

@youwei1988 ,
64bit.
winfried

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@szukw000 @youwei1988