Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault error with minimap2 #111

Closed
rehab-fouad opened this issue Feb 13, 2018 · 61 comments
Closed

Segmentation fault error with minimap2 #111

rehab-fouad opened this issue Feb 13, 2018 · 61 comments

Comments

@rehab-fouad
Copy link

I have encountered a problem when I use the minimap command line for alignment of nanopore data.
the command line is minimap2 ref.fa query.fa > approx-mapping.paf
This is the first attempt to use minimap2 ever. The error says Segmentation fault.
I am using cent OS 7.

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Screenshot? Do you have ref.fa and query.fa files? Do you put minimap2 on your $PATH? Have you run the examples in "Getting Started"?

@rehab-fouad
Copy link
Author

rehab-fouad commented Feb 13, 2018 via email

@rehab-fouad
Copy link
Author

image

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Thanks. What version are you using? How much memory does your machine have?

@rehab-fouad
Copy link
Author

rehab-fouad commented Feb 13, 2018 via email

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

I mean the minimap2 version. And memory?

@rehab-fouad
Copy link
Author

I used this command line to install it
curl -L https://github.com/lh3/minimap2/releases/download/v2.8/minimap2-2.8_x64-linux.tar.bz2
| tar -jxvf -
./minimap2-2.8_x64-linux/minimap2
which I obtain from here
https://github.com/lh3/minimap2/blob/master/README.md#started

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Thanks for the info. How much memory do you have?

@rehab-fouad
Copy link
Author

1 TB

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Interesting. I don't know what is happening. It has been a while since someone reported segfault for nanopore data. Could you try:

./minimap2 --print-qname human-refseq.fa N006_merge.fa

Also, are you sure N006_merge.fa is correctly formatted? Thanks.

@rehab-fouad
Copy link
Author

n006_fasta_image
se_fault_2

@rehab-fouad
Copy link
Author

I just uploaded images of the command run the way u type it and an image of the N006_merge file opened by Notbad++

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Ok. One possible cause: your N006_merge.fa is generated on Windows, which is malformatted on Linux. Could you can

head -100 N006_merge.fa > part.fa

and send "part.fa" to me. Or you can run

hexdump -C N006_merge.fa | head -50

and send me the screenshot. Thanks!

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

PS: your screenshot with --print-qname suggests minimap2 failed to parse N006_merge.fa. When you open the file with Notepad, there may be hidden characters that failed minimap2.

@rehab-fouad
Copy link
Author

part_fa

@rehab-fouad
Copy link
Author

I don't think N006_merg.fa has a problem because I used it successfully with blastn tool.

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Could you send me part.fa as a file, not as a screenshot?

The blastn parser is probably more robust to errors.

@rehab-fouad
Copy link
Author

I will send it by e-mail because I could not upload it here. ( format is not supported )

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Please send to: lh3@me.com

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Thanks a lot for the help!

@rehab-fouad
Copy link
Author

sent! thank you for your support

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

I was wrong. Sorry. "part.fa" is correctly formatted. I am stuck. Have you tried the example files:

git clone https://github.com/lh3/minimap2
cd minimap2 && make
./minimap2 test/MT-human.fa test/MT-orang.fa

Does the last command line segfault?

@rehab-fouad
Copy link
Author

just copy and past it?

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

yes

@rehab-fouad
Copy link
Author

example
it seems okay

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Could you use the newly compiled minimap2 (not the precompiled binary you downloaded from the release page) on your files again?

path/to/new/minimap2 human-refseq.fa N006_merge.fa

Sorry for bothering you. It is hard to fix segfault without my reproducing the segfault on my machine.

@rehab-fouad
Copy link
Author

[rehab@iria minimap2]$ path/to/new/minimap2 human-refseq.fa N006_merge.fa
-bash: path/to/new/minimap2: ??????????????????????
[rehab@iria minimap2]$

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Please replace "path/to/new/minimap2" with the path to the minimap2 you have just compiled. Thank you.

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Sorry typo. Should be: Please replace "path/to/new/minimap2" with the path to the minimap2 you have just compiled. Thank you.

@rehab-fouad
Copy link
Author

sorry, which command line you mean? the example? or not bothering at all thanks for your help.

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

You don't need to open that file. Just run:

./minimap2 --print-qname ~/N006-minimap/human-refseq.fa ~/N006-minimap/N006_merge.fa

What do you get?

@rehab-fouad
Copy link
Author

cc -c -msse4 -g -Wall -O2 -Wc++-compat -DHAVE_KALLOC -DKSW_CPU_DISPATCH ksw2_exts2_sse.c -o ksw2_exts2_sse41.o
cc -c -g -Wall -O2 -Wc++-compat -DHAVE_KALLOC -DKSW_CPU_DISPATCH -DKSW_SSE2_ONLY ksw2_extz2_sse.c -o ksw2_extz2_sse2.o
cc -c -g -Wall -O2 -Wc++-compat -DHAVE_KALLOC -DKSW_CPU_DISPATCH -DKSW_SSE2_ONLY ksw2_extd2_sse.c -o ksw2_extd2_sse2.o
cc -c -g -Wall -O2 -Wc++-compat -DHAVE_KALLOC -DKSW_CPU_DISPATCH -DKSW_SSE2_ONLY ksw2_exts2_sse.c -o ksw2_exts2_sse2.o
cc -c -g -Wall -O2 -Wc++-compat -DHAVE_KALLOC -DKSW_CPU_DISPATCH ksw2_dispatch.c -o ksw2_dispatch.o
ar -csru libminimap2.a kthread.o kalloc.o misc.o bseq.o sketch.o sdust.o options.o index.o chain.o align.o hit.o map.o format.o pe.o esterr.o ksw2_ll_sse.o ksw2_extz2_sse41.o ksw2_extd2_sse41.o ksw2_exts2_sse41.o ksw2_extz2_sse2.o ksw2_extd2_sse2.o ksw2_exts2_sse2.o ksw2_dispatch.o
cc -g -Wall -O2 -Wc++-compat main.o getopt.o -o minimap2 -L. -lminimap2 -lm -lz -lpthread
[rehab@iria minimap2]$ ./minimap2 --print-qname ~/N006-minimap/human-refseq.fa ~/N006-minimap/N006_merge.fa
./minimap2 --print-qname ~/N006-minimap/human-refseq.fa ~/N006-minimap/N006_merge.fa
[M::mm_idx_gen::26.6241.58] collected minimizers
[M::mm_idx_gen::31.396
1.78] sorted minimizers
[M::main::31.3961.78] loaded/built the index for 154045 target sequence(s)
[M::mm_mapopt_update::32.902
1.75] mid_occ = 133
[M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 154045
[M::mm_idx_stat::33.272*1.74] distinct minimizers: 17642700 (32.52% are singletons); average occurrences: 5.693; average spacing: 5.363
Segmentation fault
[rehab@iria minimap2]$ ./minimap2 --print-qname ~/N006-minimap/human-refseq.fa ~/N006-minimap/N006_merge.fa

@rehab-fouad
Copy link
Author

ÿR 125030 106176 106320 + NR_003038.2 524 284 423 63 144 34 tp:A:P cm:i:6 s1:i:63 s2:i:0 dv:f:0.1003

ÿR 125030 100981 101422 + NM_000987.3 602 92 516 60 441 0 tp:A:P cm:i:8 s1:i:56 s2:i:56 dv:f:0.1527

ÿR 125030 117223 117462 - NM_001032.4 386 59 292 45 239 5 tp:A:P cm:i:6 s1:i:44 s2:i:0 dv:f:0.1414

ÿR 125030 87077 87208 + XM_017016305.1 956 647 785 45 138 0 tp:A:P cm:i:3 s1:i:44 s2:i:44 dv:f:0.1265

ÿR 125030 99633 99737 - NM_003516.2 534 118 219 45 106 0 tp:A:P cm:i:3 s1:i:44 s2:i:44 dv:f:0.1328
[M::worker_pipeline::111.7060.82] mapped 1 sequences
QR ? 1 0
QR 0 81817
QM 81817 cap=8388608,nCore=1,largest=8388608
[M::worker_pipeline::111.710
0.82] mapped 2 sequences
QR 0 89149
QM 89149 cap=8388608,nCore=1,largest=8388608
[M::worker_pipeline::111.715*0.82] mapped 1 sequences
[M::main] Version: 2.8-r703-dirty
[M::main] CMD: ./minimap2 --print-qname /home/rehab/N006-minimap/human-refseq.fa /home/rehab/N006-minimap/N006_merge.fa
[M::main] Real time: 112.465 sec; CPU: 92.399 sec

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Your last command line worked. You can now try:

./minimap2 ~/N006-minimap/human-refseq.fa ~/N006-minimap/N006_merge.fa > approx.paf

Do you get a segfault?

@rehab-fouad
Copy link
Author

yes, I got segfault again.
[M::main] Real time: 112.465 sec; CPU: 92.399 sec
[rehab@iria minimap2]$ ./minimap2 ~/N006-minimap/human-refseq.fa ~/N006-minimap/N006_merge.fa > approx.paf
[M::mm_idx_gen::26.4481.57] collected minimizers
[M::mm_idx_gen::31.420
1.78] sorted minimizers
[M::main::31.4201.78] loaded/built the index for 154045 target sequence(s)
[M::mm_mapopt_update::33.062
1.74] mid_occ = 133
[M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 154045
[M::mm_idx_stat::33.433*1.73] distinct minimizers: 17642700 (32.52% are singletons); average occurrences: 5.693; average spacing: 5.363
Segmentation fault
[rehab@iria minimap2]$

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

I am very certain that your N006_merge.fa is malformatted. It is a bit late on my side. Is it possible at all to send N006_merge.fa to me? Is it too large? Thanks.

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

By "malformatted", I really mean the minimap2 fasta parser is not robust enough to detect the formatting issue. This is still something I need to improve in minimap2.

@rehab-fouad
Copy link
Author

I will send it. as a google drive link by email. Thanks a lot

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Could you also send the human-refseq.fa file to me? I am going to bed. One last thing: could you run the following command on your linux:

hexdump -C N006_merge.fa | head -50

and send the screenshot to me here? Thanks a million.

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

To add the path:

hexdump -C ~/N006-minimap/N006_merge.fa | head -50

and paste me the screenshot here. Thank you.

@rehab-fouad
Copy link
Author

Okay..Nights! thank you for your help despite timezones :)

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Could you send me the screenshot? Thanks.

@rehab-fouad
Copy link
Author

[rehab@iria minimap2]$ hexdump -C ~/N006-minimap/N006_merge.fa | head -50
00000000 3e 64 37 35 35 39 63 32 64 2d 65 34 30 32 2d 34 |>d7559c2d-e402-4|
00000010 37 65 37 2d 61 64 36 31 2d 38 39 64 65 61 32 39 |7e7-ad61-89dea29|
00000020 35 39 33 39 39 20 72 75 6e 69 64 3d 62 32 63 61 |59399 runid=b2ca|
00000030 34 37 38 35 37 37 39 38 62 66 64 38 65 64 62 37 |47857798bfd8edb7|
00000040 64 39 61 63 33 33 32 30 66 66 66 63 35 32 31 32 |d9ac3320fffc5212|
00000050 64 65 34 63 20 72 65 61 64 3d 31 30 30 31 20 63 |de4c read=1001 c|
00000060 68 3d 35 33 20 73 74 61 72 74 5f 74 69 6d 65 3d |h=53 start_time=|
00000070 32 30 31 37 2d 31 30 2d 31 37 54 31 38 3a 32 37 |2017-10-17T18:27|
00000080 3a 31 31 5a 0a 54 54 47 47 54 41 54 47 43 47 43 |:11Z.TTGGTATGCGC|
00000090 54 41 43 43 41 47 54 54 41 43 43 41 54 47 54 54 |TACCAGTTACCATGTT|
000000a0 47 43 54 43 41 47 41 41 41 43 41 47 41 47 41 43 |GCTCAGAAACAGAGAC|
000000b0 47 47 43 41 54 47 43 41 43 47 41 47 41 54 47 41 |GGCATGCACGAGATGA|
000000c0 54 43 54 47 47 54 47 41 47 47 43 54 47 47 41 47 |TCTGGTGAGGCTGGAG|
000000d0 54 54 47 47 47 43 43 54 54 47 54 47 54 47 43 54 |TTGGGCCTTGTGTGCT|
000000e0 43 54 54 47 41 54 43 54 43 41 47 54 43 41 43 41 |CTTGATCTCAGTCACA|
000000f0 47 43 54 41 54 47 43 41 43 54 47 41 47 43 54 47 |GCTATGCACTGAGCTG|
00000100 43 43 41 41 41 47 47 47 41 41 41 43 43 43 54 47 |CCAAAGGGAAACCCTG|
00000110 47 41 43 54 41 43 54 47 43 47 43 43 43 54 47 41 |GACTACTGCGCCCTGA|
00000120 47 54 43 43 41 43 54 41 43 54 54 41 41 47 54 43 |GTCCACTACTTAAGTC|
00000130 54 41 47 41 43 54 47 47 47 54 47 43 41 47 41 54 |TAGACTGGGTGCAGAT|
00000140 43 41 43 41 47 47 47 41 41 47 47 47 41 41 47 47 |CACAGGGAAGGGAAGG|
00000150 47 41 54 54 43 54 43 43 41 47 47 43 43 43 41 47 |GATTCTCCAGGCCCAG|
00000160 41 43 47 47 54 43 43 54 43 41 41 47 43 43 41 47 |ACGGTCCTCAAGCCAG|
00000170 47 41 47 47 43 41 47 43 41 54 47 47 47 47 41 41 |GAGGCAGCATGGGGAA|
00000180 43 54 43 43 43 41 47 41 41 41 47 47 54 41 54 47 |CTCCCAGAAAGGTATG|
00000190 43 41 41 43 41 43 54 43 43 43 43 54 43 41 43 54 |CAACACTCCCCTCACT|
000001a0 47 41 43 54 43 54 41 47 43 41 43 54 54 43 47 47 |GACTCTAGCACTTCGG|
000001b0 41 41 41 47 43 54 47 41 47 41 54 43 47 47 41 41 |AAAGCTGAGATCGGAA|
000001c0 47 47 47 43 47 54 43 47 54 47 54 41 47 47 47 41 |GGGCGTCGTGTAGGGA|
000001d0 41 41 47 41 47 54 47 54 47 41 54 43 54 43 47 47 |AAGAGTGTGATCTCGG|
000001e0 54 47 47 54 43 47 43 43 47 54 41 54 43 41 54 54 |TGGTCGCCGTATCATT|
000001f0 47 43 54 54 41 41 43 41 43 41 54 47 54 41 41 41 |GCTTAACACATGTAAA|
00000200 41 41 41 41 41 41 41 41 47 41 0a 3e 30 61 61 34 |AAAAAAAAGA.>0aa4|
00000210 31 35 30 63 2d 37 38 37 61 2d 34 32 39 36 2d 61 |150c-787a-4296-a|
00000220 32 61 61 2d 39 61 61 63 35 33 34 38 35 66 36 39 |2aa-9aac53485f69|
00000230 20 72 75 6e 69 64 3d 62 32 63 61 34 37 38 35 37 | runid=b2ca47857|
00000240 37 39 38 62 66 64 38 65 64 62 37 64 39 61 63 33 |798bfd8edb7d9ac3|
00000250 33 32 30 66 66 66 63 35 32 31 32 64 65 34 63 20 |320fffc5212de4c |
00000260 72 65 61 64 3d 31 30 30 35 20 63 68 3d 32 36 31 |read=1005 ch=261|
00000270 20 73 74 61 72 74 5f 74 69 6d 65 3d 32 30 31 37 | start_time=2017|
00000280 2d 31 30 2d 31 37 54 31 31 3a 30 39 3a 32 35 5a |-10-17T11:09:25Z|
00000290 0a 54 54 41 47 54 41 54 47 43 54 54 43 47 54 54 |.TTAGTATGCTTCGTT|
000002a0 43 41 47 54 54 41 43 47 54 41 41 54 43 43 41 41 |CAGTTACGTAATCCAA|
000002b0 41 43 47 41 41 47 41 43 41 47 43 41 47 43 41 47 |ACGAAGACAGCAGCAG|
000002c0 43 41 54 47 47 47 41 54 54 47 47 54 47 54 47 47 |CATGGGATTGGTGTGG|
000002d0 43 54 54 54 47 47 41 47 54 54 43 41 47 41 43 47 |CTTTGGAGTTCAGACG|
000002e0 54 47 54 47 43 54 43 47 41 54 43 54 41 41 43 41 |TGTGCTCGATCTAACA|
000002f0 41 43 54 41 54 47 43 47 43 41 47 47 43 54 47 43 |ACTATGCGCAGGCTGC|
00000300 43 54 47 47 47 41 47 54 54 43 54 43 54 47 43 54 |CTGGGAGTTCTCTGCT|

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

I can reproduce the segfault. Thank you very much for this example. I will come back to fix this tomorrow.

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

You can try:

tail ~/N006-minimap/N006_merge.fa

You will see:

        <requestedExecutionLevel
          level="asInvoker"
          uiAccess="false"/>
        </requestedPrivileges>
    </security>
  </trustInfo>
</assembly>

==== ===

The end of the file is malformatted. You can use the following command line to fix the error:

head -72556 N006_merge.fa > N006_merge.fixed.fa

Meanwhile, I will try to make minimap2 more robust to wrong formats. Thank you.

@rehab-fouad
Copy link
Author

[rehab@iria minimap2]$ head -72556 N006_merge.fa > N006_merge.fixed.fa
head: `N006_merge.fa' ? ????????????????: ??????????????????????
[rehab@iria minimap2]$

@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

Please use:

cd ~/N006-minimap/
head -72556 N006_merge.fa > N006_merge.fixed.fa

lh3 added a commit that referenced this issue Feb 13, 2018
The lack of robustness cost me several hours to identify.
@lh3
Copy link
Owner

lh3 commented Feb 13, 2018

I have improved the robustness of the minimap2 FASTA/Q parser. You can run the following:

cd ~
rm -fr minimap2
git clone https://github.com/lh3/minimap2
cd minimap2 && make
./minimap2 ~/N006-minimap/human-refseq.fa ~/N006-minimap/N006_merge.fa > output.paf

Of course, it is still much preferred to fix errors in your input file with:

head -72556 ~/N006-minimap/N006_merge.fa > ~/N006-minimap/N006_merge.fixed.fa

Otherwise it may cause troubles elsewhere.

I am now closing the issue. Thank you very much for your patience.

@lh3 lh3 closed this as completed Feb 13, 2018
@rehab-fouad
Copy link
Author

rehab-fouad commented Feb 14, 2018 via email

@lkie
Copy link

lkie commented Oct 19, 2018

I got "segmentation error" while mappping, when the mmi index was made from FASTA in which sequence didn't have a name (there was only single sequence, with first line ">"). Using minimap2 version 2.12-r827.
Suggestion: Would be great if it failed at the indexing step.

@tseemann
Copy link
Contributor

tseemann commented Oct 19, 2018

@lkie I just tried to reproduce that on 2.13 and it worked. The PAF output has a empty ID though. I will file an issue.

@Maxwell-codesome
Copy link

hi,I met some very strange problems.When I ran minimap2, it worked normally at first, but after I successfully processed several files, it had such a problem.

minimap2 -ax sr --split-prefix temp_name -I 20G -t -d ${db} fastp/${filename}_1_fastp.fastq fastp/${filename}_2_fastp.fastq > minimap2/${filename}_minimap2.sam
[M::mm_idx_gen::132.900*0.88] collected minimizers
[M::mm_idx_gen::173.097*0.91] sorted minimizers
[M::main::173.098*0.91] loaded/built the index for 525 target sequence(s)
[M::mm_mapopt_update::173.098*0.91] mid_occ = 1000
[M::mm_idx_stat] kmer size: 21; skip: 11; is_hpc: 0; #seq: 525
[M::mm_idx_stat::175.717*0.91] distinct minimizers: 381665452 (93.30% are singletons); average occurrences: 1.345; average spacing: 6.297
[M::worker_pipeline::193.315*0.93] mapped 980478 sequences
[M::worker_pipeline::209.299*0.94] mapped 980474 sequences
[M::worker_pipeline::225.957*0.95] mapped 980476 sequences
[M::worker_pipeline::241.763*0.96] mapped 980470 sequences
[M::worker_pipeline::257.198*0.97] mapped 980476 sequences
minimap2: format.c:378: write_sam_cigar: Assertion `clip_len[0] < qlen && clip_len[1] < qlen' failed.

Then this problem appeared again.

minimap2 -ax sr --split-prefix temp_name -t -d ${db} fastp/${filename}_1_fastp.fastq fastp/${filename}_2_fastp.fastq > minimap2/${filename}_minimap2.sam
Segmentation fault (core dumped)
minimap2 -ax sr ${db} output.fastq.gz > output_minimap2.sam
Segmentation fault (core dumped)
minimap2 -ax sr -K 80M -B 5 -w 5 --split-prefix temp_name -I 20G -t -d ${db} ${filename}_1_fastp.fastq ${filename}_2_fastp.fastq > ${filename}_minimap2.sam
Segmentation fault (core dumped)

These two problems often appear. Even the same reference genome and the same data can run normally one second, and then it will be an error if you try again. These errors appear very randomly, and now they become more and more frequent. The server I use has enough memory. I hope you can help me. Thank you very much.

@ryan-moreno
Copy link

@Maxwell-codesome did you find a solution to this? I am running into seg fault errors with minimap2 as well.

@rehab-fouad
Copy link
Author

rehab-fouad commented Apr 24, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants