Fail on correct-cnv: issue with cooler.balance

[kokonech]

Hi!

Checking the tool on the control worked well, but the problem occurred when trying to adjust own data via CNV normalisation. The procedure is further described, focus was on 10Kbp bin size.

Convert raw HiC-pro result into .cool format and then normalize it via cooler:
cooler balance -p 8 -c 10000 11EP22_10000.cool

Next, calculate-cnv, worked well:
calculate-cnv -e MboI -g hg19 -H 11EP22_10000.cool --output 11EP22_10000.cnv --cachefolder cache

Further, CNV segmentation was OK too:
segment-cnv --cnv-file 11EP22_10000.cnv --binsize 10000 --output 11EP22_10000.segmented.cnv --nproc 4

But fail occurred on the correction of COOL file:
correct-cnv --cnv-file 11EP22_10000.segmented.cnv -H 11EP22_10000.cool --nproc 4

Error was the following:

...
root                      INFO    @ 07/20/21 09:15:58: Match CNV segmentation to matrix bins
root                      INFO    @ 07/20/21 09:16:30: Perform CNV-separate matrix balancing ...
Traceback (most recent call last):
  File "/b06x/cluster/42.2/x86_64/easybuild/software/NeoLoopFinder/0.2.3-foss-2018b-Python-3.7.0-R-3.5.1/bin/correct-cnv", line 94, in run
    matrix_balance(args.hic, nproc=args.nproc)
  File "/b06x/cluster/42.2/x86_64/easybuild/software/NeoLoopFinder/0.2.3-foss-2018b-Python-3.7.0-R-3.5.1/lib/python3.7/site-packages/neoloop/cnv/correctcnv.py", line 175, in matrix_balance
    clr = ice.Cooler(cool_uri)
AttributeError: module 'cooler.balance' has no attribute 'Cooler'

Any ideas what could be the problem? Did I miss some arguments in command? How to fix it? Note that we have specific system on our cluster and installed the packages not via conda, but using local module system trying to follow all versions as close as possible. The cooler version is 0.8.7.

[XiaoTaoWang]

Hi, I believe this is a version compatibility issue of cooler, and can be solved by down-grading cooler to 0.8.6.

[kokonech]

Thanks a lot for quick reply! We used cooler 0.8.7 due to problems in installation of 0.8.6, but found a fix via usage of specific fix 0.8.6.post0 version, and indeed this fixed the problem.

But I have one more question: by some reason I can not restore duplication confirmed from WGS, RNA-seq (fusion) and hicBreakFinder. Simply, I used the coordinates from WGS result to form the input and show duplication:
chr11 chr11 ++ 63532555 65430159 duplication

The following commands were applied after CNV normalisation:

assemble-complexSVs -O 11EP22_neoLoopMain -B $SV_LOCI -H 11EP22_10000.cool
neoloop-caller -O 11EP22.main-neo-loops.txt -H 11EP22_10000.cool --assembly 11EP22_neoLoopMain.assemblies.txt --no-clustering --prob 0.95

For visualisation inspection I used the corresponding coords from
assembly = "C0 duplication,11,63532555,+,11,65430159,+ 11,63130000 11,65220000"
But no loops were found in the SV segment. Moreover, from visualisation it seems as if no strong support for duplication segment. While this is what I see from JuiceBox.

Any suggestions if something was set incorrectly in settings/SV loci assignment? Or should I play with some params? I must note that this segment of duplication actually might be touched by other SV since this a chromothripsis segment, but I wanted simply to start with one example SV.

[Spartanzhao]

Please Check the manual of how to install you package, I couldn't install the packages as the instruction you provided in your github webpage.

[Spartanzhao]

Please Check the manual of how to install you package, I couldn't install the packages as the instruction you provided in your github webpage.

[Spartanzhao]

Please Check the manual of how to install you package, I couldn't install the packages as the instruction you provided in your github webpage.

[Spartanzhao]

Please Check the manual of how to install you package, I couldn't install the packages as the instruction you provided in your github webpage.

[zhangli271828]

HI, I have a problem with correct-cnv also. THe log file has the following content:
root INFO @ 08/16/21 13:16:17:

ARGUMENT LIST:

Cooler URI = inter_30_10k.cool

CNV Profile = cnv.out

Number of Processes = 16

Log file name = cnv-norm2.log

root INFO @ 08/16/21 13:16:18: Match CNV segmentation to matrix bins
Traceback (most recent call last):
File "/usr/local/anaconda3-2020/envs/neoloop/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 3080, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 4554, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 4562, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'weight'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/anaconda3-2020/envs/neoloop/bin/correct-cnv", line 92, in run
bincnv.assign_cnv(args.hic)
File "/usr/local/anaconda3-2020/envs/neoloop/lib/python3.7/site-packages/neoloop/cnv/loadcnv.py", line 87, in assign_cnv
bias = cooler_lib.bins().fetch(ref_k)['weight'].values
File "/usr/local/anaconda3-2020/envs/neoloop/lib/python3.7/site-packages/pandas/core/frame.py", line 3024, in getitem
indexer = self.columns.get_loc(key)
File "/usr/local/anaconda3-2020/envs/neoloop/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 3082, in get_loc
raise KeyError(key) from err
KeyError: 'weight'
################################################################################

Any idea how to fix it?

[JSegueni]

Hi,

I have the same issue as @zhangli271828 ...
Did someone find a solution ?

[XiaoTaoWang]

Hi, this is a duplicated issue #8 , please make sure you have ran "cooler balance" before "correct-cnv"

[JSegueni]

Thank you for your kind comment. Now, it works :)