Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Help with Kernal dead #259

Open
Chrisdoan9 opened this issue Nov 16, 2023 · 9 comments
Open

Help with Kernal dead #259

Chrisdoan9 opened this issue Nov 16, 2023 · 9 comments
Labels
question Further information is requested

Comments

@Chrisdoan9
Copy link

Hi team,

Would you please suggest what cause the kernal dead error? Thank you so much! This cell ran for about 2 hours and then I got the error kernal dead. I put 1024Gb of memory and still got kernal dead.

from scenicplus.wrappers.run_pycistarget import run_pycistarget
run_pycistarget(
    region_sets = region_sets,
    species = 'homo_sapiens',
    save_path = os.path.join(work_dir, 'motifs'),
    ctx_db_path = rankings_db,
    dem_db_path = scores_db,
    path_to_motif_annotations = motif_annotation,
    run_without_promoters = True,
    n_tpu = 1,
    _temp_dir = os.path.join(tmp_dir, 'ray_spill'),
    annotation_version = 'v10nr_clust',
    )

This is what I have in the log file:

[I 16:02:55.296 NotebookApp] Saving file at /project/multiome.ipynb
[I 16:16:57.060 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 0fc1137e-0f27-41a3-a544-4bd674cce75b restarted
[I 16:18:55.288 NotebookApp] Saving file at /project/multiome.ipynb

I found a similar issue here: aertslab/SCENICprotocol#28
I run Jupyter notebook on a server.

Version (please complete the following information):

  • Python: [e.g. Python 3.9.12]
  • SCENIC+: [e.g. 0.1.dev40+g4ff5053.d20210520]
  • If a bug is related to another module [e.g. matplotlib 3.3.9]
@Chrisdoan9 Chrisdoan9 added the question Further information is requested label Nov 16, 2023
@SeppeDeWinter
Copy link
Collaborator

Hi @Chrisdoan9

At what step is the kernel dying?

Also the development version of the code uses less memory, so you can also consider using that version.

See #202 for more info.

All the best,

Seppe

@Chrisdoan9
Copy link
Author

Hi @SeppeDeWinter

The kernel dying at this cell:

from scenicplus.wrappers.run_pycistarget import run_pycistarget
run_pycistarget(
    region_sets = region_sets,
    species = 'homo_sapiens',
    save_path = os.path.join(work_dir, 'motifs'),
    ctx_db_path = rankings_db,
    dem_db_path = scores_db,
    path_to_motif_annotations = motif_annotation,
    run_without_promoters = True,
    n_tpu = 1,
    _temp_dir = os.path.join(tmp_dir, 'ray_spill'),
    annotation_version = 'v10nr_clust',
    )

@SeppeDeWinter
Copy link
Collaborator

Hi @Chrisdoan9

Did you have any output before the kernel died? And could you please copy-paste this? The function you pasted above will run many steps, so it's difficult for me to know what might be causing the issue.

Best,

Seppe

@Chrisdoan9
Copy link
Author

Hi @SeppeDeWinter, sure. That function printed a lot of output during about 2 hours of running:

2023-11-14 14:33:44,142 pycisTarget_wrapper INFO pbmc_tutorial/motifs folder already exists.
2023-11-14 14:33:45,422 pycisTarget_wrapper INFO Loading cisTarget database for topics_otsu
2023-11-14 14:33:45,423 cisTarget INFO Reading cisTarget database
2023-11-14 14:37:52,116 pycisTarget_wrapper INFO Running cisTarget for topics_otsu
2023-11-14 14:37:52,118 cisTarget INFO Running cisTarget for Topic1 which has 4811 regions
2023-11-14 14:38:15,144 cisTarget INFO Annotating motifs for Topic1
2023-11-14 14:38:18,547 cisTarget INFO Getting cistromes for Topic1
2023-11-14 14:38:18,939 cisTarget INFO Running cisTarget for Topic2 which has 7981 regions
2023-11-14 14:38:38,055 cisTarget INFO Annotating motifs for Topic2
2023-11-14 14:38:41,118 cisTarget INFO Getting cistromes for Topic2
2023-11-14 14:38:41,606 cisTarget INFO Running cisTarget for Topic3 which has 5233 regions
2023-11-14 14:39:01,134 cisTarget INFO Annotating motifs for Topic3
2023-11-14 14:39:04,537 cisTarget INFO Getting cistromes for Topic3
2023-11-14 14:39:05,020 cisTarget INFO Running cisTarget for Topic4 which has 3393 regions
2023-11-14 14:39:23,593 cisTarget INFO Annotating motifs for Topic4
2023-11-14 14:39:26,517 cisTarget INFO Getting cistromes for Topic4
2023-11-14 14:39:26,958 cisTarget INFO Running cisTarget for Topic5 which has 2608 regions
2023-11-14 14:39:45,569 cisTarget INFO Annotating motifs for Topic5
2023-11-14 14:39:49,003 cisTarget INFO Getting cistromes for Topic5
2023-11-14 14:39:49,659 cisTarget INFO Running cisTarget for Topic6 which has 1795 regions
2023-11-14 14:40:08,710 cisTarget INFO Annotating motifs for Topic6
2023-11-14 14:40:11,689 cisTarget INFO Getting cistromes for Topic6
2023-11-14 14:40:12,325 cisTarget INFO Running cisTarget for Topic7 which has 5383 regions
2023-11-14 14:40:31,390 cisTarget INFO Annotating motifs for Topic7
2023-11-14 14:40:34,348 cisTarget INFO Getting cistromes for Topic7
2023-11-14 14:40:34,863 cisTarget INFO Running cisTarget for Topic8 which has 6027 regions
2023-11-14 14:40:54,441 cisTarget INFO Annotating motifs for Topic8
2023-11-14 14:40:58,174 cisTarget INFO Getting cistromes for Topic8
2023-11-14 14:40:58,723 cisTarget INFO Running cisTarget for Topic9 which has 6579 regions
2023-11-14 14:41:18,911 cisTarget INFO Annotating motifs for Topic9
2023-11-14 14:41:22,620 cisTarget INFO Getting cistromes for Topic9
2023-11-14 14:41:23,350 cisTarget INFO Running cisTarget for Topic10 which has 7277 regions
2023-11-14 14:41:43,716 cisTarget INFO Annotating motifs for Topic10
2023-11-14 14:41:47,236 cisTarget INFO Getting cistromes for Topic10
2023-11-14 14:41:47,807 cisTarget INFO Running cisTarget for Topic11 which has 4394 regions
2023-11-14 14:42:06,965 cisTarget INFO Annotating motifs for Topic11
2023-11-14 14:42:10,277 cisTarget INFO Getting cistromes for Topic11
2023-11-14 14:42:10,882 cisTarget INFO Running cisTarget for Topic12 which has 8638 regions
2023-11-14 14:42:31,717 cisTarget INFO Annotating motifs for Topic12
2023-11-14 14:42:36,166 cisTarget INFO Getting cistromes for Topic12
2023-11-14 14:42:36,995 cisTarget INFO Running cisTarget for Topic13 which has 3038 regions
2023-11-14 14:42:56,722 cisTarget INFO Annotating motifs for Topic13
2023-11-14 14:43:00,047 cisTarget INFO Getting cistromes for Topic13
2023-11-14 14:43:00,567 cisTarget INFO Running cisTarget for Topic14 which has 6248 regions
2023-11-14 14:43:21,087 cisTarget INFO Annotating motifs for Topic14
2023-11-14 14:43:25,868 cisTarget INFO Getting cistromes for Topic14
2023-11-14 14:43:26,559 cisTarget INFO Running cisTarget for Topic15 which has 4824 regions
2023-11-14 14:43:46,414 cisTarget INFO Annotating motifs for Topic15
2023-11-14 14:43:49,907 cisTarget INFO Getting cistromes for Topic15
2023-11-14 14:43:50,422 cisTarget INFO Running cisTarget for Topic16 which has 9653 regions
2023-11-14 14:45:35,354 cisTarget INFO Getting cistromes for Topic3
2023-11-14 14:45:35,714 cisTarget INFO Running cisTarget for Topic4 which has 2426 regions
2023-11-14 14:45:53,764 cisTarget INFO Annotating motifs for Topic4
2023-11-14 14:45:56,499 cisTarget INFO Getting cistromes for Topic4
2023-11-14 14:45:56,838 cisTarget INFO Running cisTarget for Topic5 which has 277 regions
2023-11-14 14:46:15,899 cisTarget INFO Annotating motifs for Topic5
2023-11-14 14:46:18,438 cisTarget INFO Getting cistromes for Topic5
2023-11-14 14:46:18,892 cisTarget INFO Running cisTarget for Topic6 which has 512 regions
2023-11-14 14:46:38,179 cisTarget INFO Annotating motifs for Topic6
2023-11-14 14:46:40,905 cisTarget INFO Getting cistromes for Topic6
2023-11-14 14:46:41,348 cisTarget INFO Running cisTarget for Topic7 which has 3876 regions
2023-11-14 14:47:00,221 cisTarget INFO Annotating motifs for Topic7
2023-11-14 14:47:02,845 cisTarget INFO Getting cistromes for Topic7
2023-11-14 14:47:03,153 cisTarget INFO Running cisTarget for Topic8 which has 4326 regions
2023-11-14 14:47:23,048 cisTarget INFO Annotating motifs for Topic8
2023-11-14 14:47:26,690 cisTarget INFO Getting cistromes for Topic8
2023-11-14 14:47:27,137 cisTarget INFO Running cisTarget for Topic9 which has 4448 regions
2023-11-14 14:47:47,506 cisTarget INFO Annotating motifs for Topic9
2023-11-14 14:47:50,770 cisTarget INFO Getting cistromes for Topic9
2023-11-14 14:47:51,120 cisTarget INFO Running cisTarget for Topic10 which has 5533 regions
2023-11-14 14:48:11,100 cisTarget INFO Annotating motifs for Topic10
2023-11-14 14:48:14,493 cisTarget INFO Getting cistromes for Topic10
2023-11-14 14:48:14,909 cisTarget INFO Running cisTarget for Topic11 which has 3226 regions
2023-11-14 14:48:34,514 cisTarget INFO Annotating motifs for Topic11
2023-11-14 14:48:37,609 cisTarget INFO Getting cistromes for Topic11
2023-11-14 14:48:37,925 cisTarget INFO Running cisTarget for Topic12 which has 5220 regions
2023-11-14 14:48:59,401 cisTarget INFO Annotating motifs for Topic12
2023-11-14 14:49:03,088 cisTarget INFO Getting cistromes for Topic12
2023-11-14 14:49:03,481 cisTarget INFO Running cisTarget for Topic13 which has 855 regions
2023-11-14 14:49:22,157 cisTarget INFO Annotating motifs for Topic13
2023-11-14 14:49:24,842 cisTarget INFO Getting cistromes for Topic13
2023-11-14 14:49:25,233 cisTarget INFO Running cisTarget for Topic14 which has 704 regions
2023-11-14 14:49:44,226 cisTarget INFO Annotating motifs for Topic14
2023-11-14 14:49:47,018 cisTarget INFO Getting cistromes for Topic14
2023-11-14 14:49:47,526 cisTarget INFO Running cisTarget for Topic15 which has 3281 regions
2023-11-14 14:50:06,808 cisTarget INFO Annotating motifs for Topic15
2023-11-14 14:50:10,182 cisTarget INFO Getting cistromes for Topic15
2023-11-14 14:50:10,678 cisTarget INFO Running cisTarget for Topic16 which has 7734 regions
2023-11-14 14:50:32,461 cisTarget INFO Annotating motifs for Topic16
2023-11-14 14:50:38,245 cisTarget INFO Getting cistromes for Topic16
2023-11-14 14:50:38,818 cisTarget INFO Done!
2023-11-14 14:50:38,821 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_topics_otsu_No_promoters folder already exists.
2023-11-14 14:50:39,748 pycisTarget_wrapper INFO Running DEM for topics_otsu
2023-11-14 14:50:39,749 DEM INFO Reading DEM database
2023-11-14 14:53:15,796 DEM INFO Creating contrast groups
2023-11-14 14:53:32,679 DEM INFO Computing DEM for Topic1
2023-11-14 14:53:44,185 DEM INFO Computing DEM for Topic2
2023-11-14 14:53:54,264 DEM INFO Computing DEM for Topic3
2023-11-14 14:54:05,413 DEM INFO Computing DEM for Topic4
2023-11-14 14:54:12,648 DEM INFO Computing DEM for Topic5
2023-11-14 14:54:55,365 DEM INFO Computing DEM for Topic6
2023-11-14 14:55:11,273 DEM INFO Computing DEM for Topic7
2023-11-14 14:55:19,533 DEM INFO Computing DEM for Topic8
2023-11-14 14:55:35,715 DEM INFO Computing DEM for Topic9
2023-11-14 14:55:55,462 DEM INFO Computing DEM for Topic10
2023-11-14 14:56:18,485 DEM INFO Computing DEM for Topic11
2023-11-14 14:56:56,854 DEM INFO Computing DEM for Topic12
2023-11-14 14:57:30,546 DEM INFO Computing DEM for Topic13
2023-11-14 14:57:52,154 DEM INFO Computing DEM for Topic14
2023-11-14 14:59:41,912 DEM INFO Computing DEM for Topic15
2023-11-14 15:00:04,310 DEM INFO Computing DEM for Topic16
2023-11-14 15:00:47,283 DEM INFO Forming cistromes
2023-11-14 15:00:55,930 DEM INFO Done!
2023-11-14 15:00:55,937 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_otsu_All folder already exists.
2023-11-14 15:00:56,981 pycisTarget_wrapper INFO Running DEM without promoters for topics_otsu
2023-11-14 15:01:10,046 DEM INFO Creating contrast groups
2023-11-14 15:01:12,071 DEM INFO Computing DEM for Topic1
2023-11-14 15:01:26,912 DEM INFO Computing DEM for Topic2
2023-11-14 15:01:35,275 DEM INFO Computing DEM for Topic3
2023-11-14 15:01:45,815 DEM INFO Computing DEM for Topic4
2023-11-14 15:01:53,043 DEM INFO Computing DEM for Topic5
2023-11-14 15:02:07,883 DEM INFO Computing DEM for Topic6
2023-11-14 15:02:18,473 DEM INFO Computing DEM for Topic7
2023-11-14 15:02:26,438 DEM INFO Computing DEM for Topic8
2023-11-14 15:02:40,317 DEM INFO Computing DEM for Topic9
2023-11-14 15:02:58,754 DEM INFO Computing DEM for Topic10
2023-11-14 15:03:11,774 DEM INFO Computing DEM for Topic11
2023-11-14 15:03:30,788 DEM INFO Computing DEM for Topic12
2023-11-14 15:03:43,488 DEM INFO Computing DEM for Topic13
2023-11-14 15:03:57,631 DEM INFO Computing DEM for Topic14
2023-11-14 15:04:21,460 DEM INFO Computing DEM for Topic15
2023-11-14 15:04:44,351 DEM INFO Computing DEM for Topic16
2023-11-14 15:05:15,287 DEM INFO Forming cistromes
2023-11-14 15:05:23,512 DEM INFO Done!
2023-11-14 15:05:23,517 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_otsu_No_promoters folder already exists.
2023-11-14 15:05:24,533 pycisTarget_wrapper INFO Loading cisTarget database for topics_top_3
2023-11-14 15:05:24,534 cisTarget INFO Reading cisTarget database
2023-11-14 15:08:37,317 pycisTarget_wrapper INFO Running cisTarget for topics_top_3
2023-11-14 15:08:37,320 cisTarget INFO Running cisTarget for Topic1 which has 3413 regions
2023-11-14 15:08:57,989 cisTarget INFO Annotating motifs for Topic1
2023-11-14 15:09:01,201 cisTarget INFO Getting cistromes for Topic1
2023-11-14 15:09:01,554 cisTarget INFO Running cisTarget for Topic2 which has 3125 regions
2023-11-14 15:09:19,476 cisTarget INFO Annotating motifs for Topic2
2023-11-14 15:09:22,177 cisTarget INFO Getting cistromes for Topic2
2023-11-14 15:09:22,653 cisTarget INFO Running cisTarget for Topic3 which has 3206 regions
2023-11-14 15:09:41,213 cisTarget INFO Annotating motifs for Topic3
2023-11-14 15:09:44,310 cisTarget INFO Getting cistromes for Topic3
2023-11-14 15:09:44,849 cisTarget INFO Running cisTarget for Topic4 which has 3529 regions
2023-11-14 15:10:03,381 cisTarget INFO Annotating motifs for Topic4
2023-11-14 15:10:06,329 cisTarget INFO Getting cistromes for Topic4
2023-11-14 15:10:06,806 cisTarget INFO Running cisTarget for Topic5 which has 3971 regions
2023-11-14 15:10:27,135 cisTarget INFO Annotating motifs for Topic5
2023-11-14 15:10:31,127 cisTarget INFO Getting cistromes for Topic5
2023-11-14 15:10:31,747 cisTarget INFO Running cisTarget for Topic6 which has 4058 regions
2023-11-14 15:10:50,719 cisTarget INFO Annotating motifs for Topic6
2023-11-14 15:10:54,190 cisTarget INFO Getting cistromes for Topic6
2023-11-14 15:10:54,873 cisTarget INFO Running cisTarget for Topic7 which has 3471 regions
2023-11-14 15:11:14,121 cisTarget INFO Annotating motifs for Topic7
2023-11-14 15:11:17,058 cisTarget INFO Getting cistromes for Topic7
2023-11-14 15:11:17,540 cisTarget INFO Running cisTarget for Topic8 which has 3560 regions
2023-11-14 15:11:38,052 cisTarget INFO Annotating motifs for Topic8
2023-11-14 15:11:41,340 cisTarget INFO Getting cistromes for Topic8
2023-11-14 15:11:41,848 cisTarget INFO Running cisTarget for Topic9 which has 3347 regions
2023-11-14 15:12:00,712 cisTarget INFO Annotating motifs for Topic9
2023-11-14 15:12:03,890 cisTarget INFO Getting cistromes for Topic9
2023-11-14 15:12:04,506 cisTarget INFO Running cisTarget for Topic10 which has 3397 regions
2023-11-14 15:12:22,434 cisTarget INFO Annotating motifs for Topic10
2023-11-14 15:12:25,345 cisTarget INFO Getting cistromes for Topic10
2023-11-14 15:12:25,715 cisTarget INFO Running cisTarget for Topic11 which has 3435 regions
2023-11-14 15:12:44,196 cisTarget INFO Annotating motifs for Topic11
2023-11-14 15:12:47,302 cisTarget INFO Getting cistromes for Topic11
2023-11-14 15:12:47,778 cisTarget INFO Running cisTarget for Topic12 which has 3393 regions
2023-11-14 15:13:06,405 cisTarget INFO Annotating motifs for Topic12
2023-11-14 15:13:09,663 cisTarget INFO Getting cistromes for Topic12
2023-11-14 15:13:10,300 cisTarget INFO Running cisTarget for Topic13 which has 4250 regions
2023-11-14 15:13:29,467 cisTarget INFO Annotating motifs for Topic13
2023-11-14 15:13:33,116 cisTarget INFO Getting cistromes for Topic13
2023-11-14 15:13:33,677 cisTarget INFO Running cisTarget for Topic14 which has 3853 regions
2023-11-14 15:13:53,639 cisTarget INFO Annotating motifs for Topic14
2023-11-14 15:13:57,655 cisTarget INFO Getting cistromes for Topic14
2023-11-14 15:13:58,322 cisTarget INFO Running cisTarget for Topic15 which has 3489 regions
2023-11-14 15:14:17,102 cisTarget INFO Annotating motifs for Topic15
2023-11-14 15:14:20,335 cisTarget INFO Getting cistromes for Topic15
2023-11-14 15:14:20,922 cisTarget INFO Running cisTarget for Topic16 which has 3428 regions
2023-11-14 15:14:39,389 cisTarget INFO Annotating motifs for Topic16
2023-11-14 15:14:42,491 cisTarget INFO Getting cistromes for Topic16
2023-11-14 15:14:43,169 cisTarget INFO Done!
2023-11-14 15:14:43,171 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_topics_top_3_All folder already exists.
2023-11-14 15:14:44,234 pycisTarget_wrapper INFO Running cisTarget without promoters for topics_top_3
2023-11-14 15:14:53,974 cisTarget INFO Running cisTarget for Topic1 which has 2657 regions
2023-11-14 15:15:12,661 cisTarget INFO Annotating motifs for Topic1
2023-11-14 15:15:15,622 cisTarget INFO Getting cistromes for Topic1
2023-11-14 15:15:16,043 cisTarget INFO Running cisTarget for Topic2 which has 2369 regions
2023-11-14 15:15:33,105 cisTarget INFO Annotating motifs for Topic2
2023-11-14 15:15:35,660 cisTarget INFO Getting cistromes for Topic2
2023-11-14 15:15:35,985 cisTarget INFO Running cisTarget for Topic3 which has 2678 regions
2023-11-14 15:15:54,118 cisTarget INFO Annotating motifs for Topic3
2023-11-14 15:15:57,088 cisTarget INFO Getting cistromes for Topic3
2023-11-14 15:15:57,430 cisTarget INFO Running cisTarget for Topic4 which has 2527 regions
2023-11-14 15:16:15,376 cisTarget INFO Annotating motifs for Topic4
2023-11-14 15:16:18,151 cisTarget INFO Getting cistromes for Topic4
2023-11-14 15:16:18,501 cisTarget INFO Running cisTarget for Topic5 which has 567 regions
2023-11-14 15:16:36,033 cisTarget INFO Annotating motifs for Topic5
2023-11-14 15:16:38,623 cisTarget INFO Getting cistromes for Topic5
2023-11-14 15:16:39,141 cisTarget INFO Running cisTarget for Topic6 which has 1527 regions
2023-11-14 15:16:58,247 cisTarget INFO Annotating motifs for Topic6
2023-11-14 15:17:00,917 cisTarget INFO Getting cistromes for Topic6
2023-11-14 15:17:01,352 cisTarget INFO Running cisTarget for Topic7 which has 2394 regions
2023-11-14 15:17:18,215 cisTarget INFO Annotating motifs for Topic7
2023-11-14 15:17:20,749 cisTarget INFO Getting cistromes for Topic7
2023-11-14 15:17:21,015 cisTarget INFO Running cisTarget for Topic8 which has 2546 regions
2023-11-14 15:17:39,279 cisTarget INFO Annotating motifs for Topic8
2023-11-14 15:17:42,338 cisTarget INFO Getting cistromes for Topic8
2023-11-14 15:17:42,763 cisTarget INFO Running cisTarget for Topic9 which has 2262 regions
2023-11-14 15:18:02,598 cisTarget INFO Annotating motifs for Topic9
2023-11-14 15:18:05,499 cisTarget INFO Getting cistromes for Topic9
2023-11-14 15:18:05,807 cisTarget INFO Running cisTarget for Topic10 which has 2599 regions
2023-11-14 15:18:23,450 cisTarget INFO Annotating motifs for Topic10
2023-11-14 15:18:26,290 cisTarget INFO Getting cistromes for Topic10
2023-11-14 15:18:26,650 cisTarget INFO Running cisTarget for Topic11 which has 2562 regions
2023-11-14 15:18:45,195 cisTarget INFO Annotating motifs for Topic11
2023-11-14 15:18:48,162 cisTarget INFO Getting cistromes for Topic11
2023-11-14 15:18:48,491 cisTarget INFO Running cisTarget for Topic12 which has 2143 regions
2023-11-14 15:19:06,810 cisTarget INFO Annotating motifs for Topic12
2023-11-14 15:19:09,749 cisTarget INFO Getting cistromes for Topic12
2023-11-14 15:19:10,163 cisTarget INFO Running cisTarget for Topic13 which has 1375 regions
2023-11-14 15:19:27,967 cisTarget INFO Annotating motifs for Topic13
2023-11-14 15:19:30,642 cisTarget INFO Getting cistromes for Topic13
2023-11-14 15:19:31,109 cisTarget INFO Running cisTarget for Topic14 which has 329 regions
2023-11-14 15:19:49,560 cisTarget INFO Annotating motifs for Topic14
2023-11-14 15:19:52,297 cisTarget INFO Getting cistromes for Topic14
2023-11-14 15:19:52,794 cisTarget INFO Running cisTarget for Topic15 which has 2382 regions
2023-11-14 15:20:11,740 cisTarget INFO Annotating motifs for Topic15
2023-11-14 15:20:14,842 cisTarget INFO Getting cistromes for Topic15
2023-11-14 15:20:15,319 cisTarget INFO Running cisTarget for Topic16 which has 2785 regions
2023-11-14 15:20:34,905 cisTarget INFO Annotating motifs for Topic16
2023-11-14 15:20:37,960 cisTarget INFO Getting cistromes for Topic16
2023-11-14 15:20:38,368 cisTarget INFO Done!
2023-11-14 15:20:38,371 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_topics_top_3_No_promoters folder already exists.
2023-11-14 15:20:39,161 pycisTarget_wrapper INFO Running DEM for topics_top_3
2023-11-14 15:20:39,162 DEM INFO Reading DEM database
2023-11-14 15:23:03,395 DEM INFO Creating contrast groups
2023-11-14 15:23:18,277 DEM INFO Computing DEM for Topic1
2023-11-14 15:23:30,539 DEM INFO Computing DEM for Topic2
2023-11-14 15:23:37,536 DEM INFO Computing DEM for Topic3
2023-11-14 15:23:48,353 DEM INFO Computing DEM for Topic4
2023-11-14 15:23:55,796 DEM INFO Computing DEM for Topic5
2023-11-14 15:24:45,513 DEM INFO Computing DEM for Topic6
2023-11-14 15:24:57,273 DEM INFO Computing DEM for Topic7
2023-11-14 15:25:04,635 DEM INFO Computing DEM for Topic8
2023-11-14 15:25:17,356 DEM INFO Computing DEM for Topic9
2023-11-14 15:25:34,283 DEM INFO Computing DEM for Topic10
2023-11-14 15:26:00,574 DEM INFO Computing DEM for Topic11
2023-11-14 15:26:15,807 DEM INFO Computing DEM for Topic12
2023-11-14 15:26:41,715 DEM INFO Computing DEM for Topic13
2023-11-14 15:27:00,831 DEM INFO Computing DEM for Topic14
2023-11-14 15:27:55,464 DEM INFO Computing DEM for Topic15
2023-11-14 15:28:05,908 DEM INFO Computing DEM for Topic16
2023-11-14 15:28:38,845 DEM INFO Forming cistromes
2023-11-14 15:28:45,751 DEM INFO Done!
2023-11-14 15:28:45,755 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_top_3_All folder already exists.
2023-11-14 15:28:46,741 pycisTarget_wrapper INFO Running DEM without promoters for topics_top_3
2023-11-14 15:28:56,871 DEM INFO Creating contrast groups
2023-11-14 15:28:58,153 DEM INFO Computing DEM for Topic1
2023-11-14 15:29:08,454 DEM INFO Computing DEM for Topic2
2023-11-14 15:29:15,488 DEM INFO Computing DEM for Topic3
2023-11-14 15:29:23,037 DEM INFO Computing DEM for Topic4
2023-11-14 15:29:29,224 DEM INFO Computing DEM for Topic5
2023-11-14 15:29:43,191 DEM INFO Computing DEM for Topic6
2023-11-14 15:29:54,836 DEM INFO Computing DEM for Topic7
2023-11-14 15:30:01,625 DEM INFO Computing DEM for Topic8
2023-11-14 15:30:08,730 DEM INFO Computing DEM for Topic9
2023-11-14 15:30:23,133 DEM INFO Computing DEM for Topic10
2023-11-14 15:30:39,920 DEM INFO Computing DEM for Topic11
2023-11-14 15:30:59,357 DEM INFO Computing DEM for Topic12
2023-11-14 15:31:12,615 DEM INFO Computing DEM for Topic13
2023-11-14 15:31:25,791 DEM INFO Computing DEM for Topic14
2023-11-14 15:31:41,341 DEM INFO Computing DEM for Topic15
2023-11-14 15:31:59,696 DEM INFO Computing DEM for Topic16
2023-11-14 15:32:22,851 DEM INFO Forming cistromes
2023-11-14 15:32:29,807 DEM INFO Done!
2023-11-14 15:32:29,811 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_top_3_No_promoters folder already exists.
2023-11-14 15:32:30,889 pycisTarget_wrapper INFO Loading cisTarget database for DARs
2023-11-14 15:32:30,890 cisTarget INFO Reading cisTarget database
2023-11-14 15:35:56,216 pycisTarget_wrapper INFO Running cisTarget for DARs
2023-11-14 15:35:56,220 cisTarget INFO Running cisTarget for B_cells which has 11050 regions
2023-11-14 15:36:20,092 cisTarget INFO Annotating motifs for B_cells
2023-11-14 15:36:24,410 cisTarget INFO Getting cistromes for B_cells
2023-11-14 15:36:24,903 cisTarget INFO Running cisTarget for B_cells_1 which has 30035 regions
2023-11-14 15:36:59,595 cisTarget INFO Annotating motifs for B_cells_1
2023-11-14 15:37:07,747 cisTarget INFO Getting cistromes for B_cells_1
2023-11-14 15:37:08,728 cisTarget INFO Running cisTarget for B_cells_2 which has 11872 regions
2023-11-14 15:37:31,067 cisTarget INFO Annotating motifs for B_cells_2
2023-11-14 15:37:35,096 cisTarget INFO Getting cistromes for B_cells_2
2023-11-14 15:37:35,564 cisTarget INFO Running cisTarget for CD14+_Monocytes which has 29730 regions
2023-11-14 15:38:07,690 cisTarget INFO Annotating motifs for CD14+_Monocytes
2023-11-14 15:38:16,045 cisTarget INFO Getting cistromes for CD14+_Monocytes
2023-11-14 15:38:16,987 cisTarget INFO Running cisTarget for CD4_T_cells which has 16991 regions
2023-11-14 15:38:40,429 cisTarget INFO Annotating motifs for CD4_T_cells
2023-11-14 15:38:45,743 cisTarget INFO Getting cistromes for CD4_T_cells
2023-11-14 15:38:46,457 cisTarget INFO Running cisTarget for CD8_T_cells which has 9333 regions
2023-11-14 15:39:07,418 cisTarget INFO Annotating motifs for CD8_T_cells
2023-11-14 15:39:11,329 cisTarget INFO Getting cistromes for CD8_T_cells
2023-11-14 15:39:11,800 cisTarget INFO Running cisTarget for Dendritic_cells which has 24933 regions
2023-11-14 15:39:42,203 cisTarget INFO Annotating motifs for Dendritic_cells
2023-11-14 15:39:49,757 cisTarget INFO Getting cistromes for Dendritic_cells
2023-11-14 15:39:50,532 cisTarget INFO Running cisTarget for FCGR3A+_Monocytes which has 28776 regions
2023-11-14 15:40:21,710 cisTarget INFO Annotating motifs for FCGR3A+_Monocytes
2023-11-14 15:40:29,709 cisTarget INFO Getting cistromes for FCGR3A+_Monocytes
2023-11-14 15:40:30,598 cisTarget INFO Running cisTarget for NK_cells which has 7830 regions
2023-11-14 15:40:53,193 cisTarget INFO Annotating motifs for NK_cells
2023-11-14 15:40:56,845 cisTarget INFO Getting cistromes for NK_cells
2023-11-14 15:40:57,257 cisTarget INFO Done!
2023-11-14 15:40:57,259 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_DARs_All folder already exists.
2023-11-14 15:40:57,765 pycisTarget_wrapper INFO Running cisTarget without promoters for DARs
2023-11-14 15:41:08,368 cisTarget INFO Running cisTarget for B_cells which has 9314 regions
2023-11-14 15:41:29,938 cisTarget INFO Annotating motifs for B_cells
2023-11-14 15:41:33,897 cisTarget INFO Getting cistromes for B_cells
2023-11-14 15:41:34,334 cisTarget INFO Running cisTarget for B_cells_1 which has 27323 regions
2023-11-14 15:42:05,208 cisTarget INFO Annotating motifs for B_cells_1
2023-11-14 15:42:13,667 cisTarget INFO Getting cistromes for B_cells_1
2023-11-14 15:42:14,726 cisTarget INFO Running cisTarget for B_cells_2 which has 10305 regions
2023-11-14 15:42:39,156 cisTarget INFO Annotating motifs for B_cells_2
2023-11-14 15:42:43,544 cisTarget INFO Getting cistromes for B_cells_2
2023-11-14 15:42:43,976 cisTarget INFO Running cisTarget for CD14+_Monocytes which has 26947 regions
2023-11-14 15:43:15,330 cisTarget INFO Annotating motifs for CD14+_Monocytes
2023-11-14 15:43:23,399 cisTarget INFO Getting cistromes for CD14+_Monocytes
2023-11-14 15:43:24,298 cisTarget INFO Running cisTarget for CD4_T_cells which has 12957 regions
2023-11-14 15:43:45,432 cisTarget INFO Annotating motifs for CD4_T_cells
2023-11-14 15:43:50,037 cisTarget INFO Getting cistromes for CD4_T_cells
2023-11-14 15:43:50,611 cisTarget INFO Running cisTarget for CD8_T_cells which has 7977 regions
2023-11-14 15:44:13,290 cisTarget INFO Annotating motifs for CD8_T_cells
2023-11-14 15:44:17,270 cisTarget INFO Getting cistromes for CD8_T_cells
2023-11-14 15:44:17,831 cisTarget INFO Running cisTarget for Dendritic_cells which has 22578 regions
2023-11-14 15:44:46,802 cisTarget INFO Annotating motifs for Dendritic_cells
2023-11-14 15:44:53,749 cisTarget INFO Getting cistromes for Dendritic_cells
2023-11-14 15:44:54,454 cisTarget INFO Running cisTarget for FCGR3A+_Monocytes which has 26272 regions
2023-11-14 15:45:24,249 cisTarget INFO Annotating motifs for FCGR3A+_Monocytes
2023-11-14 15:45:31,902 cisTarget INFO Getting cistromes for FCGR3A+_Monocytes
2023-11-14 15:45:32,802 cisTarget INFO Running cisTarget for NK_cells which has 6644 regions
2023-11-14 15:45:53,916 cisTarget INFO Annotating motifs for NK_cells
2023-11-14 15:45:57,420 cisTarget INFO Getting cistromes for NK_cells
2023-11-14 15:45:57,806 cisTarget INFO Done!
2023-11-14 15:45:57,809 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_DARs_No_promoters folder already exists.
2023-11-14 15:45:58,552 pycisTarget_wrapper INFO Running DEM for DARs
2023-11-14 15:45:58,553 DEM INFO Reading DEM database
2023-11-14 15:48:03,119 DEM INFO Creating contrast groups
2023-11-14 15:48:14,218 DEM INFO Computing DEM for B_cells
2023-11-14 15:48:30,338 DEM INFO Computing DEM for B_cells_1
2023-11-14 15:49:35,666 DEM INFO Computing DEM for B_cells_2
2023-11-14 15:49:56,010 DEM INFO Computing DEM for CD14+_Monocytes
2023-11-14 15:51:09,374 DEM INFO Computing DEM for CD4_T_cells
2023-11-14 15:52:25,022 DEM INFO Computing DEM for CD8_T_cells
2023-11-14 15:52:40,935 DEM INFO Computing DEM for Dendritic_cells
2023-11-14 15:53:25,143 DEM INFO Computing DEM for FCGR3A+_Monocytes
2023-11-14 15:54:46,429 DEM INFO Computing DEM for NK_cells
2023-11-14 15:55:00,798 DEM INFO Forming cistromes
2023-11-14 15:55:03,233 DEM INFO Done!
2023-11-14 15:55:03,240 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_DARs_All folder already exists.
2023-11-14 15:55:03,732 pycisTarget_wrapper INFO Running DEM without promoters for DARs
2023-11-14 15:55:14,195 DEM INFO Creating contrast groups
2023-11-14 15:55:16,705 DEM INFO Computing DEM for B_cells
2023-11-14 15:55:44,133 DEM INFO Computing DEM for B_cells_1
2023-11-14 15:56:42,689 DEM INFO Computing DEM for B_cells_2
2023-11-14 15:57:13,772 DEM INFO Computing DEM for CD14+_Monocytes
2023-11-14 15:58:08,363 DEM INFO Computing DEM for CD4_T_cells
2023-11-14 15:58:35,800 DEM INFO Computing DEM for CD8_T_cells
2023-11-14 15:58:53,365 DEM INFO Computing DEM for Dendritic_cells
2023-11-14 15:59:41,567 DEM INFO Computing DEM for FCGR3A+_Monocytes
2023-11-14 16:00:37,366 DEM INFO Computing DEM for NK_cells
2023-11-14 16:00:56,239 DEM INFO Forming cistromes
2023-11-14 16:00:58,656 DEM INFO Done!
2023-11-14 16:00:58,663 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_DARs_No_promoters folder already exists.
2023-11-14 16:00:59,267 pycisTarget_wrapper INFO Saving object

@SeppeDeWinter
Copy link
Collaborator

Hi @Chrisdoan9

I don't see any errors here.
I guess you are still running out of memory ...

All the best,

Seppe

@Chrisdoan9
Copy link
Author

Hi @SeppeDeWinter,

Thank you for your suggestion. I put 1024Gb in memory, so may I know how much memory we need to run?

@SeppeDeWinter
Copy link
Collaborator

Hi @Chrisdoan9

1 Tb of memory should definitely be enough... you could try to monitor at which step it runs out of memory.

Also, you can try to use the development branch (see: #202) the code there is more memory efficient.

All the best,

Seppe

@Chrisdoan9
Copy link
Author

Hi @SeppeDeWinter

This is what I have in the end of the log file:

[I 16:02:55.296 NotebookApp] Saving file at /project/multiome.ipynb
[I 16:16:57.060 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 0fc1137e-0f27-41a3-a544-4bd674cce75b restarted
[I 16:18:55.288 NotebookApp] Saving file at /project/multiome.ipynb

Would you please tell me how to run the development branch? Thank you.
https://github.com/aertslab/scenicplus/tree/development

@SeppeDeWinter
Copy link
Collaborator

Hi @Chrisdoan9

Please refer to #202 for the use of the development branch.
You can post question in that discussion thread and I will be happy to answer them.

All the best,

Seppe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@SeppeDeWinter @Chrisdoan9