Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ability to upload a file and use it as a non-editable sequence for an assembly #406

Closed
garrettjstevens opened this issue Jul 2, 2024 · 0 comments · Fixed by #432
Closed

Add ability to upload a file and use it as a non-editable sequence for an assembly #406

garrettjstevens opened this issue Jul 2, 2024 · 0 comments · Fixed by #432
Assignees
@dariober

Comments

@garrettjstevens
Copy link
Contributor

Currently we have the options to

  • upload a file (GFF3 or FASTA) and use it to load an assembly with sequence in the database
  • use an external URL to create an assembly that doesn't store the sequence in the database

I think we need a third option:

  • upload a file (FASTA) and use it to create an assembly that doesn't store the sequence in the database

The steps I see are:

  • Allow user to upload either a FASTA/FAI or FASTA.GZ/FAI/GZI set of files.
  • Store the file IDs in the assembly (modify assembly.schema.ts) similar to the externalLocation in the assembly (perhaps called fileLocation).
  • In sequence.service.ts, check to see if the assembly has a fileLocation and if it does, fetch the sequence from the file similar to how the externalLocation fetches from a URL.
dariober added a commit that referenced this issue Jul 22, 2024
@dariober
Handling of gzip input and bgzip'd & indexed fasta
587f67b 
* CLI can upload and add assembly from gzip files (#405 and #407)

* CLI can add assembly in non-editable mode with `--no-db` flag, i.e. without loading
  sequence to mongodb (#406). Only bgzip'd & indexed fasta file are
  supported, not plain fa/fai files.

See tests in test.py: `testFeatureChecksIndexed`, `testFileUploadGzip`,
`testAddAssemblyWithoutLoadingInMongo`
garrettjstevens pushed a commit that referenced this issue Aug 30, 2024
@dariober @garrettjstevens
Handling of gzip input and bgzip'd & indexed fasta
1f59e38 
* CLI can upload and add assembly from gzip files (#405 and #407)

* CLI can add assembly in non-editable mode with `--no-db` flag, i.e. without loading
  sequence to mongodb (#406). Only bgzip'd & indexed fasta file are
  supported, not plain fa/fai files.

See tests in test.py: `testFeatureChecksIndexed`, `testFileUploadGzip`,
`testAddAssemblyWithoutLoadingInMongo`
@garrettjstevens garrettjstevens moved this to In progress in Apollo 3 Sep 16, 2024
@garrettjstevens garrettjstevens mentioned this issue Sep 16, 2024
Merged
garrettjstevens pushed a commit that referenced this issue Sep 20, 2024
@dariober @garrettjstevens
Handling of gzip input and bgzip'd & indexed fasta
2f34028 
* CLI can upload and add assembly from gzip files (#405 and #407)

* CLI can add assembly in non-editable mode with `--no-db` flag, i.e. without loading
  sequence to mongodb (#406). Only bgzip'd & indexed fasta file are
  supported, not plain fa/fai files.

See tests in test.py: `testFeatureChecksIndexed`, `testFileUploadGzip`,
`testAddAssemblyWithoutLoadingInMongo`
garrettjstevens added a commit that referenced this issue Sep 21, 2024
@garrettjstevens @dariober @shashankbrgowda
Add ability for indexed sequence file to be uploaded and used in an a…
89ccf49 
…ssembly (#432)

* Start implementing file upload|download|get

* Add delete command and various small edits

* Add FileCommand handling file operations

* Handle default profile via undefined

* Download file as stream

* Handling of gzip input and bgzip'd & indexed fasta

* CLI can upload and add assembly from gzip files (#405 and #407)

* CLI can add assembly in non-editable mode with `--no-db` flag, i.e. without loading
  sequence to mongodb (#406). Only bgzip'd & indexed fasta file are
  supported, not plain fa/fai files.

See tests in test.py: `testFeatureChecksIndexed`, `testFileUploadGzip`,
`testAddAssemblyWithoutLoadingInMongo`

* Fix linter

* Formatting

* Add description for file CLI topic

* Deduplicate sequence fetching code

* Reference File schema from Assembly schema

* Rename commands

* Remove -i option since file is positional

* Rename option --no-db and some other fixes

* Add additional type and make input arg required

* Tmp commit

* fix app start issue

* fix lint

* Tmp commit addressing #432

Fix #424

* Tmp commit

* This should address comments in #432

But most likely there will be changes to clunky code

* Temp commit to fix buffer

* Fix filehandle type

* Use files service to get filehandles

* Fix fileId vs fileIds and other issues for #432

All tests except those for chekcs passing

* Allow fasta input to be gzip

* Fixes after rebase

* Move LocalFileGzip to collab server package

* Match selector to other updated tests

---------

Co-authored-by: dariober <dario.beraldi@gmail.com>
Co-authored-by: Shashank Budhanuru Ramaraju <shashank.b.r.gowda45587@gmail.com>
@github-project-automation github-project-automation bot moved this from In progress to Done in Apollo 3 Sep 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dariober dariober

Labels
None yet
Projects
Apollo 3
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add ability for indexed sequence file to be uploaded and used in an assembly GMOD/Apollo3
2 participants
@dariober @garrettjstevens