Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testing/staging output buckets #44

Closed
jashapiro opened this issue May 22, 2024 · 2 comments · Fixed by #57
Closed

Add testing/staging output buckets #44

jashapiro opened this issue May 22, 2024 · 2 comments · Fixed by #57

Comments

@jashapiro
Copy link
Member

Currently we have only the "production" output buckets specified in the Nextflow workflow, but this is probably not the ideal way to have the defaults set up, as any errors in the workflow could result in corrupted data in the buckets used by other tools/the scpca-analysis scripts and modules .

We should probably change the defaults to some kind of staging location, and copy the results over in a separate step, or with a rerun and the final output bucket specified. This may take making a few more buckets for these purposes.

@jashapiro
Copy link
Member Author

@jaclyn-taroni, now that you are back, I wanted to alert you to this issue. In short, I think it would be a good idea to have some staging buckets to mirror openscpca-nf-workflow-results, openscpca-test-data-release-public-access and openscpca-test-workflow-results-public-access for output from OpenScPCA-nf. The latter two would not need to have public access (unlike the final versions), as they would only be used for workflow tests.

If you agree this is a good idea, I assume we would want to create an issue in https://github.com/AlexsLemonade/OpenScPCA-infra to handle creating those buckets?

@jaclyn-taroni
Copy link
Member

Yes, this makes sense to me. I would expect a ticket in OpenScPCA-infra for creating a bucket with public access blocked with the same permissions for the Nextflow, etc. We can discuss over there what account we think it belongs in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants