This repository is the third part of the FAIRification pipeline and is responsible for uploading metadata to data.bbmri.cz catalogue.

Supported sequencing types

Miseq, New Miseq, MammaPrint

How to run the scripts

Locally - Development

Using main.py

1. Install requirements

pip install -r requiremenents.txt

2. Run main.py

python main.py -r path/to/pseudonymized/runs/folder  -o /path/to/root/organisation/folder -p /path/to/patients/folder

Using docker-compose

docker-compose up -d --build

In production

Production is running on Kubernetes cluster SensitiveCloud

Using kubernetes (kubectl)

Deploy dependent secrets

## Supported sequencing types
Miseq, New Miseq, MammaPrint

## How to run the scripts
### Locally - Development
#### Using main.py
1. Install requirements
```bash
pip install -r requiremenents.txt

2. Run main.py

python main.py -r path/to/pseudonymized/runs/folder  -o /path/to/root/organisation/folder -p /path/to/patients/folder

Using docker-compose

docker-compose up -f compose.yml -d --build

In production

Production is running on Kubernetes cluster SensitiveCloud

Using kubernetes (kubectl)

Deploy dependent secrets

kubectl apply -f kubernetes/catalog-secret.yaml -n bbmri-mou-ns

kubectl apply -f kubernetes/uploader-job.yaml -n bbmri-mou-ns

Deploying new version in production

Build new docker image

docker build --no-cache <public-dockerhub-repository>/data-catalogue-organiser:<version> .
docker push <public-dockerhub-repository>/data-catalogue-organiser:<version> 
# change version in kubernetes/organiser-job.yaml

Debigging

You can visit kubernetes UI Rancher find the failing pod and investigate in logs. On how to use Rancher and SensitiveCloud visit Docs

Other option is running a testing job and investigation inside the cluster filemanager (to check user permissions etc.)

kubectl apply kubectl apply -f kubernetes/testing-job.yaml -n bbmri-mou-ns

Then connect to terminal of this job/pod on Rancher

kubectl apply -f kubernetes/organiser-job.yaml -n bbmri-mou-ns

Deploying new version in production

Build new docker image

docker build --no-cache <public-dockerhub-repository>/data-catalogue-uploader:<version> .
docker push <public-dockerhub-repository>/data-catalogue-uploader:<version> 
# change version in kubernetes/organiser-job.yaml

Debigging

You can visit kubernetes UI Rancher find the failing pod and investigate in logs. On how to use Rancher and SensitiveCloud visit Docs

Other option is running a testing job and investigation inside the cluster filemanager (to check user permissions etc.)

kubectl apply kubectl apply -f kubernetes/testing-job.yaml -n bbmri-mou-ns

Then connect to terminal of this job/pod on Rancher