Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update geo-inference parameters #579

Merged
merged 16 commits into from
Oct 11, 2024
Merged

Conversation

mpelchat04
Copy link
Collaborator

Description

This PR updates the parameters from geo-inference (workers, prediction_threshold, patch_size).
It also reduces the size of the docker image created.

Type of change

What types of changes does your code introduce? Put an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Other

How Has This Been Tested?

  • Inference integration test
  • On HPC, on multiple images using the image tst_gdl_3_1.sif

Checklist:

If you're unsure about any of these, don't hesitate to ask. We're here to help!

  • [X ] My code follows the code style of this project
  • [X ] I have performed a self-review of my own code
  • [X ] My PR has a descriptive title
  • [X ] My code follows PEP8 (most of it)
  • [X ] My code have the proper documentation in the code
  • [X ] I have made corresponding changes to the documentation (if applicable)
  • [X ] My changes generate no new errors
  • [X ] My code is covered by the existing tests (if applicable)
  • My code includes tests that show that my feature works
  • My PR passes Github Actions

@mpelchat04 mpelchat04 changed the title Develop Update geo-inference parameters Sep 6, 2024

RUN apt-get update \
&& apt-get install -y --no-install-recommends git wget unzip bzip2 build-essential sudo \
&& apt-key del 7fa2af80 \
&& wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.0-1_all.deb \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont we need these cuda keyring stuff anymore?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. Tested with a local docker image, the CI test image and on HPC. All works fine without these.

CharlesAuthier
CharlesAuthier previously approved these changes Sep 10, 2024
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks to keep the doc alive

from tqdm import tqdm
from pathlib import Path
from numbers import Number
from tempfile import mkstemp
from omegaconf import DictConfig
from typing import Dict, Sequence, Union
from dataset.stacitem import SingleBandItemEO
import rasterio
from shutil import move
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we start organizing module loading according to PEP8? i.e. system libraries in the first block, external in the second, locally written libraries in the third.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed it in this script.
Following the refactoring of all the other modules, this will be done in the other module as well.

@@ -1,15 +1,16 @@
import csv
from math import sqrt
import rasterio

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should look like this:

from shutil import move
from pathlib import Path
from numbers import Number
from tempfile import mkstemp
from typing import Dict, Sequence, Union

from tqdm import tqdm
import csv
import rasterio
from omegaconf import DictConfig

from utils.aoiutils import aois_from_csv
from dataset.stacitem import SingleBandItemEO
from utils.logger import get_logger, set_tracker
from geo_inference.geo_inference import GeoInference
from utils.utils import get_device_ids, get_key_def, set_device

but I think that PEP8 refactoring can be done later in a separate PR.

ms888ekb
ms888ekb previously approved these changes Sep 17, 2024
CharlesAuthier
CharlesAuthier previously approved these changes Oct 3, 2024
@mpelchat04
Copy link
Collaborator Author

@CharlesAuthier and @ms888ekb I added the support for Test-Time augmentation and corrected the model for the tests to pass.

Final review, I swear. :)

- pytest>=7.1
- python>=3.11
- pytorch>=2.3
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need pytorch and pytorch-cuda at the same time? pytorch-cuda can work on CPU if needed. Won't having two different versions cause conflicts?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the recommended way to install pytorch in a conda env.: https://pytorch.org/

@mpelchat04 mpelchat04 merged commit b0a2b73 into NRCan:develop Oct 11, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants