Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Girder Large Image and Tiled Image Support #1255

Merged
merged 29 commits into from
Sep 14, 2023
Merged

Conversation

BryonLewis
Copy link
Collaborator

@BryonLewis BryonLewis commented Jun 1, 2022

resolves #1253

Features/Code Implemented:

  • Installs girder-large-image and dependencies - Uses a series of wheels, I couldn't find the poetry equivalent of --find-links for pip install. There is this PR (Equivalent behaviour to pip install --find-links python-poetry/poetry#1511), but it seems the still preferred way is to use direct links to the wheels.
  • Implements a new DatasetType: 'large-image' - automatically converts NTFs, NITFs, tiffs and R# files to large image data. This new type can be uploaded using the data importer
  • LargeImageAnnotator.vue - new viewer which loads and displays large-image. Required some changes to the useMediaAnnotator loading process to work properly. Also, have some commented-out code to enable an Openstreetmaps background on large images that have geospatial images. The problem with that is I didn't implement a transform from local image-space coordinates to global (lat/lons), so commenting it out will cause annotations to not work.
  • Handles the uploading of large image new file datasets as a large-image type in the client
  • Disables running of pipelines/training on large-image datasets in the viewer and the data browser
  • If a regular PNG/JPG image is larger than 8k in any dimension it will notify the user and suggest they convert to large-image. This was chosen because it's a typical max texture size for clients. Video Cards that aren't Nvidia Quadro-level workstations have lower single-texture resolutions that can be rendered. We use a single texture for the image in geoJS so this will automatically convert it to a large image so it can be displayed in the client.
  • I fixed an issue that was bugging me where when loading a single image the UI for the timeline wouldn't look right. I updated it so that tracks and detection counts span the entire width of the timeline if there is one image. This makes more sense when utilizing large-images where there may be only a single image.
  • Used the layer-swapping method of GeoJS for traversing through sequential large images. It loads up one image and then prefetches the next frame in the background so that playback can be a bit smoother. You aren't going to get 10fps but being able to do around 1fps depending on zoom level and data size helps.

Testing Procedure:

Testing Dataset zip file(https://viame.kitware.com/api/v1/item/63481282828794fdd97cc03c/download) - warning it is ~1.8GB.

Standard Large Image Testing:

  • do a full docker-compose build --no-cache to build the system with the new poetry dependencies
  • Test the new uploading of large images. Right now that includes these formats: https://github.com/Kitware/dive/pull/1255/files#diff-a2265279f516b5cd723f8575c4e1133065c2e5a253d9a38e0abe442619b8a13bR29-R44. The R# formats are used to display some level information for nitfs I've seen in the past. The one thing I don't have a good way to determine is the difference between a non-pyramidal tiff vs a regular tiff. For this first test I would use of the single files in my test package like Seattle or Anchorage.
  • Let me know if you know of a better term for this button:
    image
  • After loading the image check that you can see it and create/save annotations.
  • Confirm in the localhost:8010/girder interface that the metadata is set correctly and that the type is 'large-image'
  • Pipelines should be disabled both in the Annotation viewer and the data browser when large-image datatypes are selected.
  • Note specifically the change in the timeline/event viewer for annotations. Before single frame datasets wouldn't display the detection count or the events in the viewer. The new update view shows them as being across the full frame

Sequenced Large Image Testing:

  • Within the sample dataset folder I have a subfolder called 'Sequence' which has a sequence of NITFs that can be loaded into the system.
  • This should be similar to image-sequence except that transitioning between frames is going to take much longer. There is buffer of one frame look ahead to ease the transition but when you randomly seek around it will require some loading and the loading indicator should be displayed showing that.

Auto Conversion of Large JPG/TIF/PNG Images:

  • Within the testing zip is a folder called 'large_jpgs' which includes several JPG files that are larger than 8k in either width or height. On regular graphics cards these won't render properly in the annotation viewer.
  • Import these as a standard image first and then launch the dataset.
  • A pop-up asking to convert it to a large-image format will be shown. Accepting that will kick off a quick job which will convert the type of the dataset to large-image and allow viewing for all people these images.

TODOs:

  • Integration testing for uploading and importing large images. Will need to update the zip file containing integration test source data as well as update the tests themselves to handle the large image.
  • Documentation Updating: Supported Formats and types, some information about Auto Conversion of large JPGS/PNGS
  • Pipeline testing: For now I've disabled pipelines on all large-images even though theoretically large JPGS/PNGS and non-tiled TIFFs should work.

@BryonLewis BryonLewis marked this pull request as ready for review October 13, 2022 15:51
@BryonLewis BryonLewis changed the title [WIP] Tiled images in DIVE Add Girder Large Image and Tiled Image Support Oct 13, 2022
@BryonLewis BryonLewis requested a review from marySalvi October 13, 2022 15:59
@marySalvi
Copy link
Collaborator

It looks good with some minor issues:

  1. As discussed the Seatle tif crashes girder.
  2. Jumping around in the image sequence can leave part of the previous image visible. Specifically from first frame to last frame.

Screenshot from 2022-10-25 09-54-40

  1. There seems to be some weird UI issues in the upload dialogue.

Screenshot from 2022-10-25 09-58-00

  1. Uploading the large jpgs and converting seems to work but there are 500 errors when interacting with the dataset.

Screenshot from 2022-10-25 10-09-32

url = `${url}?projection=${encodeURIComponent(projection)}`;
}
const { data } = await girderRest.get(url);
// const { data } = await girderRest.get(`item/${itemId}/tiles?projection=EPSG%3A3857`);
Copy link
Collaborator

@marySalvi marySalvi Oct 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment can probably be removed

@BryonLewis BryonLewis force-pushed the tiled-image-support branch 17 times, most recently from 7931a75 to 6c1ed32 Compare September 11, 2023 16:43
marySalvi
marySalvi previously approved these changes Sep 13, 2023
Copy link
Collaborator

@marySalvi marySalvi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Maybe add that tooltip but other than that I think this is gtg

</v-list-item>
<v-list-item>
<import-button
:name="`Add ${pendingUploads.length ? 'Another ' : ''}Tiled Images`"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add tooltip that indicates acceptable file types and that it can be a sequence too. Users might try to use the other sequence option otherwise.

@BryonLewis
Copy link
Collaborator Author

I merged it with main to add in the attribute rendering and also added in the tooltip you mentioned.

Copy link
Collaborator

@marySalvi marySalvi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

::shipit::

@BryonLewis BryonLewis merged commit 8671a09 into main Sep 14, 2023
4 checks passed
@BryonLewis BryonLewis deleted the tiled-image-support branch September 14, 2023 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Web based large image support
2 participants