Mavic 3M multispectral alignment issues

[ehallein]

How did you install ODM? (Docker, installer, natively, ...)?

Docker

What is the problem?

Multispectral bands are not aligned in orthophoto

What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible.

aligned bands

How can we reproduce this? What steps did you do to trigger the problem? If this is an issue with processing a dataset, YOU MUST include a copy of your dataset AND task output log, uploaded on Google Drive or Dropbox (otherwise we cannot reproduce this).

console.txt
output and images https://drive.google.com/file/d/1p2bUUqzd-TxL-kPQttbjuv-PtB49WtzS/view?usp=sharing

create project in webODM (using nvidia GPU) and run using multispectral option on TIFFs

possibly related to https://www.reddit.com/r/UAVmapping/comments/1cacp09/processing_multispectral_dji_mavic_3m_data_with/
"First, locate the fields 'Relative Optical Center X' and 'Relative Optical Center Y' in the EXIF information of each band's photograph. These values represent the positional offset of each camera band relative to the NIR band camera (in pixels). By setting the compensation values (offset) for each band's photograph (excluding the NIR band) in software like ENVI, you can correctly combine the images. The specific method for setting the compensation values for each band will need to be referenced from the manual of ENVI or similar software."

[pierotofy]

#1740

[ehallein]

#1740

That's seems to be related to differences is the RGB vs multispectral lenses? There no RGB being used in my case, just the multispectral bands.

[smathermather]

Thanks for opening the issue.

That's seems to be related to differences is the RGB vs multispectral lenses? There no RGB being used in my case, just the multispectral bands.

The problem is sufficiently related to be considered the same: either we choose a multispectral band to align to, as per @MarioTro's example here: https://gitlab.com/Yario/image_registration_dji_mavic_3m/-/blob/main/coregistration_mavic_3m_images_with_CV2.py?ref_type=heads or we align to the RGB per @lewispb's example here: https://gist.github.com/lewispb/6c7dd6022ba3266cfd4f1310d4d51f5f

In the second scenario, the exif data for that alignment is extant in the multispectral bands, so it wouldn't require RGB upload with the MS bands, so it should be independent of how exactly the data are processed.

[pierotofy]

This should be fixed with #1771

The problem was multifaceted:

• A fault in how the homography matrices were computed on images larger than 1280 pixels
• The RGB lens model being too different than the multispectral ones

Both should be addressed but some testing will be needed.

[ehallein]

Excellent. Happy to help test.

[ehallein]

This should be fixed with #1771

The problem was multifaceted:

• A fault in how the homography matrices were computed on images larger than 1280 pixels
• The RGB lens model being too different than the multispectral ones

Both should be addressed but some testing will be needed.

looks much better with now with the new code. although still a slight offset with redband compared to G,NIR and Red edge which all line up.

[pierotofy]

@ehallein could you post your task processing log?

[ehallein]

@pierotofy
here you go
console.txt

[pierotofy]

Thanks!

I don't think it might matters in this case, but always re-run from the dataset stage between version updates.

Mm, not sure why the red band ended up off. A link to the dataset (or better, a smaller subset) would help (the link on google no longer works?).

[ehallein]

Thanks!

I don't think it might matters in this case, but always re-run from the dataset stage between version updates.

Mm, not sure why the red band ended up off. A link to the dataset (or better, a smaller subset) would help (the link on google no longer works?).

here are TIFFs https://drive.google.com/file/d/1OEulM3PKqwOrRDX9oCg54osxpzyJNoTg/view?usp=sharing
The restart was just a redo to test. I've run a few times, including both mac (m1 pro) and windows with same issue. I'm trying some other datasets now.

[ehallein]

Here's image from another dataset. Still out. Only red band. But it's pretty close, so maybe it's just the error on the lens data.

[smathermather]

Definitely seeing some minor misalignment issues.

https://crankyserver.com/map/project/350/task/ff48c492-0e67-4613-bc31-03b2367e8aa6/

[ehallein]

I just tested the same dataset using DJI Terra and it is worse

[Saijin-Naib]

They may have all the internal docs and specs, but we have a Piero 🤣

[smathermather]

They may have all the internal docs and specs, but we have a Piero 🤣

🤣 Indeed. It seems there's value to calculating homography even if you have specs from the factory.

[pierotofy]

Can this be closed?

[ehallein]

Yes seems to be well aligned on my tests