-
Notifications
You must be signed in to change notification settings - Fork 3
Add sentinel-1 RTC STAC metadata generator #1
Conversation
@matthewhanson thoughts on getting this merged? |
Ah sorry @scottyhq , this feel off my radar along with most everything else when I took PTO at the end of the year. All the metadata looks good, but my only concern is getting the metadata from the .tif files, because it's much slower to generate if the files are all remote. Or if they are in a requester pays bucket. So if we wanted to create STAC for the entire sentinel-1-rtc public dataset that's a lot of files that will have to be hit. It sounds like there's some fields that can't be retrieved from just the metadata files? What fields are those? Does it make sense to have it be an option to get extended metadata from the tif files, or are their some required fields in there? |
. | ||
├── Gamma0_VH.tif | ||
├── Gamma0_VV.tif | ||
├── README.txt | ||
├── S1B_IW_GRDH_1SDV_20161121T010910_20161121T010939_003050_0052FC_EC22 | ||
│ ├── manifest.safe | ||
│ └── productInfo.json | ||
├── S1B_IW_GRDH_1SDV_20161121T010939_20161121T011004_003050_0052FC_3426 | ||
│ ├── manifest.safe | ||
│ └── productInfo.json | ||
└── local_incident_angle.tif |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note directory layout w/o a single JSON file that contains metadata necessary for a STAC entry
As far as I could tell there is no single existing metadata file here that simply needs converting to STAC. the only way to get required fields (like the bbox) is to open the tif. Just reading metadata is ~200ms from s3. fortunately the bucket isn't requester pays... currently at least (https://registry.opendata.aws/sentinel-1-rtc-indigo/). it's just CONUS, but i don't see an inventory file. A back of the envelope calculation suggests ~ 228,390 Items (933 grid squares, 230 images to-date), growing at a rate of ~1000 Items per week (each with 3 .tif assets). also could ping someone at indigo-ag but i don't see any activity in their github org related to the rtc dataset... |
@matthewhanson thoughts on merging this and trying things out? would be really nice to have a public SAR STAC to test 1.0.0rc1 |
@matthewhanson - I think this is ready to go, but I've only tested on a single acquisition. So I was thinking it might be good to generate a full static catalog with a couple acquisitions from a few grid squares.
this adds several dependencies (rasterio, geopandas, pystac), mainly because i think the best way to get metadata in this case is not reading a single .json file, but actually getting the metadata stored within .tif files.
i've added some documentation in the readme, would appreciate review at this point to figure out next steps!