Show date of aerial data

[hajo4]

When mapping in areas where demolition & construction is taking place,
and the map is different from the image, it would be useful to see
the date of aerial images shown.

[jfirebaugh]

I suppose we could special case this for Bing, which has an API that purports to provide vintage, but in general our imagery sources don't come with this data.

[pnorman]

Do the Bing images themselves come with metadata in the response headers?

As a host of imagery layers, I'd like to provide this information.

[bryceco]

The Bing response includes a line like this:
X-VE-TILEMETA-CaptureDatesRange: 7/1/2004-7/31/2004

[cgiovando]

👍 Would be nice to be able to pass some imagery metadata automatically, especially when adding custom background imagery (WMS/TMS from HOT-OSM Tasking Manager) and automatically add a source=* and source:date=YYYY-MM-DD to whatever element is created or modified with that specific imagery layer turned on.

[jfirebaugh]

Even if the image tile comes with response headers indicating the vintage, we can't make use of them -- there's no way to get response headers for an <img> element. The only way to do this is to make a separate request: either an ajax request for the same image (doubling the transfer cost for background imagery) or to another API that provides vintage in another format (hopefully for all tiles in a given area, so that we can make a single request).

[pnorman]

Based on experience, the metadata for the middle is sufficient. The per-tile info is not perfectly accurate at imagery boundaries anyways.

We should also make use of editor-imagery-index supplied dates.

[aawiseman]

Hi @bhousel I talked with @gmaclennan about this, he has worked on something similar, here's what he had to say:

I had some code from a Leaflet Bing Layer plugin that I created for parsing metadata from Bing imagery. That code is here:

https://github.com/gmaclennan/leaflet-bing-layer/blob/gh-pages/index.js#L160-L246

I mentioned this because from what somebody said at Sotmus I thought that iD did not currently display imagery date for Bing, but I see that it does actually display some metadata which you can see here:

iD/modules/renderer/background_source.js

Lines 89 to 133 in af77650

	BackgroundSource.Bing = function(data, dispatch) {
	// http://msdn.microsoft.com/en-us/library/ff701716.aspx
	// http://msdn.microsoft.com/en-us/library/ff701701.aspx
	data.template = 'https://ecn.t{switch:0,1,2,3}.tiles.virtualearth.net/tiles/a{u}.jpeg?g=587&mkt=en-gb&n=z';
	var bing = BackgroundSource(data),
	key = 'Arzdiw4nlOJzRwOz__qailc8NiR31Tt51dN2D7cm57NrnceZnCpgOkmJhNpGoppU', // Same as P2 and JOSM
	url = 'https://dev.virtualearth.net/REST/v1/Imagery/Metadata/Aerial?include=ImageryProviders&key=' +
	key + '&jsonp={callback}',
	providers = [];
	d3.jsonp(url, function(json) {
	providers = json.resourceSets[0].resources[0].imageryProviders.map(function(provider) {
	return {
	attribution: provider.attribution,
	areas: provider.coverageAreas.map(function(area) {
	return {
	zoom: [area.zoomMin, area.zoomMax],
	extent: Extent([area.bbox[1], area.bbox[0]], [area.bbox[3], area.bbox[2]])
	};
	})
	};
	});
	dispatch.change();
	});
	bing.copyrightNotices = function(zoom, extent) {
	zoom = Math.min(zoom, 21);
	return providers.filter(function(provider) {
	return _.some(provider.areas, function(area) {
	return extent.intersects(area.extent) &&
	area.zoom[0] <= zoom &&
	area.zoom[1] >= zoom;
	});
	}).map(function(provider) {
	return provider.attribution;
	}).join(', ');
	};
	bing.logo = 'bing_maps.png';
	bing.terms_url = 'https://blog.openstreetmap.org/2010/11/30/microsoft-imagery-details';
	return bing;
	};

It looks like iD is not currently reading metadata specific to the current center point, which my Leaflet plugin does. I'm not sure how this data is different, or whether it is useful - it's a while since I worked on this. The relevant docs from Bing are linked in my code comments.

[bhousel]

Nice! Thanks @aawiseman for pointing me at @gmaclennan's code... We could definitely perform that jsonp query to fetch the bing metadata.

[dalekunce]

👍 for this feature.

[bhousel]

Coming soon!
Added the code today to get the vintage from Bing's REST service, and use whatever is in editor-layer-index for all other sources..

[cgiovando]

Now, can we inject that automatically to the source tag of any feature that is created off of the image layer being displayed? ;)

[bhousel]

Now, can we inject that automatically to the source tag of any feature that is created off of the image layer being displayed? ;)

We could add source tags to everything, but it might upset some people. I regularly get people on here complaining about tags created by iD that they don't like looking at.

In fact when I map in New Jersey, I mostly use Bing even though the NJ 2015 Aerials are newer, simply because the Bing server loads much faster. I do swap between the two layers with ⌘B whenever I want a second look at something.

My next thing I want to sneak into this panel - if it doesn't take too long - is #1124 - some support for the imagery offset database.

[cgiovando]

We could add source tags to everything, but it might upset some people. I regularly get people on here complaining about tags created by iD that they don't like looking at.

I don't see why it should upset anyone. It would add a very useful piece of metadata that is often just left blank. Even if shown in the overlay box that you just added (:+1:), I don't think many people would take the time to copy and paste into the source tag of every feature they map.

Adding it directly, as I'm suggesting, would be a major improvement in quality management of the map data. We should suggest adding the same function to JOSM as well 😉

Personally I would go even further, and advocate for making the source and source:date tags mandatory for any feature in OSM!

[magol]

Is it any plans to get the vintage for other aerial images to? Is it any standard (or best practices) that providers of aerial images can use to give that possibility? If not, it is anything that should be created?

[bhousel]

@cgiovando:

I don't see why it should upset anyone. It would add a very useful piece of metadata that is often just left blank. Even if shown in the overlay box that you just added (👍), I don't think many people would take the time to copy and paste into the source tag of every feature they map.

Maybe ask on the OSM mailing lists to see what people think about this? I don't have any strong opinion about it, but I know in my own mapping I probably wouldn't take the time to adjust the source tags on each new feature that I add or update.

@magol:

Is it any plans to get the vintage for other aerial images to? Is it any standard (or best practices) that providers of aerial images can use to give that possibility? If not, it is anything that should be created?

If we can get the vintage from the other providers, I'll add code to do so. I think DigitalGlobe maps API might provide this information for the Standard layer (@kevinbullock may know?). DigitalGlobe Premium and Mapbox Satellite layer do not currently provide imagery vintage, as they are composites of several sources. (But they could eventually provide a date range the way that Bing does).

Here is another cool screenshot - I'm adding imagery vintage to the tile debugging, and I'll add a button to toggle this display:

[boothym]

@bhousel Just to let you know, using Ctrl+Shift+B as described in the keyboard shortcuts swaps between the background like when using Ctrl+B.

Also, your changelog shows ⌘⇧B - you should really add the Windows shortcut as well 😉

[magol]

In the next version, it should also be any way to toggle this and the other panels even if I don't remember the shortcuts. The panels is great, but I always forgot the shortcuts. Maybe create a separate issue for that?

[bhousel]

@bhousel Just to let you know, using Ctrl+Shift+B as described in the keyboard shortcuts swaps between the background like when using Ctrl+B.

Thanks @boothym can you open up a new issue for this? The fix is probably to stopPropagation once the event is handled, so Ctrl+B doesn't get a chance to happen.

Also, your changelog shows ⌘⇧B - you should really add the Windows shortcut as well 😉

Good suggestion - I agree, the changelog should show both. I try to write release notes for a broader audience now, since new releases link to it via the 🎁 icon.

[bhousel]

In the next version, it should also be any way to toggle this and the other panels even if I don't remember the shortcuts. The panels is great, but I always forgot the shortcuts. Maybe create a separate issue for that?

It's a good suggestion but I don't want to add buttons to the main iD user interface for these advanced features. iD's main strength is its simplicity, and the panel shortcuts are discoverable from the keyboard shortcuts screen ?.

I have considered adding a button bar that only appears if the user has enabled the panels. This is partly because I'd like to better support mobile/tablet users someday. Right now I'm still waiting to see how this feature evolves, whether people use it, whether we add more of them, etc.

[d3netxer]

I did not know this feature existed. It is very useful. I think it is even more useful now that it has been implemented to work with the ESRI World Imagery layer as well. I often get questions of users asking if there is a way to tell when the date when the imagery was collected. I would like to see the date of the imagery in the bottom bar (along where the scale is and the usernames are displayed). Or maybe in the bottom bar there can be a metadata icon that will display the metadata panel. Metadata is important! @bhousel @magol

[tordans]

@bhousel did this feature (showing the date of the image) make it to the live system? I cannot find it in the editor.

I am in a HOT task and wondering if a building was demolished or build from one image to the next.

[bhousel]

did this feature (showing the date of the image) make it to the live system? I cannot find it in the editor.

Yes it was merged to iD about a year ago.. Try using the Ctrl+Shift+B / Command+Shift+B key combo mentioned above to bring up the background info panel.. any luck?

[tordans]

Yes, thanks, thats great.
For this task there does not seem to be any year-information unfortunately :).
I will need to spent a bit more time with this info layer to understand it, there seems to be quite a few states.

Btw, this image date feature might also be a thing to consider for osmlab/osm-planning#18.

[boothym]

@bhousel For me in the UK, the date shows up on Bing, but not anymore for the Esri imagery - something has changed recently.

The DG imagery has a "show vintage" button which then requires you to zoom out to see the yellow boxes/text, but it's always been like that.

[bhousel]

For me in the UK, the date shows up on Bing, but not anymore for the Esri imagery - something has changed recently.

Thanks @boothym it looks like openstreetmap.org needed a rule added to allow the connection to the Esri metadata servers. @tomhughes fixed this today - thanks Tom!

[bgirardot]

ETA: Could this be related to DGP switching to transparent tiles at z18+ ? Sorry just ran into a similar thing in JOSM, my bing layer showed through, but this is in iD which should not have that problem.

Hi all, Kevin Bullock suggested I put my issue in this thread, but I am not 100% sure this is the best place.

My issue is that when using DG Premium imagery, I can see the date using the imagery info panel and vintage overlay.

However when I zoom in, the imagery clearly changes to older imagery for DG Premium (DGP) which is not reflected in the vintage information. this happens in a lot of places around the world for the DGP layer. It of course also happens in other editors besides iD (well, it happens in JOSM at least too).

You can see that ESRI World Imagery, which is the same the DGP layer does not do this and stays the same imagery even at z18 and higher.

Please see below.

DGP z17 is probably the 2017 the vintage layer says:

However, when I zoom into the z18 you can see the imagery clearly changes to what I am pretty sure is older imagery as significant development disappears without a trace. This is no longer the 2017 imagery the vintage layer indicates:

You can see that Esri World imagery, which is the same as the DGP imagery, stays the same even at z18 and higher:

This really makes it impossible to date the DGP imagery that is actually used at a comfortable mapping z lvl and makes the DGP vintage layer incorrect as it is only for the z17 and lower z lvls.

[bhousel]

hey @bgirardot - the best place to discuss the Digital Globe vintage might be here: osmlab/editor-layer-index#371

There is some more info about them on that thread, and @marracci might be able to answer specific questions. If the imagery changes when you zoom in, then you're right that it's kind of impossible to date it this way.

We all see these raster layers as kind of a "beta" feature, with the desired goal to either have a metadata server to query for more accurate results, or maybe store the metadata in vector tiles.

[bhousel]

also osmlab/osm-planning#18 has some more thoughts. I'd really like to build something to solve this!

[tordans]

did this feature (showing the date of the image) make it to the live system? I cannot find it in the editor.

Yes it was merged to iD about a year ago.. Try using the Ctrl+Shift+B / Command+Shift+B key combo mentioned above to bring up the background info panel.. any luck?

@bhousel I don't manage to see the image-dates for this are https://www.openstreetmap.org/edit?editor=id&#map=19/-4.44157/37.16185 - not for DG sources or Esri. I am pretty sure I tried all possible shortcut combinations (this time ;)).

I cannot see any broken request in the network panel, so maybe I am really doing it wrong cause no API is requested? Or is it something else?

Apart from the shortcut, is there a clickable way to activate this layer/info?

[bhousel]

@bhousel I don't manage to see the image-dates for this are https://www.openstreetmap.org/edit?editor=id&#map=19/-4.44157/37.16185 - not for DG sources or Esri. I am pretty sure I tried all possible shortcut combinations (this time ;)).

Ah yeah I don't see them either. It looks like 2 separate problems.

For the Esri layers, it would show the vintage on the Background Info Panel, and also on the tiles themselves, but apparently not working now.

For the DG layers, you need to click "Show Vintage" and it will show some raster overlays. These are supposed to have the vintage dates on them, but they don't have them.

[d3netxer]

I don't see image dates for Bing either.

[quincylvania]

@d3netxer If the dates aren't showing up in iD then it's most likely that they're not provided by Bing.

[d3netxer]

Thanks. I now checked different cities around the world (Madrid, Paris, London, New York City, Lima, Sydney) and it is displaying at multiple zoom levels 'Vintage:Unknown'.