Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vector Tile Specification Version 2.0 #39

Merged
merged 22 commits into from
Dec 4, 2015
Merged

Vector Tile Specification Version 2.0 #39

merged 22 commits into from
Dec 4, 2015

Conversation

flippmoke
Copy link
Member

I would like to introduce everyone to the initial draft of the vector tile 2.0 specification. I am really excited about how much more specific the new specification is then the previous version. There is a lot more information packed into the specification with a whole lot more examples. Additionally, we have thought out some how we can move forward during future changes to the specification!

Why a New Version

I know that previously some people might have noticed that there was a pull request for a version 1.0.2. However after very long talks with others at Mapbox, it was decided that perhaps the best way to proceed would be to bump to a new version.

Our reasoning behind this is that new decoders might depend on the new definitions defined in the changes and therefore older vector tiles might be invalid. However, we limited the scope of the changes such that old decoders should still be able to process version 2.0 tiles.

What changed?

The updated CHANGELOG has a very good overview of all the changes that we have made so far, I highly encourage people familiar with the specification reading this prior to reading the entire new specification.

Before Contributing!

For those interested in reviewing or editing the we have started off by adding a CONTRIBUTING file and it is worth a read (and review)!

Next we have shifted from a major.minor.patch versioning system to a major.minor versioning system! We have added a note in the README about our philosophy.

Resolving Issues

I have walked through all the tickets I would like to see closed prior to releasing of the specification and have created the 2.0 milestone. To prevent detailed conversations about the spec in the pull request please open new issues! I will add them to the milestone.

Decoder/Encoder Developers

I want to help transition all existing encoders and decoders to use the newest version of the specification! If things are not clear or you can't understand what in the new text of the specification PLEASE comment or ask questions!

Thank You

I really want to thank everyone who has helped us develop the specification up to what it is now. To thank everyone individually would be a massive wall of names.

Pings to Notify Everyone

Pardon the long list of names (hopefully I get everyone!)

@springmeyer @yhahn @tmcw @ericfischer @mourner @incanus @artemp @lbud @jfirebaugh @kkaefer @celoyd @morganherlocker @mourner @sgillies @1ec5 @timrobertson100 @pnorman @flemmens @strk @pramsey @hallahan @paulmach @ycabon @zerebubuth @bcamper @jakepruitt @pauldendulk @rmarianski @yohanboniface @talaj @hkrishna @kaidamasaki @joto @danpat @tomhughes @kapouer @StevenLooman @Vross @hjanetzek @cschwarz @ahocevar @ansis @anandthakker @dnomadb @JesseCrocker @psylum @bertt @fitnr @aaronlidman @halset @NielsCharlier
@groldan @ShibaBandit @meetar @jvrousseau @lxbarth @emka @mojodna

Also removed "number of points" -- pairs of ParameterIntegers do not
define points, they define deltas.
This is more clearly mandated by the command grammar for polygon
geometries.
@flippmoke
Copy link
Member Author

Review Deadline

In my initial post, I noticed that I failed to note a planned release date. I know that it is not a large amount of time but I would like all reviews of the pull request to please completed by Nov 27th! This gives us time to correct issues and have a final release by Dec 4th. I understand this is the holiday season and everyone is very busy, but your help with this would be greatly appreciated.

Interested in Helping?

There are somethings that are loose ends that I haven't had time to do and would love community help on:

EDIT People in previous post might not have been notified (as their names are not black?) Adding them here to try to make sure:

@groldan @ShibaBandit @meetar @jvrousseau @lxbarth @emka @mojodna

@flippmoke flippmoke added this to the v2.0 milestone Nov 18, 2015

A layer MUST contain a `version` field with the major version number of the Vector Tile specification to which the layer adheres. For example, a layer adhering to version 2.1 of the specification contains a `version` field with the integer value `2`. The `version` field MUST be the first field within the layer. Decoders SHOULD parse the `version` first to ensure that they are capable of decoding each layer. When a Vector Tile consumer encounters a Vector Tile layer with an unknown version, it MAY make a best-effort attempt to interpret the layer, or it MAY skip the layer. In either case it SHOULD continue to process subsequent layers in the Vector Tile.

A layer MUST contain a `name` field. A Vector Tile MUST NOT contain two or more layers whose `name` values are byte-for-byte identical. Prior to appending a layer to an existing Vector Tile, an encoder MUST check the existing `name` fields in order to prevent duplication.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that a vector tile may have two layers which have the same unicode canonical representation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pnorman I would think it would be possible

@pnorman
Copy link

pnorman commented Nov 19, 2015

for issue linking purposes, this should fix

Some (all?) of those issues have been closed, which seems premature. They're not fixed yet.

@ShibaBandit
Copy link

Good comments from @pnorman.

Is it equivalent to describe the individual polygons as being 'simple'? If so, would that be an easier way of describing it?

@pnorman pnorman mentioned this pull request Nov 19, 2015
- Explained the different commands used in encoding.
- Added the concept of the `cursor` when encoding and decoding a vector tile.
- Explained the coordinate system of a vector tile.
- Made it clear that specification could be used in projections other then Mercator.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then -> than

@flippmoke
Copy link
Member Author

Some (all?) of those issues have been closed, which seems premature. They're not fixed yet.

@pnorman I am attempting to close issues as they are "fixed" so I can track all of the issues remaining

@ShibaBandit
Copy link

In that case @flippmoke , would it make sense reference where you pulled the definition from, then enumerate the properties?

I am also concerned about the 'experimental' raster field. People who implement vector tile parsers or provide 'vector tile support' are going to want to guarantee their program works with vector tile inputs w/ some version subset. They need to be able to know what type of raster data could appear, or what guarantees they can make about their system.

Asking some naive questions but here it goes:
- Since the mapnik-vector-tile format is young, why is keeping backwards compatibility a primary concern?
- Will there be a solid reference implementation before the 2.0 spec is approved?

* Eliminate the pseudo-grammar -- it was imprecise and misleading
  with regards to command repetition vs command counts > 1
* Don't use the term "point" except when speaking about actual POINT
  geometries.
@flippmoke
Copy link
Member Author

@ShibaBandit

I think I would rather have a definition rather then just reference OGC simplicity, which is similar to the rest of the specification.

They need to be able to know what type of raster data could appear, or what guarantees they can make about their system

I have started #50 to hopefully address our reasoning behind this and to have further conversation.

Since the mapnik-vector-tile format is young, why is keeping backwards compatibility a primary concern?

The Mapbox Vector Tile format already has a lot of information stored in it already, so while it is young it is not simple to change a large amount of existing code bases. We do want to eventually consider many more improvements to the standard in future releases, however, we were attempting to limit the number of changes in this version. The primary goal of this version is the clarification and solidification of the standard and clearly defining polygon encoding.

Will there be a solid reference implementation before the 2.0 spec is approved?

https://github.com/mapbox/mapnik-vector-tile will be updated to the spec shortly after it is improved. A few changes are required for it to meet the spec that have not yet been fixed, but it follows most of the specification.

@pnorman
Copy link

pnorman commented Nov 19, 2015

Leaving aside the discussion about the definition of the raster field for now, this isn't within the scope of 2.0, which is clarification of the intent of the intial version of the specification.

@flippmoke
Copy link
Member Author

@pnorman can we continue that discussion in #50

@pnorman
Copy link

pnorman commented Nov 20, 2015

@pnorman can we continue that discussion in #50

I'm drawing a distinction between #50 and if adding a new type of data is within the stated scope of 2.0.


A Vector Tile consists of a set of named layers. A layer contains geometric features and their metadata. The layer format is designed so that the data required for a layer is contiguous in memory, and so that layers can be appended to a Vector Tile without modifying existing data.

A Vector Tile SHOULD contain at least one layer. A layer SHOULD contain at least one feature.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about why we would say that. "A Vector Tile SHOULD contain at least one layer" may be wrong just because some area is covered by no layer, and "A layer SHOULD contain at least one feature" may be wrong when some area is not including any feature. So, as for my understanding, those two sentences should be removed.

@flippmoke
Copy link
Member Author

@yohanboniface In most situations where a layer is empty, you simply do not provide a layer or do not send a tile if there are no layers. The wording here is SHOULD as well rather then MUST, so it is not strictly required but suggested.

@flippmoke
Copy link
Member Author

All issues on the v2.0 milestone are currently closed. If no other issues are raised prior to Friday 4th, I will be merging this branch and calling the new specification official.


## 2.2. Multipurpose Internet Mail Extensions (MIME)

When serving Vector Tiles the MIME type SHOULD be `application/vnd.mapbox-vector-tile`.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're not waiting for registration, we should pull out 2.2

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding this could take a very long time for the registration process to complete? / cc @sgillies Additionally if we needed to change this we could provide an update to the spec and make a 2.1 -- until then I would hope that we can start having people adopt using it.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - I think it's better to have a defined, if unofficial, MIME type than wait for one the registration process to complete. The instruction is "SHOULD", so anyone who feels very strongly about using unapproved MIME types can always ignore it.

@pnorman
Copy link

pnorman commented Dec 3, 2015

@flippmoke
Copy link
Member Author

//bump to @pnorman about last comment

flippmoke added a commit that referenced this pull request Dec 4, 2015
Vector Tile Specification Version 2.0
@flippmoke flippmoke merged commit d6b5d8c into master Dec 4, 2015
@mourner mourner deleted the v2.0-development branch December 9, 2015 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants