protoparse: weird multi-line type references

[jhump]

It turns out that the grammar in protoc parses qualified identifiers in a very different way from the protoparse package. It supports arbitrary whitespace in between name components and the dot separators. Both of the following, for example, are accepted by protoc:

syntax = "proto3";

package foo.bar;

message Baz {
  // spaces between name components and dots
  foo.  bar . Baz child = 1;
}

message Blah {
  // newlines, too!
  foo.
    bar
      . Baz
        child = 1;
}

Neither of these are accepted by the protoparse package. This package only allows whitespaces between name components and dots if it occurs after the name component and before the dot. Files that use whitespace after the dot but before the subsequent name component are rejected with a syntax error like so:

t.proto:6:6: syntax error: unexpected '.'

This requires a change to the grammar so that qualified identifiers are recognized by productions in the grammar, vs. being recognized by the lexer.

cc @pedgeio

[jhump]

#211 attempted to fix this issue. But it only fixed one class of issues that surfaced in some of the Google API protobufs. A proper fix, to be just as flexible as protoc, will require a slightly more involved change.

[bufdev]

We should maybe start mentioning protocolbuffers/protobuf#6188 in these kinds of issues to keep track heh. I'm thinking eventually for that issue that your grammar might be the starting point for https://github.com/protocolbuffers/protobuf-grammar