Make separate lexer/parser support relative paths

[marcohu]

I'm currently writing an ANTLR rule for Bazel and I'm still running into the issue mentioned in #2065: processing of separated lexer/parser grammars in a subdirectory only works with absolute paths. Because even with -Xexact-output-dir it practically means you are forced to use -package and can't process different packages in a single run. Or you have to embed the package names into the grammars (and I have to analyze the created files to put them into the correct subdirectories). Workable, but not ideal.

Bottom line : I would prefer that the separated lexer/parser grammars use case will be fixed for relative paths.

Looking into the code, the problem seems to be that TokenVocabParser only looks in the root of the output directory, but the .tokens file has been written to a subdirectory. Now I don't know who is at fault here. Should the .tokens file be written to the root directory instead? Or should ANTLR look in the subdirectories for .tokens files as well?

I think the simplest thing to do would be to keep track of the directories that have been created below the output directory and have ANTLR not only check the explicit lib directory, but these implicit lib directories as well. Does this sound like a feasible approach? If so, I will send a PR. Thanks.