Improve TypeToken creation validation

[Marcono1234]

Resolves Relates to #1219 (by preventing to capture type variables using TypeToken anonymous subclass)
Related to #1741 (comment).
Includes the changes of #1753, without addressing any of the TODO comments in the description there.

Changes:

• Minor refactoring, removing redundant code and reducing internal method visibility
• Enforce only direct subclasses of TypeToken
  Previously Gson allowed subclasses of subclasses (...) of TypeToken. This could lead to incorrect behavior because the type argument lookup simply retrieved the type argument without resolving it, so the type argument might have been completely unrelated to TypeToken's type variable.
• Prevent TypeToken from capturing type variables (Don't silently ignore missing type information from TypeTokens. #1219)
  As mentioned in that issue, this is pretty error-prone and has lead to multiple quite some confusion already.

No hurry to integrate this, but it might be nice to have these changes eventually. Any feedback is appreciated.

[eamonnmcmanus]

If we were designing the code now, this would be a good check to make. In the deserialization case allowing a type variable allows a silent unchecked conversion. But there's a lot of existing client code which might be working perfectly well with these arguably-illegitimate captured type variables. To get an idea, I ran this change against all of Google's internal tests. There were two test failures due to this new exception. One of them looked like this, in outline:

public abstract class AbstractAction<D extends Dto, T extends Service<D>> implements Action {
  ...
  private final Type resultListType = new TypeToken<List<D>>() {}.getType();
  ...
  protected PlainTextResponse resultToJson(D result) {
    return new PlainTextResponse(GSON.toJson(result, getDtoClass()));
  }

  protected PlainTextResponse resultToJson(List<D> result) {
    return new PlainTextResponse(GSON.toJson(result, resultListType));
  }

  public Class<D> getDtoClass() {
    throw new UnsupportedOperationException("not implemented yet");
  }
}

Then subclasses implement getDtoClass() appropriately.

I think this code would work just as well if the declaration were this:

  private final Type resultListType = new TypeToken<List<? extends Dto>>() {}.getType();

However I also think the code is correct as written. It wouldn't be correct if it were trying to deserialize a List<D>, but here apparently the code only serializes.

So I think this change breaks some correct code, and I don't feel we can justify that.

(The other test failure was deserializing, but it also apparently worked despite being unsound.)

The rest of the PR looks like a good set of improvements, if you want to just remove the verifyNoTypeVariable part.

[Marcono1234]

Even serialization can be broken due to this. For your case let's imagine there is a Dto subclass which has a custom type adapter. Then serializing it with resultToJson(D) works fine, but resultToJson(List<D>) erroneously uses the adapter for Dto instead of D.

The current situation is really unfortunate... I think it would be really useful to have the type variable check. For this PR I can omit it, but I am not sure about completely giving up on that check.

Luckily if the type variable has no bounds, then ObjectTypeAdapter uses the runtime type adapter, so at least in these cases it is probably not causing such big issues (unless a user explicitly wanted to serialize as supertype T instead of using the runtime type, if they have separate adapters).

As side note: I would recommend changing your AbstractAction code to construct the resultListType using Gson's TypeToken.getParameterized(Type, Type...) (if possible), that would fix all issues with it. Sorry if that is a bit presumptuous 😅

[Marcono1234]

@eamonnmcmanus, what do you think?

[eamonnmcmanus]

My assumption is that if Google's code includes two unrelated tests that fail with this check, other people's code might too. I can fix the Google ones but not the other ones. I'm just not feeling that this is a worthwhile check to make now, even though (as I said) I completely agree that it would have made sense if it had always been present.

[Marcono1234]

Should this stricter validation be considered for a future version again? As pointed out above, most likely all cases where a TypeToken captures a type variable are not safe, potentially in very subtle ways.

Just recently two Stack Overflow questions were created where the author stumbled exactly over this issue:

• https://stackoverflow.com/q/74321188
• https://stackoverflow.com/q/74357747

[eamonnmcmanus]

I ran this against all of Google's internal tests and found no problems. Thanks!

[Marcono1234]

Thanks for merging! What do you think about the above #2072 (comment)?