e2e tests exhibiting flaky behavior

[LisaLudique]

I saw a test failure for one of the e2e tests on my local fork after merging master/upstream.

TestSnapshotCacheSingleEnvoyAndXdsRelayServer: e2e_test.go:106: 
1132
        Timed out after 1.000s.
1133
        Error: Unexpected non-nil/non-zero extra argument at index 1:
1134
        	<int>: 9
1135
--- FAIL: TestSnapshotCacheSingleEnvoyAndXdsRelayServer (1.23s)
1136
FAIL
1137
FAIL	github.com/envoyproxy/xds-relay/integration	2.266s
1138
FAIL
1139
make: *** [Makefile:25: e2e-tests] Error 1
1140
##[error]Process completed with exit code 2.

The flakiness is likely due to timeouts.

[LisaLudique]

failing test run: https://github.com/LisaLudique/xds-relay/runs/766779309

1122
{"level":"info","ts":1591994119.0154526,"logger":"e2e","msg":"envoy request error","err":"expected envoy response: \"Hi, there!\\n\", got: \"\""}
1123
{"level":"info","ts":1591994119.0161037,"logger":"e2e","msg":"envoy request error","err":"expected envoy response: \"Hi, there!\\n\", got: \"\""}
1124
{"level":"info","ts":1591994119.0163982,"logger":"e2e","msg":"envoy request error","err":"expected envoy response: \"Hi, there!\\n\", got: \"\""}
1125
{"level":"info","ts":1591994119.0164754,"logger":"e2e","msg":"envoy request error","err":"expected envoy response: \"Hi, there!\\n\", got: \"\""}
1126
{"level":"info","ts":1591994119.0167916,"logger":"e2e","msg":"envoy request error","err":"expected envoy response: \"Hi, there!\\n\", got: \"\""}
1127
{"level":"info","ts":1591994119.0168443,"logger":"e2e","msg":"envoy request error","err":"expected envoy response: \"Hi, there!\\n\", got: \"\""}
1128
{"level":"info","ts":1591994119.0170515,"logger":"e2e","msg":"envoy request error","err":"expected envoy response: \"Hi, there!\\n\", got: \"\""}
1129
{"level":"info","ts":1591994119.01722,"logger":"e2e","msg":"envoy request error","err":"expected envoy response: \"Hi, there!\\n\", got: \"\""}
1130
{"level":"info","ts":1591994119.0172505,"logger":"e2e","msg":"asserting envoy listeners configured: ok 0, failed 9"}
1131
    TestSnapshotCacheSingleEnvoyAndXdsRelayServer: e2e_test.go:106: 
1132
        Timed out after 1.000s.
1133
        Error: Unexpected non-nil/non-zero extra argument at index 1:
1134
        	<int>: 9
1135
--- FAIL: TestSnapshotCacheSingleEnvoyAndXdsRelayServer (1.23s)
1136
FAIL
1137
FAIL	github.com/envoyproxy/xds-relay/integration	2.266s
1138
FAIL
1139
make: *** [Makefile:25: e2e-tests] Error 1
1140
##[error]Process completed with exit code 2.```