Taskcluster is upset when >1000 files are changed

[Hexcles]

Example: #18856

All jobs are red: https://tools.taskcluster.net/groups/KotEQoQUSxS4anep8szbsw with the same traceback:

+ ./tools/ci/run_tc.py --oom-killer --browser=firefox --browser=chrome --channel=experimental --no-hosts --xvfb wptrunner_infrastructure tools/ci/ci_wptrunner_infrastructure.sh
git show --no-patch --format=%P task_head
git branch base_head b4236a6d2cf913f7ad4ed17b0be71fc6b519fe61
git branch pr_head 5af05f4ea9afa58dae15995adbf10bc47784d17b
git fetch --quiet origin master:master
Job wptrunner_infrastructure
/home/test/web-platform-tests/wpt test-jobs
Traceback (most recent call last):
  File "/home/test/web-platform-tests/wpt", line 5, in <module>
    wpt.main()
  File "/home/test/web-platform-tests/tools/wpt/wpt.py", line 164, in main
    rv = script(*args, **kwargs)
  File "/home/test/web-platform-tests/tools/ci/jobs.py", line 132, in run
    paths = get_paths(**kwargs)
  File "/home/test/web-platf+ ./tools/ci/run_tc.py --oom-killer --browser=firefox --browser=chrome --channel=experimental --no-hosts --xvfb wptrunner_infrastructure tools/ci/ci_wptrunner_infrastructure.sh
git show --no-patch --format=%P task_head
git branch base_head b4236a6d2cf913f7ad4ed17b0be71fc6b519fe61
git branch pr_head 5af05f4ea9afa58dae15995adbf10bc47784d17b
git fetch --quiet origin master:master
Job wptrunner_infrastructure
/home/test/web-platform-tests/wpt test-jobs
Traceback (most recent call last):
  File "/home/test/web-platform-tests/wpt", line 5, in <module>
    wpt.main()
  File "/home/test/web-platform-tests/tools/wpt/wpt.py", line 164, in main
    rv = script(*args, **kwargs)
  File "/home/test/web-platform-tests/tools/ci/jobs.py", line 132, in run
    paths = get_paths(**kwargs)
  File "/home/test/web-platform-tests/tools/ci/jobs.py", line 86, in get_paths
    changed, _ = files_changed(revish)
  File "/home/test/web-platform-tests/tools/wpt/testfiles.py", line 212, in files_changed
    include_new=include_new)
  File "/home/test/web-platform-tests/tools/wpt/testfiles.py", line 155, in repo_files_changed
    assert not files_list[-1]
AssertionError
Traceback (most recent call last):
  File "./tools/ci/run_tc.py", line 290, in <module>
    main()
  File "./tools/ci/run_tc.py", line 274, in main
    if fn():
  File "./tools/ci/run_tc.py", line 271, in <lambda>
    (lambda:include_job(job), "CI required jobs includes '%s'" % job)]
  File "./tools/ci/run_tc.py", line 201, in include_job
    "test-jobs"], return_stdout=True)
  File "./tools/ci/run_tc.py", line 63, in run
    return f(cmd, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 223, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['/home/test/web-platform-tests/wpt', 'test-jobs']' returned non-zero exit status 1
[taskcluster 2019-09-04 23:02:54.976Z] === Task Finished ===
[taskcluster 2019-09-04 23:02:55.072Z] Unsuccessful task run with exit code: 1 completed in 75.691 secondsorm-tests/tools/ci/jobs.py", line 86, in get_paths
    changed, _ = files_changed(revish)
  File "/home/test/web-platform-tests/tools/wpt/testfiles.py", line 212, in files_changed
    include_new=include_new)
  File "/home/test/web-platform-tests/tools/wpt/testfiles.py", line 155, in repo_files_changed
    assert not files_list[-1]
AssertionError
Traceback (most recent call last):
  File "./tools/ci/run_tc.py", line 290, in <module>
    main()
  File "./tools/ci/run_tc.py", line 274, in main
    if fn():
  File "./tools/ci/run_tc.py", line 271, in <lambda>
    (lambda:include_job(job), "CI required jobs includes '%s'" % job)]
  File "./tools/ci/run_tc.py", line 201, in include_job
    "test-jobs"], return_stdout=True)
  File "./tools/ci/run_tc.py", line 63, in run
    return f(cmd, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 223, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['/home/test/web-platform-tests/wpt', 'test-jobs']' returned non-zero exit status 1
[taskcluster 2019-09-04 23:02:54.976Z] === Task Finished ===
[taskcluster 2019-09-04 23:02:55.072Z] Unsuccessful task run with exit code: 1 completed in 75.691 seconds

Presumably the list is too long?

[Hexcles]

cc @jgraham any guesses?

[foolip]

This is similar to #18608 but the failure mode isn't the same.

@Hexcles is this affecting many runs? Seems like the prio of the two issues should be the same, so one should change :)

[jgraham]

https://searchfox.org/mozilla-central/source/testing/web-platform/tests/tools/wpt/testfiles.py#155 looks like the failure, which means that the output doesn't have the trailing null we expect. So it could be that we're not reading the complete output in that case for some reason, or there's some other output that's confusing matters (e.g. something written to stderr).

[Hexcles]

(e.g. something written to stderr).

That's a useful insight! git is indeed printing warnings about diff being too big that makes rename detection unreliable. Let me try discarding stderr.

[Hexcles]

Closing as duplicate of #18608