tap2junit fails on test input when tests time out

[sam-github]

2:03:21 mkdir out/junit || true
12:03:21 tap2junit -i test.tap -o out/junit/test.xml
12:03:21 tap2junit -i cctest.tap -o out/junit/cctest.xml || true
12:03:21 rm -Rf out/Release/obj.target
12:03:21 [aix61-ppc64] $ /bin/sh -xe /tmp/jenkins5548162882024815593.sh
12:03:22 + mkdir out/junit
12:03:22 mkdir: 0653-358 Cannot create out/junit.
12:03:22 out/junit: Do not specify an existing file.
12:03:22 + true
12:03:22 + tap2junit -i test.tap -o out/junit/test.xml
12:03:27 Traceback (most recent call last):
12:03:27   File "/usr/bin/tap2junit", line 11, in <module>
12:03:27     sys.exit(main())
12:03:27   File "/opt/freeware/lib/python2.7/site-packages/tap2junit/__main__.py", line 44, in main
12:03:27     result = parse(os.path.splitext(args.input.name)[0], data)
12:03:27   File "/opt/freeware/lib/python2.7/site-packages/tap2junit/__main__.py", line 15, in parse
12:03:27     t = TestCase(test.description, None, test.yaml['duration_ms'])
12:03:27 TypeError: 'NoneType' object has no attribute '__getitem__'

• https://ci.nodejs.org/job/node-test-commit-aix/nodes=aix61-ppc64/23968/console

I sshed in and reproduced it on AIX, then copied the file to my laptop, ubuntu, and reproduced here. tap file is https://gist.github.com/c15ba6bd9ff7b529a7aa04fa176f068a

core/build (centos7-python-3.7 $% u=) % scp  test-osuosl-aix61-ppc64_be-1:/home/iojs/issue.tap .                                                                             
issue.tap                                           100%  185KB   1.3MB/s   00:00
core/build (centos7-python-3.7 $% u=) % pip install tap2junit
Collecting tap2junit
  Downloading https://files.pythonhosted.org/packages/af/22/f8436b6d750114dd5047970ad878259edeafa89fac72b8dced476965c6d6/tap2junit-0.1.4.tar.gz                              
Collecting junit-xml (from tap2junit)
  Downloading https://files.pythonhosted.org/packages/a6/2a/f8d5aab80bb31fcc789d0f2b34b49f08bd6121cd8798d2e37f416df2b9f8/junit-xml-1.8.tar.gz                                
Collecting yamlish (from tap2junit)
  Downloading https://files.pythonhosted.org/packages/29/fd/d607e225d757aa2950abff9ba19145be3c72b74266fc48f3266ec7ba3087/yamlish-0.18.1-py2-none-any.whl                     
Requirement already satisfied: six in /home/sam/.local/lib/python2.7/site-packages (from junit-xml->tap2junit) (1.12.0)                                                      
Requirement already satisfied: PyYAML>=3.09 in /usr/lib/python2.7/dist-packages (from yamlish->tap2junit) (3.13)                                                             
Building wheels for collected packages: tap2junit, junit-xml
  Running setup.py bdist_wheel for tap2junit ... done
  Stored in directory: /home/sam/.cache/pip/wheels/12/a9/54/a5f8ed9d0adfc510303204543f743bac3fc3a1b236d4545b10                                                               
  Running setup.py bdist_wheel for junit-xml ... done
  Stored in directory: /home/sam/.cache/pip/wheels/c6/c5/65/5b166afb6eac87dc300368ec4d92f39502dd41cdc73056d49e                                                               
Successfully built tap2junit junit-xml
Installing collected packages: junit-xml, yamlish, tap2junit
  The script tap2junit is installed in '/home/sam/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.                                                          
Successfully installed junit-xml-1.8 tap2junit-0.1.4 yamlish-0.18.1
core/build (centos7-python-3.7 $% u=) % /home/sam/.local/bin/tap2junit -i issue.tap -o test.xml                                                                              
Traceback (most recent call last):
  File "/home/sam/.local/bin/tap2junit", line 10, in <module>
    sys.exit(main())
  File "/home/sam/.local/lib/python2.7/site-packages/tap2junit/__main__.py", line 44, in main                                                                                
    result = parse(os.path.splitext(args.input.name)[0], data)
  File "/home/sam/.local/lib/python2.7/site-packages/tap2junit/__main__.py", line 15, in parse                                                                               
    t = TestCase(test.description, None, test.yaml['duration_ms'])
TypeError: 'NoneType' object has no attribute '__getitem__'

[sam-github]

Did a pip install tap.py and it processed the tap file OK, so I don't think its malformed. Also couldn't see anything about file the format visually, it looks pretty boring.

[sam-github]

Not a one-off, also happened on https://ci.nodejs.org/job/node-test-commit-aix/nodes=aix61-ppc64/23969/console

[sam-github]

What's happening is that if the tap file starts with a plan, like 1..2701, but then only has 2410 tests in it, tap2junit tries to append tests for each missing one, but there is a type in the comment field, and the tests lack the yaml property, so the main blows up.

Doesn't answer "why is the tap file sometimes truncated", does anybody have an idea about that?

[sam-github]

I think that the job is being killed, see [test-ci] Killed, possibly by jenkins because of a timeout. Does anyone have any ideas of what might be killing the test runner before it is done?

Possibly this is another symptom of #1820, not sure.

12:00:26 ok 2410 addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max
12:00:26   ---
12:00:26   duration_ms: 113.396
12:00:26   ...
12:03:21 gmake: *** Waiting for unfinished jobs....
12:03:21 gmake[1]: *** Waiting for unfinished jobs....
12:03:21 gmake[1]: *** [test-ci] Killed
12:03:21 Build step 'Execute shell' marked build as failure

• https://ci.nodejs.org/job/node-test-commit-aix/nodes=aix61-ppc64/23968/console

[addaleax]

Does anyone have any ideas of what might be killing the test runner before it is done?

Since it’s failing at the stringbytes-external-exceed-max tests, maybe it’s the operating system’s OOM killer? At least for Linux that would seem plausible, I obviously don’t know how that works on AIX…

[sam-github]

• 23975: 3

18:54:05 ok 2410 addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max
18:54:05   ---
18:54:05   duration_ms: 2.951
18:54:05   ...
18:56:35 gmake: *** Waiting for unfinished jobs....
18:56:35 gmake[1]: *** Waiting for unfinished jobs....
18:56:36 Build step 'Execute shell' marked build as failure
18:56:41 Performing Post build task...
18:56:41 Match found for : : True
18:56:41 Logical operation result is TRUE
18:56:41 Running script  : #/bin/bash -x +e
18:56:41 

• https://ci.nodejs.org/job/node-test-commit-aix/23975/nodes=aix61-ppc64/console
• 23971: 1 same as above, last test was also stringbytes-external-exceed-max

So, this issue looks highly correlated with that specific test. I'll look into this.

[sam-github]

However, I'm also seeing it on other tests, so leaving this note here, too:

• 23969: 3

13:00:06 ok 2581 sequential/test-async-wrap-getasyncid
13:00:06   ---
13:00:06   duration_ms: 1.946
13:00:06   ...
13:02:10 gmake: *** [run-ci] Killed
13:02:10 gmake: *** wait: There are no child processes..  Stop.
13:02:11 Build step 'Execute shell' marked build as failure

• 23968: 1 ditto

[sam-github]

Stress test for addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max: https://ci.nodejs.org/job/node-stress-single-test/2235/

[sam-github]

tap2junit just needs reworking. Its crud python code, but the problem is easy to reproduce if anyone wants to take a shot at, and doesn't require access to ci, or even to node!

Its not a priority though, because it is only a follow-on problem, caused by a previous failure.

[sam-github]

It also may need updating for python3, see #1830 (comment)

[Trott]

It also may need updating for python3, see #1830 (comment)

tap2junit 0.1.5 is compatible with Python 3 and would seem to fix other bugs. Refs: nodejs/node#30955 (comment)

Can we update to tap2junit 0.1.5?

[davidstanke]

Hey folks, I'm starting to investigate this... currently setting up a local environment to reproduce. I'll report back as I learn things.

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.