Add checks of output to tests #231
Conversation
test/compare_outputs.py
Outdated
| with open(file1) as f1, open(file2) as f2: | ||
| diff = list(d.compare(f1.readlines(), f2.readlines())) | ||
| agree = True | ||
| for line in diff: |
There was a problem hiding this comment.
I would filter diff already now, since you need it both directly below and in the log (line 77). Something like:
# Only keep lines that start with diff markers and are not comment lines
diff = [
line for line in d.compare(f1.readlines(), f2.readlines())
if line.startswith(('-','+')) and not line[2:].startswith('#')
]
There was a problem hiding this comment.
Good suggestion -- done.
| return agree | ||
|
|
||
|
|
||
| class FITSImage(object): |
There was a problem hiding this comment.
Please add a note that this class was copied (almost) verbatim from the Rapthor repository: rapthor/lib/fitsimage.py. Maybe add a hyperlink to this file on GitLab as well?
|
One more thing. I see that the loglevel is set to |
|
O, and when I try to run the test it fails with the following error: Which is kind of curious, because these files seem to be present. 🤔 |
I already see what's the problem. The missing outputs are not in |
|
That's strange -- the files are in the right place (in |
|
Ah, I found the problem I think: it could write the |
|
I think all comments have been addressed. I also reduced the verbosity of the comparison as requested. |
This PR adds checking of the outputs to the main test script (see RAP-694; it is also relevant to issue #227). The new tests compare the outputs to a reference set (stored in
tests/reference_outputs). Currently, it is checked that:The contents of text files and catalogs are not currently checked, as the details of the fitting (e.g., Gaussian properties, number of Gaussians, their ordering, etc.) may vary due to differences in the libraries, etc. used between the test run and the run used to generate the reference outputs.
Some cleanup was also done, including the removal of old, unused test files.