Some file tests that should have failed always pass in Github repo.

[piux2]

PROBLEM:

Though some file tests should have failed, they passed when we merged the pull request in the master branch in Github.

CAUSE:

There is a syncWanted flag in tests/file.go. The default is true. The actual result is compared with the wanted results in the OUTPUT and REALM comments sections of xyz-file-test.gno. If the comparison does not match, the actual result is written back to xyz-file-test.gno. It overwrites the wanted results in the comments.

This is A FEATURE that we want when we run a local test. By running "git status," we can see all the discrepancies and changed files at once without being interrupted by each failure testing case.

However, when github action flow triggers "make test", the tests pass because there is no error or panic.

Failure Testing Cases:

After I turned off the syncWanted flag and ran "make test", the following tests failed for different reasons.

examples/gno.land/r/demo/releases-example/
release0_filetest.gno
release1_filetest.gno

This one is tricky. The actual is the same as and wanted results in the test file. However, in the run time, when go.parser parses the wanted comment, it recognizes up to two lines of // style comments. parser.ParseComments() already returns two newline max. Since prinln(renderedContent) returns three new lines, it always fails. Therefore, we can not use println in the file test if the rendered content already has two new lines.

tests/files2/
std7.gno
zrealm_exmaple.gno
zrealm_test0.gno

All these tests failed because we restructured the example naming space, and the package path changed. As a result, the package address and realm object hashes changed. However, the package address in std7.gno and realm object hashes in zrealm_exmaple.gno, zrealm_test0.gno still have the value before the changes.

FIX:

I will make separate 3 PR to fix it for easy review and verification.
No. 3 needs to be merged after 1 and 2. Otherwise, the test will always fail and panic when we merge the codes in the main branch.

1. Fix examples/gno.land/r/demo/releases-example/release0_filetest.gno, release1_filetest.gno
2. Fix tests/files2/std7.gno, zrealm_exmaple.gno, zrealm_test0.gno
3. Set syncWanted to false and Add Sync flag to enable syncWanted in gnodev/test.go for local tests

[moul]

@piux2 can we close this?

Else, can you update the current status and next steps, please?

[moul]

Is this issue still a thing? if yes, can you update us, please?

[thehowl]

Closing as the referenced PRs have been merged, and by simple empirical experience I know the CI complains very loudly when a test is failing.

If you feel that this has not yet been fully addressed, of course, feel free to re-open.