proposal: removing filetests for end-users

[thehowl]

Opinionated take stemming from discussion on #891.

I propose to remove filetests as a tool for testing Gno realms in favour of more flexible programmatic approaches to verify Gno programs. I still think their functionality can be kept internally for the gnovm/tests/files directory, as we have an ever-expanding set of tests that verify core functionality of the GnoVM that would otherwise be difficult to test; however, for end-users, I don't see much benefits compared to using "normal" testing features.

I'll go through the directives and explain briefly by what I think they could be replaced:

• // PKGPATH: -- TestSetOrigCaller or TestSetPrevRealm
• // MAXALLOC: -- a potential debug (Add "debug" stdlib #1266) function (so no-op onchain), with signature func WithAllocationLimit(n int64, f func())
• // SEND: -- TestSetOrigSend
• // Output: -- instead of working with using println and similar, which are not available on-chain anyway, this can work by simply changing the test functions to return values instead and verify their output.
• // Error: -- defer func() { err := recover(); /* verify value of err... */ }()
• // Realm: -- none: how we represent realm objects should be internal information which we can change from one release to the next, used for testing realm storage but useless for end users.

So we're removing duplicate ways to do thing, while creating a testing system which does not require learning about a new one when switching from Go to Gno (except for a few functions to manipulate the tests, like the TestSet functions).

Additionally, by making sure that all of the functionality of filetests can be done in normal tests, we remove the necessity of creating one file per test (which can easily pollute a package's directory) and allow for flexibility in testing out a lot of different parameters (ie. having a table-test for checking many different OrigCallers).

[moul]

My opinion:

• I support making this a feature exclusive to maintainers.
• We should consider implementing a similar pattern in Go, such as func ExampleXXX.
• Additionally, I propose supporting txtar files or an equivalent directly from the same folder. We can fully rely on Go in _test.go for only having Go tests.
• Although _filetest should remain accessible, it should be discouraged for users. It still looks like a very good format for gnovm/tests and some debugging patterns.

[thehowl]

From the retreat AtticWG:

• @deelawn is in favour of this
• @harry-hov raises concerns about filetests actually having some good functionality for end-users, citing feat: grc20 dynamic call with registered interface #1275 as an example. I personally think that the given example is still not a good one for this, as it could still be implemented with the normal testing tools.
• manfred wrote his comments above.