Merge bitcoin/bitcoin#30872: test: fix exclude parsing for functional…

… runner

72b46f2 test: fix exclude parsing for functional runner (Max Edwards)

Pull request description:

  This restores previous behaviour of being able to exclude a test by name without having to specify .py extension.

  It was noticed in bitcoin/bitcoin#30851 that tests were no longer being excluded.

  PR bitcoin/bitcoin#30244 introduced being able to exclude a specific tests based on args (such as `--exclude "rpc_bind.py --ipv6`) but it made the wrong assumption that test names intended to be excluded would include the .py extension.

  The following bitcoin/bitcoin#30244 (comment) shows that this is not how the `--exclude` flag was used in CI.

  bitcoin/bitcoin#30244 (comment) gave three examples of `--exclude` being used in CI so I compared the number of tests that the runner would run for these three examples in three situations, before #30244 was introduced, in master today and with this PR applied.

  Example:

  `--previous-releases --coverage --extended --exclude feature_dbcrash`

  Test count:
  Before #30244 introduced: 314
  Master: 315
  With this PR: 314

  Example:

  `--exclude feature_init,rpc_bind,feature_bind_extra`

  Test count:
  Before #30244 introduced: 306
  Master 311
  With this PR: 306

  Example:

  `--exclude rpc_bind,feature_bind_extra`

  Before #30244 introduced:  307
  Master 311
  With this PR: 307

  I've also tested that the functionality introduced with #30244 remains and we can still exclude specific tests by argument.

ACKs for top commit:
  maflcko:
    review ACK 72b46f2
  willcl-ark:
    ACK 72b46f2

Tree-SHA512: 37c0e3115f4e3efdf9705f4ff8cd86a5cc906aacc1ab26b0f767f5fb6a953034332b29b0667073f8382a48a2fe9d649b7e60493daf04061260adaa421419d8c8

test/functional/test_runner.py

@@ -522,23 +522,24 @@ def main():
         test_list += BASE_SCRIPTS
 
     # Remove the test cases that the user has explicitly asked to exclude.
+    # The user can specify a test case with or without the .py extension.
     if args.exclude:
         def print_warning_missing_test(test_name):
             print("{}WARNING!{} Test '{}' not found in current test list.".format(BOLD[1], BOLD[0], test_name))
+        def remove_tests(exclude_list):
+            if not exclude_list:
+                print_warning_missing_test(exclude_test)
+            for exclude_item in exclude_list:
+                test_list.remove(exclude_item)
+
         exclude_tests = [test.strip() for test in args.exclude.split(",")]
         for exclude_test in exclude_tests:
-            if exclude_test.endswith('.py'):
-                # Remove <test_name>.py and <test_name>.py --arg from the test list
-                exclude_list = [test for test in test_list if test.split('.py')[0] == exclude_test.split('.py')[0]]
-                if not exclude_list:
-                    print_warning_missing_test(exclude_test)
-                for exclude_item in exclude_list:
-                    test_list.remove(exclude_item)
+            # A space in the name indicates it has arguments such as "wallet_basic.py --descriptors"
+            if ' ' in exclude_test:
+                remove_tests([test for test in test_list if test.replace('.py', '') == exclude_test.replace('.py', '')])
             else:
-                try:
-                    test_list.remove(exclude_test)
-                except ValueError:
-                    print_warning_missing_test(exclude_test)
+                # Exclude all variants of a test
+                remove_tests([test for test in test_list if test.split('.py')[0] == exclude_test.split('.py')[0]])
 
     if args.filter:
         test_list = list(filter(re.compile(args.filter).search, test_list))