improve cucumber tags and speed

Add flaky and broken tags
tari-project · Oct 11, 2021 · 30e9dee · 30e9dee
1 parent 43b2033
commit 30e9dee
Show file tree

Hide file tree

Showing 22 changed files with 1,583 additions and 1,545 deletions.
diff --git a/integration_tests/features/BaseNodeAutoUpdate.feature b/integration_tests/features/BaseNodeAutoUpdate.feature
@@ -9,5 +9,4 @@ Feature: AutoUpdate
     @broken
     Scenario: Auto update ignores update with invalid signature on base node
         Given I have a node NODE_A with auto update configured with a bad signature
-        And I wait 10 seconds
         Then NODE_A does not have a new software update
diff --git a/integration_tests/features/BaseNodeConnectivity.feature b/integration_tests/features/BaseNodeConnectivity.feature
@@ -1,12 +1,14 @@
 @base-node-connectivity
 Feature: Base Node Connectivity
 
+    @critical @base-node
     Scenario: Basic connectivity between 2 nodes
         Given I have a seed node SEED_A
         And I have a base node NODE_A connected to all seed nodes
         When I wait for NODE_A to connect to SEED_A
         Then SEED_A is connected to NODE_A
 
+    @critical @base-node @wallet
     Scenario: Basic connectivity between nodes and wallet
         Given I have a seed node SEED_A
         And I have wallet WALLET_A connected to all seed nodes

diff --git a/integration_tests/features/BlockExplorerGRPC.feature b/integration_tests/features/BlockExplorerGRPC.feature
@@ -1,7 +1,6 @@
 @block-explorer
 Feature: Block Explorer GRPC
 
-  @critical
   Scenario: As a user I want to get the network difficulties
     Given I have a seed node NODE
     And I have wallet WALLET connected to all seed nodes

diff --git a/integration_tests/features/Mempool.feature b/integration_tests/features/Mempool.feature
@@ -1,7 +1,7 @@
 @mempool
 Feature: Mempool
 
-  @flaky
+  @critical @base-node @flaky
   Scenario: Transactions are propagated through a network
     #
     # The randomness of the TX1 propagation can result in this test not passing.
@@ -18,7 +18,7 @@ Feature: Mempool
     Then TX1 is in the MEMPOOL of all nodes, where 1% can fail
 
 
-  @flaky
+  @critical @base-node @flaky
   Scenario: Transactions are synced
     Given I have 2 seed nodes
     And I have a base node SENDER connected to all seed nodes
@@ -35,7 +35,7 @@ Feature: Mempool
     Then SENDER has TX1 in MINED state
     Then TX1 is in the MINED of all nodes
 
-  @flaky @broken
+  @base-node @flaky @broken
   Scenario: Clear out mempool
     Given I have 1 seed nodes
     And I have a base node SENDER connected to all seed nodes
@@ -62,7 +62,7 @@ Feature: Mempool
     Then SENDER has TX3 in MINED state
 
 
-  @critical @long-running
+  @critical @base-node
   Scenario: Double spend
     Given I have 1 seed nodes
     And I have a base node SENDER connected to all seed nodes
@@ -75,72 +75,53 @@ Feature: Mempool
     Then SENDER has TX1 in MEMPOOL state
     Then SENDER has TX2 in MEMPOOL state
     When I mine 1 blocks on SENDER
-    Then SENDER has TX1 in NOT_STORED state
+    # a transaction that was removed from the pool will be reported as unknown as long as it is stored in the reorg pool for 5 mins
+    Then SENDER has TX1 in UNKNOWN state
     Then SENDER has TX2 in MINED state
+
+ @long-running @base-node
+  Scenario: Double spend eventually ends up as not stored
+    Given I have 1 seed nodes
+    And I have a base node SENDER connected to all seed nodes
+    When I mine a block on SENDER with coinbase CB1
+    When I mine 4 blocks on SENDER
+    When I create a custom fee transaction TX1 spending CB1 to UTX1 with fee 80
+    When I create a custom fee transaction TX2 spending CB1 to UTX2 with fee 100
+    When I submit transaction TX1 to SENDER
+    When I submit transaction TX2 to SENDER
+    Then SENDER has TX1 in MEMPOOL state
+    Then SENDER has TX2 in MEMPOOL state
     When I mine 1 blocks on SENDER
+    # a transaction that was removed from the pool will be reported as unknown as long as it is stored in the reorg pool for 5 mins
     Then SENDER has TX1 in NOT_STORED state
     Then SENDER has TX2 in MINED state
 
-  @critical
+  @critical @base-node
   Scenario: Mempool clearing out invalid transactions after a reorg
-    #
-    # Chain 1:
-    #   Collects 7 coinbases into one wallet, send 7 transactions
-    #   Stronger chain
-    #
     Given I do not expect all automated transactions to succeed
     Given I have a seed node SEED_A
-    And I have a base node NODE_A1 connected to seed SEED_A
-    And I have wallet WALLET_A1 connected to seed node SEED_A
-    And I have wallet WALLET_A2 connected to seed node SEED_A
-    And I have mining node MINER_A1 connected to base node SEED_A and wallet WALLET_A1
-    When I wait 5 seconds
-    When mining node MINER_A1 mines 7 blocks with min difficulty 200 and max difficulty 100000
-    Then node SEED_A is at height 7
-    Then node NODE_A1 is at height 7
-    When I mine 3 blocks on SEED_A
-    Then wallet WALLET_A1 detects at least 7 coinbase transactions as Mined_Confirmed
-    Then node SEED_A is at height 10
-    Then node NODE_A1 is at height 10
-    And I multi-send 7 transactions of 1000000 uT from wallet WALLET_A1 to wallet WALLET_A2 at fee 100
-    Then wallet WALLET_A1 detects all transactions are at least Broadcast
-    When I wait 1 seconds
-    #
-    # Chain 2:
-    #   Collects 7 coinbases into one wallet, send 7 transactions
-    #   Weaker chain
-    #
-    And I have a seed node SEED_B
-    And I have a base node NODE_B1 connected to seed SEED_B
-    And I have wallet WALLET_B1 connected to seed node SEED_B
-    And I have wallet WALLET_B2 connected to seed node SEED_B
-    And I have mining node MINER_B1 connected to base node SEED_B and wallet WALLET_B1
-    When I wait 5 seconds
-    When mining node MINER_B1 mines 7 blocks with min difficulty 1 and max difficulty 100
-    Then node SEED_B is at height 7
-    Then node NODE_B1 is at height 7
-    When I mine 5 blocks on SEED_B
-    Then wallet WALLET_B1 detects at least 7 coinbase transactions as Mined_Confirmed
-    Then node SEED_B is at height 12
-    Then node NODE_B1 is at height 12
-    And I multi-send 7 transactions of 1000000 uT from wallet WALLET_B1 to wallet WALLET_B2 at fee 100
-    Then wallet WALLET_B1 detects all transactions are at least Broadcast
-    When I wait 1 seconds
-    #
-    # Connect Chain 1 and 2 in stages
-    #    New node connects to weaker chain, receives all broadcast (not mined) transactions into mempool
-    #    New node connects to stronger chain, then reorgs its complete chain
-    #    New node mines blocks; no invalid inputs from the weaker chain should be used in the block template
-    #
-    And I have a base node NODE_C connected to seed SEED_B
-    Then node NODE_C is at height 12
-    # Wait for the reorg to filter through
-    And I connect node SEED_A to node NODE_C and wait 30 seconds
-    Then all nodes are at height 10
-    When I mine 6 blocks on NODE_C
-    Then all nodes are at height 16
+    And I have a base node NODE_A connected to seed SEED_A
+    When I mine a block on NODE_A with coinbase CB_A
+    When I mine 3 blocks on NODE_A
+    Given I have a seed node SEED_B
+    And I have a base node NODE_B connected to seed SEED_B
+    When I mine a block on NODE_B with coinbase CB_B
+    When I mine 10 blocks on NODE_B
+    When I create a custom fee transaction TXA spending CB_A to UTX1 with fee 80
+    When I create a custom fee transaction TXB spending CB_B to UTX1 with fee 80
+    When I submit transaction TXA to NODE_A
+    When I submit transaction TXB to NODE_B
+    Then NODE_A has TXA in MEMPOOL state
+    Then NODE_B has TXB in MEMPOOL state
+    When I mine 1 blocks on NODE_A
+    When I mine 1 blocks on NODE_B
+    And I connect node NODE_A to node NODE_B and wait 1 seconds
+    Then all nodes are at height 12
+    Then NODE_A has TXA in NOT_STORED state
+    Then NODE_A has TXB in MINED state
+
 
-  @critical
+  @critical @base-node
   Scenario: Zero-conf transactions
     Given I have 1 seed nodes
     And I have a base node SENDER connected to all seed nodes
@@ -173,7 +154,7 @@ Feature: Mempool
     Then SENDER has TX12 in MINED state
     Then SENDER has TX13 in MINED state
 
-  @critical
+  @base-node
   Scenario: Mempool unconfirmed transactions
     Given I have 1 seed nodes
     And I have a base node BN1 connected to all seed nodes
@@ -191,7 +172,7 @@ Feature: Mempool
     When I submit transaction TX5 to BN1
     Then I wait until base node BN1 has 5 unconfirmed transactions in its mempool
 
-  @critical
+  @base-node
   Scenario: Mempool unconfirmed transaction to mined transaction
     Given I have 1 seed nodes
     And I have a base node BN1 connected to all seed nodes

diff --git a/integration_tests/features/MergeMining.feature b/integration_tests/features/MergeMining.feature
@@ -1,7 +1,7 @@
 @merge-mining
   Feature: Merge Mining
 
-    @critical
+    @base-node
     Scenario: Merge Mining Functionality Test Without Submitting To Origin
       Given I have a seed node NODE
       And I have wallet WALLET connected to all seed nodes
@@ -13,7 +13,7 @@
       When I submit a block through proxy PROXY
       Then Proxy response block submission is valid without submitting to origin
 
-    @critical
+    @base-node
     Scenario: Merge Mining Functionality Test With Submitting To Origin
       Given I have a seed node NODE
       And I have wallet WALLET connected to all seed nodes
@@ -29,7 +29,7 @@
       When I ask for a block header by hash using last block header from proxy PROXY
       Then Proxy response for block header by hash is valid
 
-    @critical
+    @critical @base-node
     Scenario: Simple Merge Mining
       Given I have a seed node NODE
       And I have wallet WALLET connected to all seed nodes

diff --git a/integration_tests/features/Propagation.feature b/integration_tests/features/Propagation.feature
@@ -7,22 +7,20 @@ Feature: Block Propagation
     And I have a SHA3 miner MINER connected to all seed nodes
     And mining node MINER mines <NumBlocks> blocks
     Then all nodes are at height <NumBlocks>
-    @critical
-    Examples:
-      | NumSeeds | NumNonSeeds | NumBlocks |
-      | 1        | 1           | 5         |
 
+    @base-node
     Examples:
       | NumSeeds | NumNonSeeds | NumBlocks |
+      | 1        | 1           | 5         |
       | 1        | 10          | 5         |
       | 4        | 10          | 5         |
 
-    @long-running
+    @base-node @long-running
     Examples:
       | NumSeeds | NumNonSeeds | NumBlocks |
       | 8        | 40          | 10        |
 
-  @critical
+  @critical @base-node
   Scenario: Simple propagation
     Given I have 2 seed nodes
     And I have 4 base nodes connected to all seed nodes
@@ -31,6 +29,7 @@ Feature: Block Propagation
     Then node MINER is at height 5
     Then all nodes are at height 5
 
+  @critical @base-node
   Scenario: Duplicate block is rejected
     Given I have 1 seed nodes
     And I have a base node MINER connected to all seed nodes
@@ -45,6 +44,7 @@ Feature: Block Propagation
     When I mine 1 blocks on MINER
     Then all nodes are at height 2
 
+  @base-node
   Scenario: Submit orphan
     Given I have 1 seed nodes
     And I have a base node MINER connected to all seed nodes
@@ -58,41 +58,41 @@ Feature: Block Propagation
     Then I receive an error containing 'Orphan block'
     And all nodes are at height 1
 
-  @non-sync-propagation @long-running
+  @base-node @non-sync-propagation
   Scenario: Nodes should never switch to block sync but stay synced via propagation
     Given I have 1 seed nodes
     Given I have a SHA3 miner MINER connected to all seed nodes
     And I have a lagging delayed node LAG1 connected to node MINER with blocks_behind_before_considered_lagging 10000
     Given I have a lagging delayed node LAG2 connected to node MINER with blocks_behind_before_considered_lagging 10000
     # Wait for node to so start and get into listening mode
-    When I wait 100 seconds
+    Then node LAG1 has reached initial sync
+    Then node LAG2 has reached initial sync
     When mining node MINER mines 5 blocks
     Then all nodes are at height 5
     Given mining node MINER mines 15 blocks
     Then all nodes are at height 20
 
-  @long-running
+
+  @base-node
   Scenario: Node should lag for while before syncing
     Given I have 1 seed nodes
     Given I have a SHA3 miner MINER connected to all seed nodes
     And I have a lagging delayed node LAG1 connected to node MINER with blocks_behind_before_considered_lagging 6
     Given mining node MINER mines 1 blocks
-    When I wait 100 seconds
+    Then all nodes are at height 1
     When I stop node LAG1
-    When I wait 10 seconds
     And mining node MINER mines 5 blocks
-    When I wait 100 seconds
+    Then node MINER is at height 6
     When I start base node LAG1
     # Wait for node to so start and get into listening mode
-    When I wait 100 seconds
-    Then node MINER is at height 6
+    Then node LAG1 has reached initial sync
     #node was shutdown, so it never received the propagation messages
     Then node LAG1 is at height 1
     Given mining node MINER mines 1 blocks
     Then node MINER is at height 7
-    When I wait 20 seconds
     Then all nodes are at height 7
 
+  @base-node @critical
   Scenario: Pruned node should prune outputs
     Given I have 1 seed nodes
     And I have a base node SENDER connected to all seed nodes