scripts: twister: Add more Status assignment guards

[LukaszMrugala]

TwisterStatus now should not crash on wrong assignments of type: TwisterStatus.NONEXISTENT_STATUS and
TwisterStatus('NONEXISTENT_STATUS') and instead fail the current test and go on.

Two new errors were created in order to give users more information.

Fixes #80866

Further comparison:

Case 1 - self.instance.status = "PASA"

Previously:

ERROR   - Twister call stack dump:
  <full call stack>
ERROR   - TestInstance assigned status PASA, which could not be cast to a TwisterStatus.
ERROR   - qemu_x86/atom             tests/kernel/mutex/mutex_api/kernel.mutex          ERROR: Incorrect status assignment
ERROR   - see: <ZEPHYR_BASE>/twister-out/qemu_x86_atom/tests/kernel/mutex/mutex_api/kernel.mutex/build.log

Now:
The same as above.

Case 2 - self.instance.status = TwisterStatus.PASA

Previously:

ERROR   - General exception: PASA
ERROR   - Process 2254707 failed, aborting execution

Now:

ERROR   - Attempted access to nonexistent TwisterStatus.PASA. Please verify the status list.
ERROR   - qemu_x86/atom             tests/kernel/mutex/mutex_api/kernel.mutex          ERROR: Incorrect status assignment
ERROR   - see: <ZEPHYR_BASE>/twister-out/qemu_x86_atom/tests/kernel/mutex/mutex_api/kernel.mutex/build.log

Case 3 - self.instance.status = TwisterStatus('pass')

Previously:

ERROR   - General exception: 'pass' is not a valid TwisterStatus
ERROR   - Process 2256571 failed, aborting execution

Now:

ERROR   - Twister call stack dump:
  <full call stack>
ERROR   - Attempted initialisation of status pass, which could not be cast to a TwisterStatus.
ERROR   - qemu_x86/atom             tests/kernel/mutex/mutex_api/kernel.mutex          ERROR: Incorrect status assignment
ERROR   - see: <ZEPHYR_BASE>/twister-out/qemu_x86_atom/tests/kernel/mutex/mutex_api/kernel.mutex/build.log

[golowanow]

Fixes #80866

sorry, but I don't see how this PR fixes #80866 still terminating Twister execution abruptly with a general exception.

What I've got trying this PR:

1

ERROR   - General exception: Attempted access to nonexistent TwisterStatus.PASA. Please verify the status list.
ERROR   - Process 13783 failed, aborting execution                                                             

diff --git a/scripts/pylib/twister/twisterlib/runner.py b/scripts/pylib/twister/twisterlib/runner.py   
index 1c9cc2f4f82..8d44a8a6ec6 100644                                                                  
--- a/scripts/pylib/twister/twisterlib/runner.py                                                       
+++ b/scripts/pylib/twister/twisterlib/runner.py                                                       
@@ -1017,6 +1017,10 @@ class ProjectBuilder(FilterBuilder):                                            
         elif op == "build":                                                                           
             try:                                                                                      
                 logger.debug(f"build test: {self.instance.name}")                                     
+                                                                                                      
+                self.instance.status = TwisterStatus.PASA                                             
+                                                                                                      
                 ret = self.build()                                                                    
                 if not ret:                                                                           
                     self.instance.status = TwisterStatus.ERROR                                       

2

ERROR   - General exception: 'pass'                                       
ERROR   - Process 13947 failed, aborting execution                        

diff --git a/scripts/pylib/twister/twisterlib/runner.py b/scripts/pylib/twister/twisterlib/runner.py 
index 1c9cc2f4f82..0d9faedf130 100644                                                                
--- a/scripts/pylib/twister/twisterlib/runner.py                                                     
+++ b/scripts/pylib/twister/twisterlib/runner.py                                                     
@@ -1017,6 +1017,9 @@ class ProjectBuilder(FilterBuilder):                                           
         elif op == "build":                                                                         
             try:                                                                                    
                 logger.debug(f"build test: {self.instance.name}")                                   
+                                                                                                    
+                self.instance.status = TwisterStatus['pass']                                        
+                                                                                                    
                 ret = self.build()                                                                  
                 if not ret:                                                                         
                     self.instance.status = TwisterStatus.ERROR                                      

3

ERROR   - General exception: Attempted initialisation of status "pass", which could not be cast to a TwisterStatus. 
ERROR   - Process 14106 failed, aborting execution                                                                  

diff --git a/scripts/pylib/twister/twisterlib/runner.py b/scripts/pylib/twister/twisterlib/runner.py  
index 1c9cc2f4f82..b3038706f0a 100644                                                                 
--- a/scripts/pylib/twister/twisterlib/runner.py                                                      
+++ b/scripts/pylib/twister/twisterlib/runner.py                                                      
@@ -1017,6 +1017,9 @@ class ProjectBuilder(FilterBuilder):                                            
         elif op == "build":                                                                          
             try:                                                                                     
                 logger.debug(f"build test: {self.instance.name}")                                    
+                                                                                                     
+                self.instance.status = TwisterStatus('pass')                                         
+                                                                                                     
                 ret = self.build()                                                                   
                 if not ret:                                                                          
                     self.instance.status = TwisterStatus.ERROR                                       

[LukaszMrugala]

I didn't get General Exceptions during my testing, but the specific Errors. I'll need to investigate.

How are you launching the code? Via west?

[LukaszMrugala]

Interesting case.
1 and 2 (your numbering) were caused by the Exceptions being imported differently between statuses.py, which raised them, and runner.py, which caught them.
3 - I've added a catch for that case, too.