Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cFE Application Behavior API functional tests #807

Closed
skliper opened this issue Aug 18, 2020 · 3 comments · Fixed by #1778 or #1808
Closed

Add cFE Application Behavior API functional tests #807

skliper opened this issue Aug 18, 2020 · 3 comments · Fixed by #1778 or #1808
Assignees
Milestone

Comments

@skliper
Copy link
Contributor

skliper commented Aug 18, 2020

Is your feature request related to a problem? Please describe.
Need open source functional tests for certifiability

Describe the solution you'd like
Add cFE Application Behavior API functional tests
– CFE_ES_RunLoop - Check for Exit, Restart, or Reload commands.
– CFE_ES_WaitForStartupSync - Allow an Application to Wait for the "OPERATIONAL" global system state.
– CFE_ES_WaitForSystemState - Allow an Application to Wait for a minimum global system state.
– CFE_ES_IncrementTaskCounter - Increments the execution counter for the calling task.
– CFE_ES_ExitApp - Exit a cFE Application.

Describe alternatives you've considered
N/A

Additional context
N/A

Requester Info
Jacob Hageman - NASA/GSFC

EDIT by @skliper: removed CFE_ES_RegisterApp since it no longer exists.

@skliper skliper added this to the 7.0.0 milestone Aug 18, 2020
@jphickey
Copy link
Contributor

Depending on the testing goals it may require some creativity in the test structure, such as starting a separate/dedicated app to call these APIs - or perhaps even several apps.

The general "cfe_testrunner" app already registers itself using CFE_ES_RegisterApp() and CFE_ES_WaitForStartupSync() which in turn calls CFE_ES_WaitForSystemState() (the former being just a simplified form of the latter) so attempts to call these directly in a later functional test script will result in no effect. Likewise the CFE_ES_ExitApp will actually exit the app, so it isn't something that should be done from the context of the generalized test runner app.

@skliper
Copy link
Contributor Author

skliper commented Aug 4, 2021

Caelum test scenario/pseudo-code brainstorming (future work could enhance testing w a companion app: #1761, but not required this round):

CFE_ES_GetTaskInfo and save the ExecutionCounter
Confirm CFE_ES_RunLoop(&RunStatus) returns true with RunStatus set to CFE_ES_RunStatus_APP_RUN
NOTE - depending on #1762, may add check of RunStatus... for now though the API claims it's just input
CFE_ES_GetTaskInfo and confirm ExecutionCounter increments by 1
assert VOID macro on CFE_ES_IncrementTaskCounter
CFE_ES_GetTaskInfo and confirm ExecutionCounter increments by 1

Technically for CFE_ES_RunLoop should also send the other valid codes (EXIT/ERROR) which right now is OK from the functional test based on implementation but not really documented in the API to not actually do anything except return false... so debatable since we wouldn't want to actually trigger an exit.

Also call CFE_ES_WaitForSystemState for each possible system state, anything less than the expected current should return CFE_SUCCESS, anything greater than current should return CFE_ES_OPERATION_TIMED_OUT

Call CFE_ES_WaitForStartUp (void). Good to check that time elapsed (before/after) is < timeout which would show it didn't time out.

Skip CFE_ES_ExitApp for now from the functional test since that would be better using #1761. Maybe we could add a call to it from sample_app based on some criteria just to show it works? Would be useful to provide an actual use case (if there is one) example from the sample context, since it seems somewhat questionable based on current implementation (the app could just end the main loop).

@jphickey
Copy link
Contributor

jphickey commented Aug 4, 2021

FWIW, the testcase app does call CFE_ES_ExitApp at the end of its run, but it just doesn't get into the log for obvious reasons.

Suggestions for the other APIs seem reasonable enough.

At some point, we probably will have to introduce a second test app that the main test app can start on demand, which would also help for the reload/restart cases, but hopefully that level of detail can be postponed till the next cycle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants