Errata WIP for remaining UI Failures #15085

damoore044 · 2024-05-16T16:41:43Z

Problem Statement

Fix remaining UI errata failures (8-10 in recent CI 6.15.0/stream)

Fixture improvements
Remove some remaining outdated setup/ unusable fixtures
Fix intermittent failures of test_end_to_end and _pagination
Makes use of api_factory method proposed (closed)
API-factory: Method to Associate needed entities, then Register the Host #14974

Ongoing Failures

test_positive_content_host_previous_env: Skipped for now, until BZ #2280882 is resolved.
test_positive_apply_for_all_hosts: ~~Testing locally~~ Fixed- Airgun PR #1387. Added End2End flag for this case, as it is now very thorough and makes use of multiple install methods.

PRT Case

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py
airgun: 1387

damoore044 · 2024-05-16T16:41:49Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py

damoore044 · 2024-05-16T17:57:28Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py

Satellite-QE · 2024-05-16T19:48:03Z

PRT Result

Build Number: 6968
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/ui/test_errata.py --external-logging
Test Result : ======= 9 failed, 9 passed, 1842 warnings, 1 error in 6026.68s (1:40:26) =======

damoore044 · 2024-05-16T20:04:38Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py

Satellite-QE · 2024-05-16T21:52:43Z

PRT Result

Build Number: 6975
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/ui/test_errata.py --external-logging
Test Result : ====== 4 failed, 5 passed, 1764 warnings, 10 errors in 5999.92s (1:39:59) ======

damoore044 · 2024-05-17T13:56:55Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py

Satellite-QE · 2024-05-17T15:49:29Z

PRT Result

Build Number: 6992
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/ui/test_errata.py --external-logging
Test Result : ====== 5 failed, 13 passed, 1861 warnings, 1 error in 6449.62s (1:47:29) =======

damoore044 · 2024-05-17T17:22:17Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py

Satellite-QE · 2024-05-17T18:57:28Z

PRT Result

Build Number: 6994
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/ui/test_errata.py --external-logging
Test Result : ====== 8 failed, 8 passed, 1853 warnings, 2 errors in 5132.86s (1:25:32) =======

damoore044 · 2024-05-17T19:11:15Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py

damoore044 · 2024-05-17T19:46:04Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py

Satellite-QE · 2024-05-17T21:31:59Z

PRT Result

Build Number: 6997
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/ui/test_errata.py --external-logging
Test Result : ====== 8 failed, 10 passed, 2099 warnings, 2 errors in 5952.10s (1:39:12) ======

damoore044 · 2024-05-18T13:54:49Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py
airgun: 1387

damoore044 · 2024-06-17T18:09:00Z

Pulled in changes and suggestions from #15125, thanks @ColeHiggins2 !

Satellite-QE · 2024-06-17T19:25:45Z

PRT Result

Build Number: 7413
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/ui/test_errata.py --external-logging
Test Result : ========== 17 passed, 1 skipped, 1960 warnings in 5935.16s (1:38:55) ===========

vsedmik

Adding a few comments for now, I will probably need another iteration to complete the review.

vsedmik · 2024-06-18T09:59:48Z

robottelo/host_helpers/api_factory.py

+                        'value': int(True),
+                    }
+                )
+                repo.read().sync()


Nothing against syncing it here, but if we don't set enable_repos param, we need to sync it elsewhere. To me it feels a bit unrelated to enabling the repo at AK. Like two different things.
If you decide to keep it maybe just update the docstring that the repo gets synced here too.

Agreed, removed it from here, repos can be synced before or after registration

tests/foreman/ui/test_errata.py

vsedmik · 2024-06-18T14:10:31Z

tests/foreman/ui/test_errata.py

+    # no repos given/present, subscription-manager should report error
+    if len(repos) == 0:
+        assert client.execute(r'subscription-manager repos --enable \*').status == 1


Not sure if we need to test this, it's rather sub-man functionality.

hmm but if sub-man fails to find repos to enable, then that will probably mean errata applicability won't work either, correct?

Yes, but here we just assert that it really fails when no repos provided.
If len(repos) == 0 means that we have no repos where we would install or apply errata from anyway. So is it allowed variant?

gotcha yeah I see, code is a little different but my thought process was this;
if we want to register host with no repos at time of reg
(allowed & default variant of the chost fixture)...
I wanted to ensure sub-manager also found no repos to enable.
Seems unnecessary though?

Because after registration, we can always add repos to cv and publish/promote, and use subman as normal after. So by default (no repos) I also wanted to check subman found nothing

Ok, gotcha. It just appeared odd to me that we check sub-man functionality here, not Sat (like do we want to fail Satellite test when sub-man is broken actually). But we rely on sub-man repos and others anyway, so I don't mind to keep it.

@vsedmik Should we also check some satellite functionality too? Such as making sure no repos enabled in AK, when none are specified?

That would rather be a negative test case for the Activation-key component. I wouldn't put it in Errata testing.

tests/foreman/ui/test_errata.py

vsedmik · 2024-06-18T16:55:26Z

tests/foreman/ui/test_errata.py

+        for key in actual_values:
+            assert expected_values[key] in actual_values[key], 'Expected text not found'


Since expected_values and actual_values are same length (same keys), I guess this should work

Suggested change

for key in actual_values:

assert expected_values[key] in actual_values[key], 'Expected text not found'

assert expected_values == actual_values, 'Expected text not found'

So this assertion fails due to one differing value

E Differing items: E {'Errata': 'Security errata installable'} != {'Errata': 'Security errata installable clear'}

The existing test already had this key-val looping assertion, so should we keep the same?

Or we could revise our expectation if we get something unexpected but it seems to be correct anyway.

expected_values = { 'Status': 'Error', 'Errata': 'Security errata installable clear', 'Subscription': 'Fully entitled', }

Btw, why 'clear'??

I am not sure about the clear, first time I am seeing it,
will dig in to see why it's relevant

damoore044 · 2024-06-20T19:16:59Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py

damoore044 · 2024-06-20T19:34:53Z

trigger: test-robottelo
pytest: tests/foreman/ui/test_errata.py

Satellite-QE · 2024-06-20T21:38:20Z

PRT Result

Build Number: 7462
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/ui/test_errata.py --external-logging
Test Result : ========== 17 passed, 1 skipped, 2166 warnings in 6961.59s (1:56:01) ===========

damoore044 · 2024-06-21T17:31:25Z

Latest push was a minor docstring change, and rebase

JacobCallahan

ACK, nice work!

* API-factory setup registered host method * UI::ERRATA (cherry picked from commit 0e90a95)

Errata WIP for remaining UI Failures (#15085) * API-factory setup registered host method * UI::ERRATA (cherry picked from commit 0e90a95) Co-authored-by: David Moore <109112035+damoore044@users.noreply.github.com>

* API-factory setup registered host method * UI::ERRATA

damoore044 added No-CherryPick PR doesnt need CherryPick to previous branches Stream Introduced in or relating directly to Satellite Stream/Master labels May 16, 2024

damoore044 self-assigned this May 16, 2024

damoore044 force-pushed the Errata-WIP branch 2 times, most recently from 09216c3 to 291c4b5 Compare May 16, 2024 17:57

damoore044 force-pushed the Errata-WIP branch from 291c4b5 to 2b98d0f Compare May 16, 2024 19:40

Satellite-QE added the PRT-Failed Indicates that latest PRT run is failed for the PR label May 16, 2024

damoore044 force-pushed the Errata-WIP branch from 2b98d0f to 0b7ee6f Compare May 16, 2024 20:03

damoore044 force-pushed the Errata-WIP branch from 0b7ee6f to 38ef57e Compare May 17, 2024 13:56

damoore044 force-pushed the Errata-WIP branch from 38ef57e to 6d2d124 Compare May 17, 2024 17:17

damoore044 force-pushed the Errata-WIP branch from 6d2d124 to 45cff78 Compare May 17, 2024 17:46

damoore044 force-pushed the Errata-WIP branch from d9d8b0b to 6dfa067 Compare May 17, 2024 19:43

damoore044 added 6.15.z Introduced in or relating directly to Satellite 6.15 CherryPick PR needs CherryPick to previous branches and removed No-CherryPick PR doesnt need CherryPick to previous branches labels May 17, 2024

damoore044 mentioned this pull request May 18, 2024

Errata install, and legacy chost methods SatelliteQE/airgun#1387

Merged

damoore044 force-pushed the Errata-WIP branch from 6dfa067 to 5799fa3 Compare May 18, 2024 13:50

damoore044 requested review from a team June 17, 2024 18:09

Satellite-QE added PRT-Passed Indicates that latest PRT run is passed for the PR and removed PRT-Failed Indicates that latest PRT run is failed for the PR labels Jun 17, 2024

Satellite-QE mentioned this pull request Jun 18, 2024

[6.15.z] Errata install, and legacy chost methods SatelliteQE/airgun#1428

Merged

vsedmik reviewed Jun 18, 2024

View reviewed changes

damoore044 force-pushed the Errata-WIP branch from 2251d28 to e171a51 Compare June 20, 2024 19:10

Satellite-QE removed the PRT-Passed Indicates that latest PRT run is passed for the PR label Jun 20, 2024

damoore044 force-pushed the Errata-WIP branch from e171a51 to f805490 Compare June 20, 2024 19:13

damoore044 force-pushed the Errata-WIP branch 2 times, most recently from d666f17 to a2c1181 Compare June 20, 2024 19:34

Satellite-QE added the PRT-Passed Indicates that latest PRT run is passed for the PR label Jun 20, 2024

vsedmik approved these changes Jun 21, 2024

View reviewed changes

damoore044 added 2 commits June 21, 2024 13:30

API-factory setup registered host method

524944b

UI::ERRATA

f2d598f

damoore044 force-pushed the Errata-WIP branch from a2c1181 to f2d598f Compare June 21, 2024 17:31

Satellite-QE removed the PRT-Passed Indicates that latest PRT run is passed for the PR label Jun 21, 2024

JacobCallahan approved these changes Jun 21, 2024

View reviewed changes

JacobCallahan merged commit 0e90a95 into SatelliteQE:master Jun 21, 2024
9 checks passed

github-actions bot pushed a commit that referenced this pull request Jun 21, 2024

Errata WIP for remaining UI Failures (#15085)

fc82bb4

* API-factory setup registered host method * UI::ERRATA (cherry picked from commit 0e90a95)

Satellite-QE mentioned this pull request Jun 21, 2024

[6.15.z] Errata WIP for remaining UI Failures #15483

Merged

damoore044 added a commit that referenced this pull request Jun 24, 2024

Errata WIP for remaining UI Failures (#15085)

c7c6728

* API-factory setup registered host method * UI::ERRATA (cherry picked from commit 0e90a95)

jyejare pushed a commit to jyejare/robottelo that referenced this pull request Oct 19, 2024

Errata WIP for remaining UI Failures (SatelliteQE#15085)

515b93f

* API-factory setup registered host method * UI::ERRATA

		for key in actual_values:
		assert expected_values[key] in actual_values[key], 'Expected text not found'

	for key in actual_values:
	assert expected_values[key] in actual_values[key], 'Expected text not found'
	assert expected_values == actual_values, 'Expected text not found'

Errata WIP for remaining UI Failures #15085

Errata WIP for remaining UI Failures #15085

Conversation

damoore044 commented May 16, 2024 • edited Loading

Problem Statement

Ongoing Failures

PRT Case

damoore044 commented May 16, 2024

damoore044 commented May 16, 2024

Satellite-QE commented May 16, 2024

damoore044 commented May 16, 2024

Satellite-QE commented May 16, 2024

damoore044 commented May 17, 2024

Satellite-QE commented May 17, 2024

damoore044 commented May 17, 2024

Satellite-QE commented May 17, 2024

damoore044 commented May 17, 2024

damoore044 commented May 17, 2024

Satellite-QE commented May 17, 2024

damoore044 commented May 18, 2024

damoore044 commented Jun 17, 2024

Satellite-QE commented Jun 17, 2024

vsedmik left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damoore044 Jun 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damoore044 Jun 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damoore044 commented Jun 20, 2024

damoore044 commented Jun 20, 2024

Satellite-QE commented Jun 20, 2024

damoore044 commented Jun 21, 2024

JacobCallahan left a comment

Choose a reason for hiding this comment

damoore044 commented May 16, 2024 •

edited

Loading

vsedmik left a comment •

edited

Loading

damoore044 Jun 20, 2024 •

edited

Loading

damoore044 Jun 20, 2024 •

edited

Loading