Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify big test config templating for modules #3238

Merged
merged 8 commits into from
Sep 2, 2021
Merged

Conversation

chrzaszcz
Copy link
Member

@chrzaszcz chrzaszcz commented Aug 31, 2021

Improve the consistency of the module config templates used for tests.

Use test.config options for static and dynamic domains and do not repeat the same configuration in test suites.
This applies to two modules recently converted to use dynamic domains: mod_private and mod_vcard.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Aug 31, 2021

small_tests_24 / small_tests / 914ef28
Reports root / small


internal_mnesia_24 / internal_mnesia / 914ef28
Reports root/ big
OK: 1588 / Failed: 0 / User-skipped: 288 / Auto-skipped: 0


small_tests_22 / small_tests / 914ef28
Reports root / small


dynamic_domains_24 / pgsql_mnesia / 914ef28
Reports root/ big
OK: 1690 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


dynamic_domains_23 / pgsql_mnesia / 914ef28
Reports root/ big
OK: 1690 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


small_tests_23 / small_tests / 914ef28
Reports root / small


ldap_mnesia_24 / ldap_mnesia / 914ef28
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


ldap_mnesia_22 / ldap_mnesia / 914ef28
Reports root/ big
OK: 1518 / Failed: 1 / User-skipped: 361 / Auto-skipped: 0

mod_event_pusher_rabbit_SUITE:group_chat_message_publish:group_chat_message_received_event_properly_formatted
{error,
  {{assertMatch,
     [{module,mod_event_pusher_rabbit_SUITE},
      {line,435},
      {expression,
        "get_decoded_message_from_rabbit ( AliceGroupChatMsgRecvRK )"},
      {pattern,
        "# { << \"from_user_id\" >> := BobRoomJID , << \"to_user_id\" >> := AliceFullJID , << \"message\" >> := Message }"},
      {value,
        #{<<"from_user_id">> => <<"muc_publish@muc.localhost">>,
        <<"message">> => <<>>,
        <<"to_user_id">> =>
          <<"alice_unnamed_11.330969@localhost/res1">>}}]},
   [{mod_event_pusher_rabbit_SUITE,
      '-group_chat_message_received_event_properly_formatted/1-fun-0-',3,
      [{file,
         "/home/circleci/app/big_tests/tests/mod_event_pusher_rabbit_SUITE.erl"},
       {line,435}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1754}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1263}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1195}]}]}}

Report log


ldap_mnesia_23 / ldap_mnesia / 914ef28
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


mssql_mnesia_24 / odbc_mssql_mnesia / 914ef28
Reports root/ big
OK: 3099 / Failed: 1 / User-skipped: 184 / Auto-skipped: 0

pep_SUITE:pep_tests:unsubscribe_after_presence_unsubscription
{error,
  {{badmatch,
     [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alice_unsubscribe_after_presence_unsubscription_16.633933@localhost">>},
         {<<"to">>,
        <<"bob_unsubscribe_after_presence_unsubscription_16.633933@localhost/res1">>},
         {<<"type">>,<<"headline">>}],
        [{xmlel,<<"event">>,
           [{<<"xmlns">>,
           <<"http://jabber.org/protocol/pubsub#event">>}],
           [{xmlel,<<"items">>,
            [{<<"node">>,<<"+lqMFazpbCtO+J5zauoPAw==">>}],
            [{xmlel,<<"item">>,
               [{<<"id">>,<<"salmon">>}],
               [{xmlel,<<"entry">>,
                  [{<<"xmlns">>,
                  <<"http://www.w3.org/2005/Atom">>}],
                  []}]}]}]},
         {xmlel,<<"headers">>,
           [{<<"xmlns">>,<<"http://jabber.org/protocol/shim">>}],
           []}]}]},
   [{pep_SUITE,'-unsubscribe_after_presence_unsubscription/1-fun-0-',2,
      [{file,"/home/circleci/app/big_tests/tests/pep_SUITE.erl"},
       {line,382}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1292}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1224}]}]}}

Report log


pgsql_mnesia_23 / pgsql_mnesia / 914ef28
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


mysql_redis_24 / mysql_redis / 914ef28
Reports root/ big
OK: 3070 / Failed: 1 / User-skipped: 201 / Auto-skipped: 0

service_domain_db_SUITE:db:db_keeps_syncing_after_cluster_join
{error,{test_case_failed,{[<<"example1.com">>,<<"example2.com">>,
               <<"example3.com">>],
              [<<"example1.com">>,<<"example2.com">>,
               <<"example3.com">>,<<"example4.com">>]}}}

Report log


elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 914ef28
Reports root/ big
OK: 1891 / Failed: 0 / User-skipped: 284 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 914ef28
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


pgsql_mnesia_22 / pgsql_mnesia / 914ef28
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / 914ef28
Reports root/ big
OK: 1738 / Failed: 0 / User-skipped: 287 / Auto-skipped: 0

@codecov
Copy link

codecov bot commented Aug 31, 2021

Codecov Report

Merging #3238 (562798e) into master (3e61c13) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #3238   +/-   ##
=======================================
  Coverage   80.35%   80.35%           
=======================================
  Files         398      398           
  Lines       32526    32526           
=======================================
  Hits        26135    26135           
  Misses       6391     6391           
Impacted Files Coverage Δ
src/event_pusher/mod_event_pusher_sns.erl 84.21% <0.00%> (-5.27%) ⬇️
src/domain/mongoose_domain_loader.erl 75.47% <0.00%> (-1.89%) ⬇️
src/rdbms/mongoose_rdbms.erl 60.88% <0.00%> (-0.81%) ⬇️
src/offline/mod_offline.erl 77.90% <0.00%> (-0.56%) ⬇️
src/mod_muc_log.erl 77.88% <0.00%> (ø)
src/ejabberd_c2s.erl 89.20% <0.00%> (ø)
src/mod_muc_room.erl 77.18% <0.00%> (+0.05%) ⬆️
src/pubsub/mod_pubsub.erl 73.23% <0.00%> (+0.05%) ⬆️
src/ejabberd_sm.erl 84.59% <0.00%> (+0.32%) ⬆️
src/global_distrib/mod_global_distrib_utils.erl 65.42% <0.00%> (+0.93%) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e61c13...562798e. Read the comment docs.

- For static domains, there are two distinct host type
- For dynamic domains, there is only one host type
@mongoose-im
Copy link
Collaborator

mongoose-im commented Sep 1, 2021

small_tests_24 / small_tests / 50d8df9
Reports root / small


small_tests_23 / small_tests / 50d8df9
Reports root


dynamic_domains_23 / pgsql_mnesia / 50d8df9
Reports root


ldap_mnesia_23 / ldap_mnesia / 50d8df9
Reports root


internal_mnesia_24 / internal_mnesia / 50d8df9
Reports root/ big
OK: 1588 / Failed: 0 / User-skipped: 288 / Auto-skipped: 0


pgsql_mnesia_23 / pgsql_mnesia / 50d8df9
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


dynamic_domains_23 / pgsql_mnesia / 50d8df9
Reports root


small_tests_22 / small_tests / 50d8df9
Reports root


ldap_mnesia_22 / ldap_mnesia / 50d8df9
Reports root


small_tests_23 / small_tests / 50d8df9
Reports root / small


ldap_mnesia_23 / ldap_mnesia / 50d8df9
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


dynamic_domains_23 / pgsql_mnesia / 50d8df9
Reports root/ big
OK: 1690 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


small_tests_23 / small_tests / 50d8df9
Reports root / small


ldap_mnesia_23 / ldap_mnesia / 50d8df9
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


ldap_mnesia_22 / ldap_mnesia / 50d8df9
Reports root


small_tests_22 / small_tests / 50d8df9
Reports root / small


pgsql_mnesia_22 / pgsql_mnesia / 50d8df9
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented Sep 2, 2021

small_tests_24 / small_tests / b2dc276
Reports root / small


internal_mnesia_24 / internal_mnesia / b2dc276
Reports root/ big
OK: 1588 / Failed: 0 / User-skipped: 288 / Auto-skipped: 0


dynamic_domains_24 / pgsql_mnesia / b2dc276
Reports root/ big
OK: 1690 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


small_tests_22 / small_tests / b2dc276
Reports root / small


dynamic_domains_23 / pgsql_mnesia / b2dc276
Reports root/ big
OK: 1690 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


small_tests_23 / small_tests / b2dc276
Reports root / small


ldap_mnesia_24 / ldap_mnesia / b2dc276
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


ldap_mnesia_23 / ldap_mnesia / b2dc276
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


ldap_mnesia_22 / ldap_mnesia / b2dc276
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / b2dc276
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


pgsql_mnesia_22 / pgsql_mnesia / b2dc276
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / b2dc276
Reports root/ big
OK: 1891 / Failed: 0 / User-skipped: 284 / Auto-skipped: 0


pgsql_mnesia_23 / pgsql_mnesia / b2dc276
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


mssql_mnesia_24 / odbc_mssql_mnesia / b2dc276
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


mysql_redis_24 / mysql_redis / b2dc276
Reports root/ big
OK: 3142 / Failed: 2 / User-skipped: 201 / Auto-skipped: 0

amp_big_SUITE:basic:notify_deliver_to_stranger_test
{error,
  {{assertion_failed,assert,is_presence,
     {xmlel,<<"stream:error">>,[],
       [{xmlel,<<"conflict">>,
          [{<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-streams">>}],
          []},
        {xmlel,<<"text">>,
          [{<<"xml:lang">>,<<"en">>},
           {<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-streams">>}],
          [{xmlcdata,<<"Replaced by new connection">>}]}]},
     "<stream:error><conflict xmlns='urn:ietf:params:xml:ns:xmpp-streams'/><text xml:lang='en' xmlns='urn:ietf:params:xml:ns:xmpp-streams'>Replaced by new connection</text></stream:error>"},
   [{escalus_new_assert,assert_true,2,
      [{file,
         "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
       {line,84}]},
    {escalus_story,'-drop_presences/2-lc$^0/1-0-',1,
      [{file,
         "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,190}]},
    {escalus_story,drop_presences,2,
      [{file,
         "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,190}]},
    {escalus_story,'-start_ready_clients/2-fun-0-',3,
      [{file,
         "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,135}]},
    {lists,foldl,3,[{file,"lists.erl"},{line,1267}]},
    {escalus_story,start_ready_clients,2,
      [{file,
         "/home/circleci/app/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,128}]},
   ...

Report log

mam_SUITE:rdbms_prefs_cases:messages_filtered_when_prefs_default_policy_is_roster
{error,{test_case_failed,"ASSERT EQUAL\n\tExpected []\n\tValue [ok]\n"}}

Report log


riak_mnesia_24 / riak_mnesia / b2dc276
Reports root/ big
OK: 1738 / Failed: 0 / User-skipped: 287 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented Sep 2, 2021

small_tests_24 / small_tests / 562798e
Reports root / small


internal_mnesia_24 / internal_mnesia / 562798e
Reports root/ big
OK: 1588 / Failed: 0 / User-skipped: 288 / Auto-skipped: 0


small_tests_22 / small_tests / 562798e
Reports root / small


dynamic_domains_24 / pgsql_mnesia / 562798e
Reports root/ big
OK: 1690 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


dynamic_domains_23 / pgsql_mnesia / 562798e
Reports root/ big
OK: 1690 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


small_tests_23 / small_tests / 562798e
Reports root / small


ldap_mnesia_22 / ldap_mnesia / 562798e
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


ldap_mnesia_24 / ldap_mnesia / 562798e
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


ldap_mnesia_23 / ldap_mnesia / 562798e
Reports root/ big
OK: 1515 / Failed: 0 / User-skipped: 361 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 562798e
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


pgsql_mnesia_22 / pgsql_mnesia / 562798e
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


elasticsearch_and_cassandra_24 / elasticsearch_and_cassandra_mnesia / 562798e
Reports root/ big
OK: 1891 / Failed: 0 / User-skipped: 284 / Auto-skipped: 0


mssql_mnesia_24 / odbc_mssql_mnesia / 562798e
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


pgsql_mnesia_23 / pgsql_mnesia / 562798e
Reports root/ big
OK: 3088 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


mysql_redis_24 / mysql_redis / 562798e
Reports root/ big
OK: 3071 / Failed: 0 / User-skipped: 201 / Auto-skipped: 0


riak_mnesia_24 / riak_mnesia / 562798e
Reports root/ big
OK: 1738 / Failed: 0 / User-skipped: 287 / Auto-skipped: 0

@chrzaszcz chrzaszcz marked this pull request as ready for review September 2, 2021 10:44
@vkatsuba vkatsuba merged commit 5697ae0 into master Sep 2, 2021
@vkatsuba vkatsuba deleted the test-unification branch September 2, 2021 12:29
@Premwoik Premwoik added this to the 5.0.0 milestone Oct 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants