allure will mark flaky test as default category(Production defects or Test defects), the customized categories.json does not work for flaky test

[sukercjw]

Would like to know if anyone encounter this issue?
allure will mark flaky test as default category(Production defects or Test defects), the customized categories.json does not work for flaky test, even it has meen matched with test status and message regex. i am using pytest for allure
how could we fix that?

thanks a lot

[baev]

It seems like flaky tests will be only matched with categories with the flaky: true matcher; see https://github.com/allure-framework/allure2/blob/main/allure-generator/src/main/java/io/qameta/allure/category/CategoriesPlugin.java#L164:

final boolean matchesFlaky = result.isFlaky() == category.isFlaky();

Since category.flaky is boolean, it is false by default, so flaky tests will not be matched. The more correct solution IMHO would be to change the type of category.flaky to Boolean and change the condition:

final boolean matchesFlaky = Objects.isNotNull(category.isFlaky()) && result.isFlaky() == category.isFlaky();

This way flaky results will be matched to all the categories by default unless you directly add flaky: false matcher.

[baev]

I created a patch #2796