History reports not linked

[mattsmith321]

Hi everyone,

I'm new to Allure this week but so far everything has fallen in place pretty easily based on the documentation. I am running Allure 2.30.0 on Windows and I am able to successfully generate and view the report.

However, I am now digging into history and retries and running into an issue where the history reports are listed on the History tab but they are not linked. If I run my tests multiple times and generate the report without doing the history steps, I do get linked retry reports on the Retries tab.

I am following the documentation at How to enable history and have tried several different variations of those instructions to no avail.

What am I missing?

[baev]

For history links to work, you need to provide the report URL. You can do this via the executor.json file.

[mattsmith321]

Oh. I see that the executor.json file is expected/generated from the CI tool. This is actually forcing me to confront the fact that I'm trying to use Allure differently than intended.

From the first line of the docs: "Allure Report is a popular open source tool for visualizing the results of a test run." I'm trying to aggregate test results across multiple test runs and multiple sources, including manual runs. Something similar to sending test results to an ELK Stack and then configuring a Kibana dashboard.

The reason I am down this path is a little hard to explain, especially without full context but I was excited to find Allure Report and the features that it offered. I just might be using differently than intended.

[mattsmith321]

Adding the executor.json file did provide a link for the reports in the History tab. Now to figure out what it should actually be linking to and whether or not I can get Allure to do what I want.

[baev]

I'm trying to aggregate test results across multiple test runs and multiple sources, including manual runs.

All that is available as the part of Allure TestOps, you should give it a try.

[baev]

Oh. I see that the executor.json file is expected/generated from the CI tool. This is actually forcing me to confront the fact that I'm trying to use Allure differently than intended.

The Allure Report is mostly focused on visualising the single test run. However, Allure enriches the report with historical data. Since we're limited to the single run, all we can provide is the generic meta (like success rate and previous status) and the link to the previous reports.

[mattsmith321]

I'm trying to aggregate test results across multiple test runs and multiple sources, including manual runs.

All that is available as the part of Allure TestOps, you should give it a try.

Ha! I wish I could. Unfortunately we are not in a position to make that jump.

For now I am going to hack something together. Working on sending the test results to a central location. Then having a script to generate the reports on a regular basis from any runs that have occurred. The report would get posted to a yyyy-mm\yyyy-mm-dd folder served up by the web server. Then there would be a daily script involved to try to keep history from each day. It's definitely a hack but I think it might work for some basic needs. Then if that gets some traction and people start running into limitations, then I can make the pitch for Allure TestOps. But I'm not holding my breath. Especially since we are an incumbent SI vendor for a State agency and the primary technology decisions have recently been given to a different SI vendor. This is really just me trying to scratch an itch that is too difficult to reach if I have to run the bureaucracy gauntlet.

Thanks for your help and the pointer. Not sure what the process is but it might be good if the "How to enable history" section included something along the lines of your original comment about the executor.json so that someone else doesn't spend 3-4 hours trying to figure out why it isn't working.