Same xmlsuite is added multiple times when tests are run using TestNG.SetXmlSuites method

[virenv]

@cbeust,

Here is the description of the problem.

My trimmed down run engine code

    Parser parser = new Parser(".\\CompleteSuite.xml");
    List<XmlSuite> xmlSuite = null;
    try {
        xmlSuite = parser.parseToList();
    } catch (ParserConfigurationException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (SAXException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    // List<String> suites = new ArrayList<String>();
    TestNG testng = new TestNG();
    TestListener listner = new TestListener();
    testng.addListener(listner);
    testng.addListener(new SuiteListner());
    // suites.add(".\\CompleteSuite.xml");
    testng.setXmlSuites(xmlSuite);
    // testng.setTestSuites(suites);
    testng.setParallel("parallel");
    testng.setSuiteThreadPoolSize(5);
    testng.setDataProviderThreadCount(5);
    testng.setOutputDirectory("path to output");
    testng.run();

My suite XML looks like this

<suite name="suite of suites">
    <suite-files>
        <suite-file path="TestNG.xml" />
        <suite-file path="TestNG2.xml" />
    </suite-files>
</suite>

you can have any two testNG suites as mentioned here.

Problem
When you run this code you will get an exception stating
Two suites cannot have the same name: Suite2

This exception will come even if you have two different suite names. The reason is because of this code in initializeSuitesAndJarFile of TestNG class

    for (String suiteFile : s.getSuiteFiles()) {
        try {
            Collection<XmlSuite> childSuites = getParser(suiteFile).parse();
            for (XmlSuite cSuite : childSuites){
                cSuite.setParentSuite(s);
                s.getChildSuites().add(cSuite);
            }
        }

here do we really need adding of same suite back to the parent? s.getChildSuites().add(cSuite);

Virender

[juherr]

Could you share the content of TestNG.xml and TestNG2.xml?

[virenv]

TestNG1 - >

<suite name="Suite2" verbose="1"  parallel="methods">
  <test name="test002" >
    <classes>
       <class name="TestClasses.TestClass002" />
    </classes>
  </test>
</suite>

TestNG2 ->

<suite name="Suite1" verbose="1" parallel="methods" thread-count="1">
  <test name="test001" >
    <classes>
       <class name="TestClasses.TestClass001" />
    </classes>
  </test>
</suite>

[virenv]

Hi juherr,

I am on testng 6.9.4. I would have tried the latest version but saw that this part of the code hasn't changed.

[virenv]

@juherr, Does it look like a bug?

[juherr]

Yes, it looks like. Would you like to try to fix it?
First thing to do is adding a test that reproduces the issue.

[virenv]

Let me take a look at it and will validate my understanding of the issue with the community.

Is there any documentation which talks about running TestNG unit tests on local machine?

[juherr]

running TestNG unit tests on local machine?

Just run testng.xml suite in IDE, or use the build tool you prefer between gradle, maven or kobalt.

[virenv]

Alright, let me put this up and I will send a PR for it.

[virenv]

It seems that this issue(behavior) was modified as a part of #829

[juherr]

In fact, I think #829 is just hiding the issue as it allows to have duplicate name.
But in our case, "suite of suites", Suite1" and Suite2" are not duplication.

Now, I think you have another issue: some suites are run twice.
Could you confirm?

[virenv]

@juherr

Yes suites are now run twice. Let me send a PR and lets discuss the impact of the fix with some code.

[virenv]

@cbeust @juherr in the context of this issue, the core problem is that in #829 we introduced a check to allow multiple suites with same name. I think it will be explicit that a suite with same name will be actually pointing to the same set of test. If this is the case we will have test sanity check that will not allow same test names in a run

 /**

• Ensure that two XmlTest within the same XmlSuite don't have the same name
  */
  private void checkTestNames(List suites) {

in testng.java

Am i correct with my understanding? if this is the case then the change in#829 might not have the desired functionality.

I will send a PR with my assumption and lets see how it goes.

[juherr]

I think the solution should not be to check what is duplication or what is not, but find why TestNG is adding the same suite many times and fix it.

[virenv]

@juherr This fix and the fix in #829 are related. Here is the PR, if we put this code we will not be able to have multiple suite files in a run. There by rendering the fix in 829 unusable

[vazotov]

Please check my solution for making your TestNG suites more DRY https://stackoverflow.com/a/51666801/4165898