Parallel test execution mixes up standard output from all tests

[arnath]

Description

If you enable parallel test execution, the standard output from the tests gets all mixed together.

Steps to reproduce

1. Enable parallel execution of tests.
2. Have several tests that produce some output to stdout (e.g., using Console.WriteLine).

Expected behavior

Each test's output would be isolated in the output display.

Actual behavior

The output from all the tests gets mixed together, making them all useless.

Environment

On the latest version of mstest v2.

[arnath]

TestContext.WriteLine does not have this same problem. However, because the TestContext is not available globally, I would either have to pass it around everywhere or do some super weird hacks to get it to be available for all my stdout logging.

[dannycrone]

Hi arnath, I am also getting this and it does not matter if I use Console.WriteLine or TestContext.WriteLine, it all gets mixed up. Sometimes one test gets all of the output from the other test. I am using Specflow on top aswell. This is causing major grief

[dannycrone]

I tell a lie, TestContext.WriteLine is working just fine

[karanjitsingh]

@arnath, if you want to isolate test related output TestContext is the only way. For output that needs to be immediately visible on console, it's by the virtue of its nature that with parallel tests output will get mixed up.

You can keep TestContext as a static variable in your class, I really won't consider that to be a hack especially when writing tests.

[arnath]

@karanjitsingh As far as I can tell, the static thing literally doesn't work. If you try to define a static property, you get the following error:

class UnitTests.UnitTest1 does not have valid TestContext property. TestContext must be of type TestContext, must be non-static, public and must not be read-only. For example: public TestContext TestContext.

If you use a static variable and assign it from ClassInitialize, one of the tests gets all the output (which is equally unhelpful).

[karanjitsingh]

@arnath Not sure how you're using it, but the following absolutely works

using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;

namespace UnitTestProject15
{
    [TestClass]
    public class UnitTest1
    {
        public static TestContext context;

        [ClassInitialize]
        public static void ClassInitialize(TestContext testContext)
        {
            context = testContext;
        }

        [TestMethod]
        public void TestMethod1()
        {
            context.WriteLine("Test method");
        }
    }
}

[arnath]

@karanjitsingh
If you run this with parallel execution enabled, you get one test with no output and one test with 20 lines interleaved of "Test1" and "Test2" as output.

    [TestClass]
    public class UnitTest1
    {
        private static TestContext TestContext;

        [ClassInitialize]
        public static void ClassInitialize(TestContext testContext)
        {
            TestContext = testContext;
        }

        [TestMethod]
        public async Task Test1()
        {
            for (int i = 0; i < 10; i++)
            {
                TestContext.WriteLine("Test1");
                await Task.Delay(500);
            }
        }

        [TestMethod]
        public async Task Test2()
        {
            for (int i = 0; i < 10; i++)
            {
                TestContext.WriteLine("Test2");
                await Task.Delay(500);
            }
        }
    }

[karanjitsingh]

@karanjitsingh
If you run this with parallel execution enabled, you get one test with no output and one test with 20 lines interleaved of "Test1" and "Test2" as output.

Sorry about that I meant to suggest the TestContext property.

Try using keeping public TestContext TestContext { get; set; } in your class and it should get populated automatically.

[JayHatchTesting]

Did you ever find a solution to this @arnath ? When trying the latest comment using TestContext property, one test gets the output of every other test running rather than outputting to each individual test.

[Evangelink]

@JayHatchTesting this should be fixed on MSTest v3.1.1 that we just released. If you are still experiencing any issue, please do create a new issue and try to share a repro with us so we can work on fixing it.

[JayHatchTesting]

It does work now thanks but have a different issue with Teamcity jumbling up the ordering of the test context messages.

[Evangelink]

@JayHatchTesting If it's on TeamCity there is nothing I can do but if there is another issue with MSTest please feel free to create another issue with as much content as you can so we can investigate!