I have an extensive suite of tests across four different class libraries in a VS solution, and I just started using CodeRush for Roslyn after using ReSharper for a couple years. In Resharper, there was an option to exclude tests by category, which I found useful because we have four test categories in our MSTest projects: UnitTest, IntegrationTest, DevTest, and SmokeTest. I wanted to exclude the DevTest and SmokeTest tests, because they require custom setup. When I entered these categories into the Resharper exclusion option, semicolon-delimited, the tests did not show up in my test runner as I expected they wouldn't.
When I try the same thing in the CodeRush test runner options (excluding DevTest and SmokeTest tests), the tests continue to appear in the Main test runner session, and continue to be run every time. I've tried adding these two categories to the exclusion textbox with a single semicolon between them, with a semicolon and a space between then, with a semicolon between and after them, and nothing seems to work. And after making the changes to the test options, I always click Apply. The only thing that prevents them from being run is adding the Ignore attribute to the tests in question and unchecking the "Show Ignored" button.
I've also tried grouping by category, which works some of the time, and grabbing the unit and integration tests and putting them into a new session and running those tests, but when I choose Run All, those tests don't even run (it shows me 0 tests have completed), but I suspect that's another bug.
If I'm misunderstanding how this option is supposed to work, please enlighten me, as I would like to be able to suppress DevTest and SmokeTest tests from being run in the test runner.
If it's any help, the attributes for the TestMethod and TestCategory are together inside one set of brackets above each test method, like so:
[TestMethod, TestCategory("DevTest")]
public void TestIDontWantToRun()
{
}
Kerry,
Thank you for your report.
It appears that this is an issue in our current implementation of the MSTest framework provider. Excluded categories are not considered and are removed after running test cases.
I have passed this ticket to our developers. You will be notified as soon as this issue is resolved.