Why aren't my unit tests being discovered by VSTest?



  • So I'm trying to be good about writing tests. I have currently two test classes in separate files:

    CampaignTest.cs -- CampaignTest
                       |- TestAddCharacter()
    
    DataTypesTest.cs -- DataTypesTest
                       |- TestAC()
                       |- TestDiceValuePositiveMod()
                       |- (total of 12 tests so far in this class)
    

    All the methods are marked with [TestMethod], and the classes are marked with [TestClass].

    But only the last three of the 12 in one file are being run. The other file is ignored completely, and the first 9 show up as inconclusive (and don't display even a method name).

    :wtf: is going on?



  • @benjamin-hall Are you using the latest MSTest framework and adapter? Does your default processor architecture for testing match that of your code under test? Have you tried re-opening VS?



  • @magus said in Why aren't my unit tests being discovered by VSTest?:

    @benjamin-hall Are you using the latest MSTest framework and adapter? Does your default processor architecture for testing match that of your code under test? Have you tried re-opening VS?

    Yes (I think? Nothing has been bugging for updates and I'm running VS2017)

    Should be--both are set to Any CPU

    Many many times across multiple machines (laptop and desktop, both synced to a repo). One's even a mac (using VSMac). Problem persists on both.

    Edit: the using line is
    using Microsoft.VisualStudio.TestTools.UnitTesting;
    and I haven't (consciously) pulled in a new nuget package.


  • Fake News

    All our current tests are built using xUnit but I seem to recall that MSTest would cache the list of test classes and fail to refresh.

    I have found a StackOverflow thread about missing tests in 2015, could you check if those folders would still apply?

    (Note that Visual Studio 2017 has internal version 15.0 IIRC)



  • @jbert said in Why aren't my unit tests being discovered by VSTest?:

    All our current tests are built using xUnit but I seem to recall that MSTest would cache the list of test classes and fail to refresh.

    I have found a StackOverflow thread about missing tests in 2015, could you check if those folders would still apply?

    (Note that Visual Studio 2017 has internal version 15.0 IIRC)

    I looked for those folders and don't seem to find them anywhere...not sure where the %TEMP% folder is on VSMac (which is what I'm using right now).



  • @benjamin-hall said in Why aren't my unit tests being discovered by VSTest?:

    and I haven't (consciously) pulled in a new nuget package.

    Check that. Pull the latest. MSTest2 is nuget-based, and doesn't even require its own project type, so old packages could potentially cause issues.



  • @magus said in Why aren't my unit tests being discovered by VSTest?:

    @benjamin-hall said in Why aren't my unit tests being discovered by VSTest?:

    and I haven't (consciously) pulled in a new nuget package.

    Check that. Pull the latest. MSTest2 is nuget-based, and doesn't even require its own project type, so old packages could potentially cause issues.

    Checked--I'm on 1.2 which seems to be the latest.



  • @benjamin-hall Do you know if alternate test runners can pick up the tests? Live Unit Testing (built in), NCrunch, or Resharper's?

    Also, are any of your tests asynchronous? MSTest ignores public async void tests, but not public async Task tests.



  • @magus said in Why aren't my unit tests being discovered by VSTest?:

    @benjamin-hall Do you know if alternate test runners can pick up the tests? Live Unit Testing (built in), NCrunch, or Resharper's?

    Also, are any of your tests asynchronous? MSTest ignores public async void tests, but not public async Task tests.

    Live Unit Tests requires enterprise (I'm on community). Haven't tried NCrunch, don't own (or want to pay for Re#).

    None are async. All are public void. They're all "baby's first unit tests:"

    Not-working (tests the code that adds characters to the right bucket based on role):

    [TestMethod]
            public void TestAddCharacter()
            {
                var testPc = new Character(CharacterType.PC);
                var testEnemy = new Character(CharacterType.Monster);
                var testAlly = new Character(CharacterType.Ally);
                var c = new Campaign();
                c.AddCharacter(testPc);
                Assert.AreEqual(1, c.PCs.Count());
                c.AddCharacter(testAlly);
                Assert.AreEqual(1, c.NPCs.Count());
                Assert.AreNotEqual(2, c.PCs.Count());
                c.AddCharacter(testEnemy);
                Assert.AreNotEqual(2, c.NPCs.Count());
                Assert.AreNotEqual(3, c.PCs.Count());
            }
    

    working (tests that the output is right for a couple different cases)--

            [TestMethod]
            public void TestCreatureFeature()
            {
                var dv = new DiceValue(1, 6, 2);
                var ar = new AttackRoll("Test", 4, dv);
                var obj = new CreatureFeature(FeatureType.Action, "test", "test", ar);
                var nullobj = new CreatureFeature(FeatureType.Legendary, "test", "test", null);
                Assert.AreEqual(ar.ToString(), obj.Feature.attack);
                Assert.AreEqual("", nullobj.Feature.attack);
                Assert.IsInstanceOfType(obj.Feature, typeof(Schemas.creatureAction));
            }
    

  • Impossible Mission - B

    You should use NUnit. Then your tests would be discovered every time... after churning for five minutes every single time you rebuild.



  • @benjamin-hall That's super weird. Another thing you could try is creating a class library, pulling in the MSTest framework and adapter packages, and moving your tests there, then restarting VS. It usually doesn't detect tests in normal class libraries until after a full IDE restart for some reason, but that's the configuration I typically use now.

    I really am baffled, though. You're doing everything right, and as normally as possible it seems like.



  • @magus said in Why aren't my unit tests being discovered by VSTest?:

    @benjamin-hall That's super weird. Another thing you could try is creating a class library, pulling in the MSTest framework and adapter packages, and moving your tests there, then restarting VS. It usually doesn't detect tests in normal class libraries until after a full IDE restart for some reason, but that's the configuration I typically use now.

    I really am baffled, though. You're doing everything right, and as normally as possible it seems like.

    I'll try that when I get home (so I'm not on this crappy Mac running a knock-off Visual Studio). It may be that the tests work there (I just don't remember). If so, I'll just not test on the mac. Otherwise, I'll try your solution.

    Thanks for the help!



  • @Magus I'm going to chalk it up to VSMac being stupid. The tests discover and work fine on my Windows box at home. I'll just remember not to try testing while at school.


Log in to reply