Test list not refreshing if test project is not rebuilt

I have a use case where I get test data as names of files in the file system.

        [TestCaseSource(nameof(GetAllFileNames))]
        public void TestFile(string sourceFileNameWithExtension) {...}
        public static List<string> GetAllFileNames() {...}

The tests are not discovered properly. The test explorer sees the list of tests from the time the project was last rebuilt, but if the file system changes and test code doesn’t the list of tests will not be refreshed. The console shows: Error] Test adapter sent back a result for an unknown test case. Ignoring result for, which I guess means that test adapter did see the change from TestCaseSource, but VS Test Explorer refused to refresh the tests list. I can get the correct behavior by manually building the project after the file system change, which is inconvenient for me.

• NUnit 3.10.1
• NUnit3TestAdapter 3.10.0
• Visual Studio Community 2017 15.5.0
• Framework: netcoreapp2.0
• Runtime: portable

[OsirisTerje]

Discovery is being triggered by the build, and I see that when I just build (not rebuild) after adding a file, the project is being built, since msbuild detects there is a new file. However, the output is not changed, and that means that discovery doesn’t seem to be triggered, or the results not being acted upon.
So yes, I can repro the behaviour. I believe this has to do with VSTest, but I’ll check a bit more.
Steps: -I have 5 files (or whatever number you have)

• I have one trivial test, so should have 6 in total -Project is built/rebuilt and I see 6 tests
• I add a new file
• I just Build the solution The discovery is being started, but no new tests are discovered And Test Explorer are still showing 6

The test project now looks like:

So, it seems that the VSTest discovery is somehow deciding it doesn’t need to call the adapter. I think some MSFT need to have a look at this, I’ll point them to it.

[OsirisTerje]

The issue has not been voted up nor commented on at the developer community site, so closing it here.