Sunday, 19 April 2015

Could Not Detect Test Framework error when running ApprovalTests from an automated build

I was making some changes to an automated continuous integration build in TeamCity when I encountered an unexpected error. I had a suite of unit tests using the ApprovalTests framework but when the build was triggered all of the tests failed, tests which ran perfectly from within Visual Studio.

Investigation in TeamCity revealed the following error details (the same for each test):

Test(s) failed. System.Exception : 
Could Not Detect Test Framework

Either:
1) Optimizer Inlined Test Methods

Solutions:
a) Add [MethodImpl(MethodImplOptions.NoInlining)]
b) Set Build->Opitmize Code to False 
   & Build->Advanced->DebugInfo to Full

or
2) Approvals is not set up to use your test framework.
It currently supports [NUnit, MsTest, MbUnit, xUnit.net, xUnit.extensions, Machine.Specifications (MSpec)]

Solution:
To add one use ApprovalTests.Namers.StackTraceParsers.StackTraceParser.AddParser() method to add implementation of ApprovalTests.Namers.StackTraceParsers.IStackTraceParser with support for your testing framework.
To learn how to implement one see http://blog.approvaltests.com/2012/01/creating-namers.html
   at ApprovalTests.Namers.StackTraceParsers.StackTraceParser.Parse(StackTrace stackTrace)
   at ApprovalTests.Namers.UnitTestFrameworkNamer..ctor()
   at ApprovalTests.Approvals.<.cctor>b__c()
   at ApprovalTests.Approvals.GetDefaultNamer()
   at ApprovalTests.Approvals.Verify(IApprovalWriter writer)
   at ApprovalTests.Approvals.VerifyFile(String receivedFilePath)
   at ApprovalTests.Approvals.Verify(FileInfo receivedFilePath)

 

SNAGHTML472c94c

The first point to note is that the build was set to run using the Release configuration.

SNAGHTML4758484

The fix requires changes to the Visual Studio project settings for the test project.
Open the test project settings for the Release configuration. On the Build tab uncheck Optimize code and click Advanced…

SNAGHTML479828c

In the advanced settings dialog set Debug Info to full.

image

OK the dialog and save the project. Commit changes to source control and trigger the build again.

That should do it!

The error is caused by the absence of stacktrace information when the source is built using the Release configuration. Approvals Tests infers the name the approval file by walking the stack trace to get the test method name. No stacktrace, no inferred method name and therefore ApprovalTests can’t find the approval file.