Unit testing is a common software development process used to ensure software quality. In unit testing each part of the software being developed is tested in isolation to ensure it works correctly. Microsoft Visual Studio contains a built-in unit testing tool (MSTest.exe or VSTest.Console.exe) that comes with all editions of Visual Studio (including the Express Editions), but there are other unit testing tools available for .NET code (e.g. NUnit, xUnit.net, etc.) that can be used as well.
Regardless of which tool is used for unit testing, a question that inevitably comes up is “when are we done writing unit tests?” The answer depends on the level of testing that is required or desired. Ideally unit tests will execute each line of code at least once. This level of code coverage is referred to as Statement Coverage. Other common levels include Branch Coverage (every branch of every control structure has been executed) and Condition Coverage (both the true and false case of every Boolean sub-expression has been executed).
Without a code coverage tool, it is very difficult to know for sure if each line in the code under test has been executed. It is even more difficult to know if every branch or condition in the code under test has been executed.
If one is fortunate enough to be on a project where Visual Studio Premium or Ultimate is used, the built-in code coverage tool can be used. Unfortunately, one is often restricted to Visual Studio Professional (or even Visual Studio Express) which does not come with a built-in code coverage tool.
Luckily the open source community has two free tools available to .NET developers that provide code coverage and reporting: OpenCover (http://opencover.codeplex.com/) and ReportGenerator (http://www.palmmedia.de/OpenSource/ReportGenerator).
OpenCover
OpenCover is an open source code coverage tool that can be used with both the 32-bit and 64-bit versions of the Microsoft .NET Framework 2.0 and above. The easiest way to install OpenCover is via NuGet, but it can also be downloaded from the OpenCover website (http://opencover.codeplex.com/). After it is installed, a user manual can be found in the OpenCover “docs” directory.
OpenCover is run from the Windows Command Prompt. The following is a simple example showing how to run OpenCover to record the code coverage of a set of unit tests:
OpenCover.Console.exe ‑register:user ‑target:"VSTest.Console.exe" ‑targetargs:"MyUnitTests.dll" ‑output:MyCodeUnderTest.Results.xml "-filter:+[MyCodeUnderTest]*"
For the example above:
· OpenCover.Console.exe is the OpenCover tool.
· -register:user allows the tool to register and un-register the code coverage profiler using a non-administrator user account.
· -target:"VSTest.Console.exe" indicates what application OpenCover should execute. In this case OpenCover will run the built-in Visual Studio unit testing tool (VSTest.Console.exe) that comes with Visual Studio 2012 and 2013. If another unit testing tool is being used, it would be specified here. For example, NUnit would be nunit-console.exe, Visual Studio 2010’s unit testing tool would be MSTest.exe, etc.
· -targetargs:"MyUnitTest.dll" specifies the command line options OpenCover should use when executing the application specified by the ‑target command line argument. In this case the command line option is the name of the assembly that contains the unit tests that the VSTest.Console.exe tool should execute.
· -output:MyCodeUnderTest.Results.xml specifies the output file for the code coverage results.
· "-filter:+[MyCodeUnderTest]*" specifies that all of the code in the MyCodeUnderTest assembly should be included in the code coverage results. The “–filter:” command line option can contain a space separated list of filters to apply to selectively include or exclude assemblies and classes from the coverage results. The format of the filters is ±[assemblyName]className. If no filter is provided, the default “include all” filter (i.e. +[*]*) is used.
The output file that is generated by the OpenCover tool is an XML document. OpenCover includes a simple XSLT that can be used to transform the output file into an HTML document, but the OpenCover output file can be used by other tools, such as ReportGenerator, to generate nice code coverage reports.
ReportGenerator
ReportGenerator is an open source tool that converts the XML reports generated by OpenCover, PartCover, Visual Studio and NCover into an HTML report containing the code coverage statistics and a visualization of which lines in the source code have been covered.
Example Summary Screenshot
Example Details Screenshot
ReportGenerator is run from the Windows Command Prompt. The following is a simple example showing how to run ReportGenerator to generate an HTML code coverage report for the OpenCover output file generated in the previous example:
ReportGenerator.exe "-reports:*Results.xml" "-targetdir:codecoverage/"
For the example above:
· ReportGenerator.exe is the ReportGenerator tool.
· "-reports:*Results.xml" specifies that any code coverage results files with a name ending in “Results.xml” should be included in the report.
· "-targetdir:codecoverage/" indicates the folder where the output code coverage report can be found.
Automating the Process
It is often helpful to create a batch file to automate the process of running OpenCover and ReportGenerator and displaying the results. The following is an example batch file that uses the Google Chrome web browser to display the output from OpenCover and ReportGenerator:
@ECHO OFF
REM Setup Browser
IF EXIST %USERPROFILE%\AppData\Local\Google\Chrome\Application\chrome.exe SET Browser=%USERPROFILE%\AppData\Local\Google\Chrome\Application\chrome.exe
IF EXIST "%ProgramFiles(x86)%\Google\Chrome\Application\chrome.exe" SET Browser=%ProgramFiles(x86)%\Google\Chrome\Application\chrome.exe
REM Setup OpenCover & RepportGenerator & MSTest
SET OpenCover=packages\OpenCover.4.5.2506\OpenCover.Console.exe
SET ReportGenerator=packages\ReportGenerator.1.9.1.0\ReportGenerator.exe
SET MSTest=%VS110COMNTOOLS%\..\IDE\CommonExtensions\Microsoft\TestWindow\VSTest.Console.exe
REM Setup Output Directory
IF NOT EXIST codecoverage MKDIR codecoverage
del codecoverage\*.* /Q
ECHO Execute Tests with code coverage...
%OpenCover% -register:user -target:"%MSTest%" -targetargs:"MyUnitTests\bin\Debug\MyUnitTests.dll" -output:codecoverage/MyUnitTests.Results.xml "-filter:+[MyCodeUnderTest]*"
ECHO.
ECHO Generate Code Coverage Reports...
%ReportGenerator% "-reports:codecoverage/*Results.xml" "-targetdir:codecoverage/"
ECHO.
REM View Report
"%Browser%" "%CD%\codecoverage\index.htm"
The example above is written for Visual Studio 2012 with OpenCover and ReportGenerator installed using NuGet. The batch file above should be located in the same folder as the Visual Studio solutions (*.sln) file.
Summary
Although not as full-featured as the built-in code coverage tools in Visual Studio Premium or Ultimate, the open source CodeCoverage and ReportGenerator tools provide reasonable code coverage reporting when Visual Studio Premium or Ultimate are not available.
