I’ve recently been getting my head around some new technologies that we were looking to use in an upcoming project. With a chance to work with the .Net Core framework as part of this, I wanted to look at unit testing frameworks to see what worked well. I’m familiar with MSTest in the .Net space but there’s been a shift towards xUnit as an alternative.
What is xUnit?
Simply put, xUnit is another unit testing tool. What makes it different, according to its description on GitHub, is that it is free, open source and community-focused, written by the creator of NUnit v2 (another well-established testing framework).
There are two types of test in xUnit and they are identified by the attributes ‘Fact’ and ‘Theory’ rather than ‘Test’ as you might expect in other frameworks. A fact is a test that is always true, the conditions around the test do not change. A theory on the other hand, is true for a given set of data and unlike the fact tests we pass this data into the test. I’ll look at theory tests later when we talk about data-driven testing but to begin with I’ll start simple.
Simple Time Manager
To play with xUnit I needed a .Net Core application to test so, using Visual Studio, I made the simplest thing I could think of; a C# console app that lets you create tasks and work on them. Under the guise of being a zero-distraction productivity tool in the command line, it’s a minimum viable product so I could focus on testing. This article will be based on tests I wrote for this, so it’ll be helpful to look at the project files on GitHub.
Let’s Start Testing
First we want a new project for our tests so in the solution, I add an ‘xUnit Test Project (.Net Core)’ giving it a sensible name (the name of the project you’re testing with .Tests appended is a good convention).
This creates our test project, includes dependencies for xUnit and creates the first unit test class for us. In that class is an empty test method using the ‘Fact’ attribute we talked about. This is everything we need to create our first test.
using System;
using Xunit;
namespace SimpleTimeManager.Tests
{
public class UnitTest1
{
[Fact]
public void Test1()
{
}
}
}
I want to test the SimpleTask class so in the interest of readability I start by renaming this test class to ‘SimpleTaskShould’. Following the Arrange Act Assert methodology makes for clear tests that are easy to understand so we will take that approach. The arrange step for my tests will mostly be the same as they focus around a SimpleTask object which I will want to create for each test.
Rather than including it at the start of each test method I can use the class constructor to create a new instance of the SimpleTask class. Because the constructor is called for each test, this makes the SimpleTask available to all the tests in my SimpleTimeManager.Tests class without duplicating code.
My first test just checks that my new task has the correct default properties. There’s no need to perform an action and the only arrange needed (creating a new SimpleTask) happens in the constructor. That just leaves the asserts – I’m checking that the properties on my new SimpleTask match what I expect them to be
public class SimpleTaskShould
{
private readonly SimpleTask _task;
public SimpleTaskShould()
{
_task = new SimpleTask("Sample Task");
}
[Fact]
public void GetCreatedWithDefaults()
{
Assert.Equal(TaskState.Open, _task.State);
Assert.Equal(TaskStatus.NotStarted, _task.Status);
Assert.Equal("Sample Task", _task.Name);
Assert.Equal(new TimeSpan(), _task.Duration);
Assert.Empty(_task.AuditTracker);
}
}
Asserts
So what is happening in this test? xUnit, like other test frameworks, provides an ‘Assert’ class which provides many methods for checking the things you want to test. Here I’m using the Equal method which takes an ‘expected result’ and ‘actual result’ parameter and compares them. If they are equal, the assert passes; if they are not, the test will fail. In addition to Equal, there is NotEqual which as you might expect, checks for the opposite condition.
I also use Empty which is an Assert for collections. My SimpleTask class has an AuditTracker property which is a collection used to track whenever a change is made to the task. As this is a brand new task that we haven’t acted on, the tracker should be empty. Again, we could also use NotEmpty – in fact most of the Assert methods come in that positive/negative variant; True/False, Null/NotNull, Contains/DoesNotContain etc.
There are several other Assert methods, some you can see used in this project but for a fully exhaustive list you should look at the xUnit documentation with handy comparisons if you are coming from another test framework like I was.
Now, if we right-click inside the test class we can run the test (the Ctrl+R, T shortcut, this comes in handy when you have more tests to run). The results appear in the Test Explorer window in visual studio. If you don’t see Test Explorer you can use the Ctrl+E, T shortcut or go to Test > Windows > Test Explorer in the Visual Studio toolbar.
Traits
You can see now why we named the test class and test method; the added readability in the test output. Whether using Test Explorer like we are here, or using the dotnet command line test runner, we can now clearly see what the test is for “SimpleTaskShould.GetCreatedWithDefaults”. You’ll notice something else too, our test is part of a ‘Category’ called TaskWorkflow. How did we do that?
Another attribute in xUnit is called Trait – this is a way to organise our tests into groups, again for readability but also because it means we can run all tests with a specific trait together. The syntax is straight-forward, you use the Trait attribute and pass in the name of a trait and the value you want to apply. In my example, my trait is called ‘Category’ and I want my SimpleTask tests to be a part of the ‘Task Workflow’ category. I could apply the Trait to each test like this;
[Fact]
[Trait("Category", "TaskWorkflow")]
public void GetCreatedWithDefaults()
{
But because all my tests in the class are part of this category, instead I can set it at the class level and have it apply to each test method in the class
[Trait("Category", "TaskWorkflow")]
public class SimpleTaskShould
{
As you might have guessed, I can apply this Trait to tests in other Test Classes too – this is a great way of grouping tests together based on functionality whilst keeping your test classes themselves focused on their respective application class.
Skipping Tests
Generally, I find skipping is a bad practise – if it’s a broken test you should fix it and if it’s no longer necessary you should just remove it. If you’re looking for good test coverage, a skipped test is as good as no test at all.
That said, you might want to temporarily skip a test and xUnit has a simple way to do this. In the test attribute, you provide the skip parameter with a reason;
[Fact(Skip ="This is an example skip reason")]
Then when the tests run, you can see the reason given in the output of the test
Adding Test Output
Although I don’t skip any tests in my project, I do make use of the standard output in the test results window. It’s a good way to make the tests slightly more verbose which, for larger tests, can really assist with debugging if that test starts to fail.
xUnit provides an Abstractions namespace that needs adding to the project to make use of the ‘ITestOutputHelper’ interface. If we pass this interface into the constructor of our test class, xUnit will know to pass this in when the tests run.
using SimpleTimeManager.Tasks;
using SimpleTimeManager.Tests.TestData;
using System;
using Xunit;
using Xunit.Abstractions;
namespace SimpleTimeManager.Tests
{
[Trait("Category", "TaskWorkflow")]
public class SimpleTaskShould
{
private readonly SimpleTask _task;
private readonly ITestOutputHelper _output;
public SimpleTaskShould(ITestOutputHelper output)
{
_output = output;
_task = new SimpleTask("Sample Task");
_output.WriteLine(string.Format("Created a simple task called : \"{0}\"", _task.Name));
_output.WriteLine("Leaving constructor - starting test");
}
}
}
My tests are all fairly small but I wanted to add some output from the Test Class constructor, where we create the new SimpleTask. This is mainly a demonstration of the ITestOutputHelper interface and also as a good way to show that the constructor is called before each test. In the code you can see that the ITestOutputHelper interface defines a WriteLine property (not dissimilar to Console.WriteLine you may be familiar with from C# Console Apps).
Now if we look at the test output for any of our test methods we see the output from our constructor;
Data Driven Testing
So we’re familiar with the basics but one of the more interesting features that xUnit brings to the table is the Theory test method we talked about earlier. Where a Fact runs a single test, a Theory runs the same test multiple times passing in a different set of data each time. This is a great way to reduce code duplication as you don’t need to write the same test for each data set.
Inline Data Attributes
There’s a few ways to provide the data set for a Theory test but the easiest way is using the InlineData attribute. InlineData takes an object array as a parameter meaning that it will accept any data types you want to pass in. Then you add those parameters to the test method. I only pass in a single parameter to this test but because it accepts an array you can pass in multiple and include multiple parameters in the test method.
[Theory]
[InlineData(TaskState.Open)]
[InlineData(TaskState.Closed)]
public void ReturnTasksForRequestedState(TaskState state)
{
var requestedTasks = _sampletasklist.GetTasks(state);
int expectedTaskCount = _sampletasklist.Tasks.Where(x => x.State == state).Count();
Assert.Equal(expectedTaskCount, requestedTasks.Count());
}
Because this test has two Inline Data attributes, it will run twice – once for each state in the TaskState enum. We can see this clearly in Test Explorer when we run this test
When we dig into the test result we see that it did run twice and that a different state was passed in each time and the test passed for both scenarios.
Custom Data Attributes
InlineData is great for a small number of variations but soon becomes unwieldy if you need to provide more than just a few datasets. It also doesn’t allow you to share your datasets between tests and test classes; you must specify them against every test. Finally, it limits where the data can come from, here we’re passing a static set of data but what if we wanted to pull our test data sets from json data or a database?
To get around this there are a few ways to share data for Theory tests but the implementation I found gave the cleanest looking code was to use custom data attributes. To create a custom attribute we need to create a class that inherits from xUnit’s DataAttribute class. This is part of the xUnit.Sdk namespace so we also need a using statement for that.
The last bit we need to do, as intellisense politely informs us, is to implement the abstract member from the DataAttribute class we’re inheriting from. The GetData method is what xUnit will call when passing data into our Theory tests.
The return type of GetData is IEnumerable; xUnit will iterate over it for each test, so inside this method we use a yield return for each set of data we want to test against
public class TaskIndexTestDataAttribute : DataAttribute
{
public override IEnumerable<object[]> GetData(MethodInfo testMethod)
{
yield return new object[] { 0 };
yield return new object[] { 1 };
yield return new object[] { 2 };
yield return new object[] { 3 };
yield return new object[] { 4 };
yield return new object[] { 5 };
yield return new object[] { 6 };
yield return new object[] { 7 };
yield return new object[] { 8 };
yield return new object[] { 9 };
yield return new object[] { 10 };
yield return new object[] { 11 };
}
}
That’s it – now instead of having a lots of InlineData boilerplate in our code we can reuse a single attribute;
[Theory]
[TaskIndexTestData]
public void ReturnRequestedTask(int taskIndex)
{
var requestedTask = _sampletasklist.GetTask(taskIndex);
var taskNameNumber = taskIndex + 1;
Assert.Equal("Sample Task " + taskNameNumber, requestedTask.Name);
}
And produce a set of test results on each Theory test we apply it to
Learn More
There’s plenty more you can do in xUnit, I’m just scratching the surface here but it was a good starting point and I hope you find it useful. I was able to produce 60+ tests for this tiny application in a short amount time. If you want to see more of the areas we talked about here, take a look at those test project files; as I mentioned at the start – the application and tests are on my GitHub. For everything else, check out the documentation for xUnit.
Mindfields. 