Time and Tests (and xUnit)
Time flies like an arrow; fruit flies like a banana.
– Anthony G. Oettinger
Testing time-related logic is not a new problem, but solutions vary from language to language, and even from framework to framework.
There are two main categories of time-related logic:
- Getting the current time
- Waiting for a certain amount of time
This post covers how to test both in .NET Core, using xUnit and good ol’ OOP.
Current Time
Using DateTimeOffset.Now
is an example of getting the current time.
Examples of logic that require getting the current time are the following:
- Determining if a authentication token is expired.
- Calculating the age of someone based on birthdate.
- Displaying either “good morning” or “good afternoon” in a dashboard page.
Using DateTimeOffset
ties you to the system time.
For example, if you had a wanted to test the greetings, both tests could never pass in the same testing session; it can’t be morning and afternoon at the same time on the same place in the real world.
Fortunately, the solution is easy enough:
public interface ITime
{
DateTimeOffset Now { get; }
}
public class SystemTime : ITime
{
public DateTimeOffset Now => DateTimeOffset.Now;
}
We implement an interface on which time-sensitive logic will depend on, and inject the
SystemTime
class as its implementation during runtime.
If we want to test what happens at a specific time, we could use the following:
public class FrozenTime : ITime
{
public DateTimeOffset Now { get; }
public FrozenTime(DateTimeOffset now)
{
Now = now;
}
}
Waiting for X Time
Using Task.Delay
is an example of waiting for X time, or delay.
Examples of logic that require a delay are the following:
- Waiting for some time before cancelling an operation (timeout).
- Waiting before making further HTTP requests (rate limit).
- Performing an action every X time (periodic execution).
Using Task.Delay
also ties you to the system time, but it’s more insidious
as it gives you the easy way out: just add shorter delays for testing.
Besides that, depending on your tests, they may not even be deterministic: their results may vary depending on how fast (or slow) the CI server is, on race-conditions, or just plain bad luck.
Again, the solution is fairly simple:
public interface IDelayer
{
Task Delay(TimeSpan delay, CancellationToken cancellationToken);
}
public class SystemDelayer : IDelayer
{
public Task Delay(TimeSpan delay, CancellationToken cancellationToken)
{
return Task.Delay(delay, cancellationToken);
}
}
However, this time there are some differences on how we’ll write our test classes, depending on whether we’re going to test a single delay, or multiple delays.
Single delay
If you want a delay that completes almost immediately, then use the following:
public class NoDelayDelayer : IDelayer
{
public Task Delay(TimeSpan delay, CancellationToken cancellationToken)
{
return Task.Delay(1, cancellationToken);
}
}
If you want a delay that’s completed whenever you want, consider the following:
public class CancellableDelayer : IDelayer
{
private CancellationTokenSource CancellationSource { get; }
public CancellableDelayer()
{
CancellationSource = new CancellationTokenSource();
}
public async Task Delay(TimeSpan delay, CancellationToken cancellationToken)
{
await Task.Delay(-1, CancellationSource.Token);
}
public void Cancel()
{
CancellationSource.Cancel();
}
}
You can emulate some interesting corner-cases when you have complete control of the sequence of events that happen before and after a delay is completed.
Multiple delays
This is where things get interesting.
Just a few days ago I had to test the following method:
public async Task ExecutePeriodicChecks(TimeSpan frequency, CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
// Perform the checks
var checks = await _checkRepository.GetChecksByFrequency(frequency);
var checkTasks = Task.WhenAll(checks.Select(PerformCheck));
// Wait a bit before trying again.
var waitingTask = _delayer.Delay(frequency, stoppingToken);
await Task.WhenAll(checkTasks, waitingTask);
}
}
It’s very straightforward code: it runs a process no faster than the specified frequency.
Testing that assumption is a bit harder though. Let’s say we wanted to test the following scenario: given a frequency of 1 minute, after 2 minutes and 10 seconds, 3 check cycles must have been performed. Where do we even start?
I’ll come clean now. At first I just decided to depend on system time, as follows:
[Fact]
public async Task ExecutePeriodicChecks_CheckFrequencyAsConfigured_ChecksEndBeforeTimeElapses()
{
// Arrange
var frequency = TimeSpan.FromMilliseconds(50);
var handlers = ImmutableDictionary<Type, ICheckHandler>.Empty;
var delayer = new TaskDelayDelayer();
var repo = new Mock<ICheckRepository>();
repo
.Setup(x => x.GetChecksByFrequency(frequency))
.Returns(Task.FromResult((IEnumerable<Check>)new List<Check>()));
var doc = new Doctor(repo.Object, handlers, delayer);
var cts = new CancellationTokenSource(140);
// Act
try
{
await doc.ExecutePeriodicChecks(frequency, cts.Token);
}
catch (OperationCanceledException)
{
// Do nothing; we're expecting the OCE.
}
// Assert
repo.Verify(x => x.GetChecksByFrequency(frequency), Times.Exactly(3));
}
As expected, sometimes the tests fail at the CI server.
Fundamentally, we need to abstract ourselves from the passing of time before we can test an scenario like this one. After doing so, we’ll notice that our previous statement:
Given a frequency of 1 minute, after 2 minutes and 10 seconds, 3 check cycles must have been performed.
Could also be expressed as:
After two delays have elapsed, 3 check cycles must have been performed.
Time is just an implementation detail.
What we care about here are elapsed delays, and so we implemented
a class that would elapse immediately the first N times Delay
was called and be of infinite
duration for any further calls.
public class TimesDelayer : IDelayer
{
private readonly object _mutex = new object();
private int TimesRemaining { get; set; }
public Action CallbackWhenZero;
public TimesDelayer(int times)
{
if (times < 1)
throw new ArgumentOutOfRangeException(nameof(times),
"Times can't be less than 1.");
TimesRemaining = times;
}
public Task Delay(TimeSpan delay, CancellationToken cancellationToken)
{
lock (_mutex)
{
if (TimesRemaining > 0)
{
TimesRemaining -= 1;
if (TimesRemaining == 0)
{
CallbackWhenZero?.Invoke();
}
return Task.CompletedTask;
}
}
return Task.Delay(-1, cancellationToken);
}
}
The callback is there so the calling test can be notified when Delay
was
called the configured number of times.
Notable differences from the previous test are the following:
[Fact]
public async Task ExecutePeriodicChecks_CheckFrequencyAsConfigured_ChecksEndBeforeTimeElapses()
{
// We're simulating elapsed time with the TimesDelayer due to non-deterministic
// results when using Task.Delay on slower computers / CI environment.
// Arrange
// ...
var delayer = new TimesDelayer(3);
// ...
var doc = new Doctor(repo.Object, handlers, delayer);
var cts = new CancellationTokenSource();
// Act
try
{
// Cancel ExecutePeriodicChecks only after delayer has been
// called two times.
delayer.CallbackWhenZero = () => cts.Cancel();
var checkTask = doc.ExecutePeriodicChecks(frequency, cts.Token);
await checkTask;
}
catch (OperationCanceledException)
{
// Do nothing; we're expecting the OCE.
}
// Assert
repo.Verify(x => x.GetChecksByFrequency(frequency), Times.Exactly(3));
}
We’re cancelling ExecutePeriodicChecks
after the Delay
method on
TimesDelayer
was called for its third time. By doing so we can test
that cancelling the operation while it waits works as we expect.
…but I just wanted to call Cancel more than once
public class CancellableDelayer : IDelayer
{
private readonly object _mutex = new object();
private CancellationTokenSource CancellationSource { get; set; }
public CancellableDelayer()
{
CancellationSource = new CancellationTokenSource();
}
public async Task Delay(TimeSpan delay, CancellationToken cancellationToken)
{
CancellationToken token;
lock (_mutex)
{
token = CancellationSource.Token;
}
await Task.Delay(-1, token);
}
public void Cancel()
{
lock (_mutex)
{
CancellationSource.Cancel();
CancellationSource = new CancellationTokenSource();
}
}
}
Summary
Unit testing is about testing self-contained units. System time, just like databases and web services, is an external dependency that should be abstracted for testing.
In this post we saw how to abstract the retrieval of the current time, and how to
perform delays in .NET Core by using the ITime
and IDelayer
interfaces respectively.
Finally, some example ready-to-use implementations have been provided.