Real time unit testing – or “how to mock now”

When you are working on a feature that depends on time… How do you organize unit testing ? When your unit tests scenarios depend on the way your program interprets “now”, how do you set them up ?

Second Edit: After a Couple of days reading your experience

I can see that the techniques to deal with this situation usually revolve around one of these three principles:

Add (hardcode) a dependency: add a small layer over the time function/object, and always call your datetime function through this layer. This way, you can get control of time during the test cases.
Use mocks: your code stays exactly the same. In your tests, you replace the time object by a fake time object. Sometimes, the solution involves modifying the genuine time object that is provided by your programming language.
Use dependency injection: Build your code so that any time reference is passed as a parameter. Then, you have control of the parameters during the tests.

Techniques (or libraries) specific to a language are very welcome, and will be enhanced if an illustrative piece of code comes along. Then, most interesting is how similar principles may be applied on any platform.
And yes… If I can apply it straight away in PHP, better than better 😉

Let’s start with a simple example: a basic booking application.

Let’s say we have JSON API and two messages: a request message, and a confirmation message.
A standard scenario goes like this:

You make a request. You get a response with a token. The resource that is needed to fulfill that request is blocked by the system for 5 minutes.
You confirm a request, identified by the token. If token was issued within 5 minutes, it will be accepted (the resource is still available). If more than 5 minutes have passed, you have to make a new request (the resource was freed. You need to check for its availability again).

Here come the corresponding test scenario:

I make a request. I confirm (immediately) with the token I received. My confirmation is accepted.
I make a request. I wait 3 minutes. I confirm with the token I received. My confirmation is accepted.
I make a request. I wait 6 minutes. I confirm with the token I received. My confirmation is rejected.

How can we get to program these unit tests ? What architecture should we use so that these functionalities remain testable ?

Edit Note: When we shoot a request, the time is stored in a database in a format that looses any information about milliseconds.

EXTRA – but maybe a bit verbose: Here come details about what I found out doing my “homework”:

I Built my feature with a dependency on a time function of my own. My function VirtualDateTime has a static method get_time() that I call where I used to call new DateTime(). This time function allows me to simulate and control what time is “now”, so that I can build tests like: “Set now to 21st jan 2014 16h15. Make a request. Move ahead 3 minutes. Make confirmation.”. This works fine, at the cost of the dependency, and “not so pretty code”.
A little more integrated solution would be to build a myDateTime function of my own that extends DateTime with additional “virtual time” functionnalities (most importantly, set now to what I want). This is making the first solution’s code slightly more elegant (use of new myDateTime instead of new DateTime), but ends up to be very similar: I have to build my feature using my own class, thus creating a dependency.
I have though about hacking the DateTime function, to make it work with my dependency when I need it. Is there any simple and neat way to replace a class by another, though ? (I think I got an answer to that: See namespace below).
In a PHP environment, I read “Runkit” may allow me to do this (hack the DateTime function) dynamically. What is nice is that I would be able to check I am running in test environment before modifying anything about DateTime, and leave it untouched in production [1]. This sounds much safer and cleaner than any manual DateTime hack.
Dependency injection of a clock function in each class that uses time [2]. Is this not overkill ? I see that in some environments it gets very usefull [5]. In this case, I do not like it so much, though.
Remove dependency of time in all the functions [2]. Is this always feasible ? (See more examples below)
Using namespaces [2][3] ? This looks quite good. I could mock DateTime with my own DateTime function that extends DateTime… Use DateTime::setNow(“2014-01-21 16:15:00”), and DateTime::wait(“+3 minutes”). This covers pretty much what I need. What if we use the time() function though ? Or other time functions ? I still have to avoid their use in my original code. Or I would need to make sure any PHP time function I use in my code is overridden in my tests… Is there any library available that would do just this ?
I have been looking for a way to change the system time just for a thread. It seems that there is none [4]. This is a pity: a “simple” PHP function to “Set time to 21st jan 2014 16h15 for this thread” would be a great feature for this kind of tests.
Change the system date and time for the test, using exec(). This can work out if you are not afraid to mess up with other things on the server. And you need to set back the system time after you ran your test. It can do the trick in some situations, but feels quite “hacky”.

This looks like a very standard problem to me. However, I still miss a simple an generic way to deal with it. Maybe I missed something ?
Please share your experience !

NOTE: Here are some other situations where we may have similar testing needs.

Any feature that works with some kind of timeout (e.g. chess game ?)
processing a queue that triggers events at a given time (in the above scenario we could keep going like this: one day before the reservation starts, I want to send a mail to the user with all the details. Test scenario…)
You want to set up a test environment with data collected in the past – you would like to see it like if it was now. [1]

1 https://stackoverflow.com/questions/3271735/simulate-different-server-datetimes-in-php

2 https://stackoverflow.com/questions/4221480/how-to-change-current-time-for-unit-testing-date-functions-in-php

3 http://www.schmengler-se.de/en/2011/03/php-mocking-built-in-functions-like-time-in-unit-tests/

4 https://stackoverflow.com/questions/3923848/change-todays-date-and-time-in-php

5 Unit testing time-bound code

I’m going to use Python-like pseudocode based around your test cases to (hopefully) help explain this answer a little, rather than just expositing. But in short: This answer is basically just an example of small, single-function-level dependency injection / dependency inversion, instead of applying the principle to an entire class/object.

Right now, it sounds like your tests are structured as such:

def test_3_minute_confirm():
    req = make_request()
    sleep(3 * 60)  # Baaaad
    assertTrue(confirm_request(req))

With implementations something along the lines of:

def make_request():
    return { 'request_time': datetime.now(), }

def confirm_request(req):
    return req['request_time'] >= (datetime.now() - timedelta(minutes=5))

Instead, try forgetting about “now” and tell both functions what time to use. If it’s most of the time going to be “now”, you can do one of two main things to keep the calling code simple:

Make time an optional argument that defaults to “now”. In Python (and PHP, now that I reread the question) this is simple – default “time” to None (null in PHP) and set it to “now” if no value was given.
In other languages without default arguments (or in PHP if it becomes unwieldy), it might be easier to pull the guts of the functions out into a helper that does get tested.

For example, the two functions above rewritten in the second version might look like this:

def make_request():
    return make_request_at(datetime.now())

def make_request_at(when):
    return { 'request_time': when, }

def confirm_request(req):
    return confirm_request_at(req, datetime.now())

def confirm_request_at(req, check_time):
    return req['request_time'] >= (check_time - timedelta(minutes=5))

This way, existing parts of the code don’t need to be modified to pass in “now”, while the parts you actually want to test can be tested much more easily:

def test_3_minute_confirm():
    current_time = datetime.now()
    later_time = current_time + timedelta(minutes=3)

    req = make_request_at(current_time)
    assertTrue(confirm_request_at(req, later_time))

It also creates additional flexibility in the future, so less needs to change if the important parts need to be reused. For two examples that pop into mind: A queue where requests might need to be created moments too late but still use the correct time, or confirmations have to happen to past requests as part of data sanity checks.

Technically, thinking ahead like this might violate YAGNI, but we’re not actually building for those theoretical cases – this change would just be for easier testing, that just happens to support those theoretical cases.

Is there any reason you would recommend one of these solutions over another?

Personally, I try to avoid any sort of mocking as much as possible, and prefer pure functions as above. This way, the code being tested is identical to the code being run outside of tests, instead of having important parts replaced with mocks.

We’ve had one or two regressions that no one noticed for a good month because that particular part of the code was not often tested in development (we weren’t actively working on it), was released slightly broken (so there were no bug reports), and the mocks used were hiding a bug in the interaction between helper functions. A few slight modifications so mocks weren’t necessary and a whole lot more could be easily tested, including fixing the tests around that regression.

Change the system date and time for the test, using exec(). This can work out if you are not afraid to mess up with other things on the server.

This is the one that should be avoided at all costs for unit testing. There’s really no way of knowing what kinds of inconsistencies or race conditions might be introduced to cause intermittent failures (for unrelated applications, or co-workers on the same development box), and a failed/errored test might not set the clock back correctly.

For my money, holding a dependency on some clock class is the clearest way to go about handling time-dependent testing. In PHP, it may be as simple as a a class with a single function now which returns the current time using time() or new DateTime().

Injecting this dependency allows you to manipulate the time in your tests and use the real time in production, without having to write too much extra code.

When I first learned TDD, I was working in a c# environment. The way I learned to do it is to have a complete IoC environment, which also included an ITimeService, which was responsible for all time services, and could easily be mocked in tests.

Nowadays I’m working in ruby, and stubbing time is a lot easier – you simply stub time:

describe 'some time based module' do

  before(:each) do
    @now = Time.now
    allow(Time).to receive(:now) { @now }
  end

  it 'times out after five minutes' do
    subject.do_something

    @now += 5.minutes

    expect { subject.do_another_thing }.to raise TimeoutError
  end
end

The first thing you need to do is remove the magic numbers from your code. In this case, your “5 minute” magic number. Not only will you inevitably need to change these later when some client demands it, it makes unit testing way better. Who’s going to wait 5 minutes for their test to run?

Second, if you haven’t already, use UTC for actual timestamps. This will prevent issues with daylight savings and most other clock hiccups.

In the unit test, switch the configured duration to something like 500ms and do your normal test: does it work one way before the timeout and another way after.

~1s is still fairly slow for unit tests, and you likely will need some sort of threshold to account for machine performance and clock shifts due to NTP (or other drift). But in my experience, this is the best practical way to unit test time based stuff in most systems. Simple, quick, reliable. Most other approaches (as you’ve found) require tons of work and/or are terribly error prone.

Use Microsoft Fakes – instead of altering your code to suit the tool, the tool alters your code at runtime, injecting a new Time object (that you define in your test) in place of the standard clock.

If you’re running a dynamic language – then Runkit is just the same type of thing.

For example with Fakes:

[TestMethod]
public void MyDateProvider_ShouldReturnDateAsJanFirst1970()
{
    using (Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create())
    {
        // Arrange: set up a shim to our own date object instead of System.DateTime.Now
        System.Fakes.ShimDateTime.NowGet = () => new DateTime(1970, 1, 1);

        //Act: call the method under test
        var curentDate = MyDateTimeProvider.GetTheCurrentDate();

        //Assert: that we have a moustache and a brown suit.
        Assert.IsTrue(curentDate == new DateTime(1970, 1, 1));
    }
}

public class MyDateTimeProvider
{
    public static DateTime GetTheCurrentDate()
    {
        return DateTime.Now;
    }
}

So when the text method calls return DateTime.Now it will return the time you injected, rather than the real current time.

I explored the namespaces option in PHP. Namespaces give us the opportunity to add some testing behavior to the DateTime function. Thus, our original objects stay untouched.

First, we set up a fake DateTime object in our namespace so that we may, in our tests, manage what time is considered to be “now”.

Then, in our tests, we can manage this time via the static functions DateTime::set_now($my_time) and DateTime::modify_now(“add some time”).

First comes a fake Booking Object. This object is what we want to test – observe the use of new DateTime() without parameters in two places. A short overview:

<?php

namespace BookingNamespace;
/**
* Simulate expiration process with two public methods:
*      - book will return a token
*      - confirm will take a token, and return
*                       -> true if called within 5 minutes of corresponding book call
*                       -> false if more than 5 minutes have passed
*/
class MyBookingObject
{
    ...

    private function init_token($token){ $this->tokens[$token] = new DateTime(); }

    ...

    private function has_timed_out(DateTime $booking_time)
    {
            $now = new DateTime();
            ....
    }
}

Our DateTime object overriding the core DateTime object looks like this. All function calls remain unchanged. We only “hook” the constructor to simulate our “now is whenever I choose” behavior. Most importantly,

namespace BookingNamespace;

/**
 * extend DateTime functionalities with static functions so that we are able 
 * to define what should be considered to be "now"
 *   - set_now allows to define "now"
 *   - modify_now applies the modify function on what is defined as "now"
 *   - reset is for going back to original DateTime behaviour
 *  
 * @author mika
 */
 class DateTime extends DateTime
 {
     private static $fake_time=null;

     ...

     public function __construct($time = "now", DateTimeZone $timezone = null)
     {
          if($this->is_virtual()&&$this->is_relative_date($time))
          {
           ....
          }
          else
            parent::__construct($time, $timezone);
     }
 }

Our tests are then very straightforward. It goes like this with php unit (full version here):

....
require_once "DateTime.class.php"; // Override DateTime in current namespace
....

class MyBookingObjectTest extends PHPUnit_Framework_TestCase
{
    ....

    /**
     * Create test subject before test
     */
    protected function setUp()
    {
        ....
        DateTime::set_now("2014-04-02 16:15");
        ....
    }

   ...

    public function testBookAndConfirmThreeMinutesLater()
    {
        $token = $this->booking_object->book();
        DateTime::modify_now("+3 minutes");
        $this->assertTrue($this->booking_object->confirm($token));
    }

    ...
}

This structure can be easily reproduced wherever I need to manage “now” manually: I only need to include the new DateTime object, and use its static methods in my tests.

Filed under: softwareengineering - @ 07:36

Thẻ: c++, real-time, ruby, unit-testing

Real time unit testing – or “how to mock now”

Second Edit: After a Couple of days reading your experience

I can see that the techniques to deal with this situation usually revolve around one of these three principles:

Add (hardcode) a dependency: add a small layer over the time function/object, and always call your datetime function through this layer. This way, you can get control of time during the test cases.
Use mocks: your code stays exactly the same. In your tests, you replace the time object by a fake time object. Sometimes, the solution involves modifying the genuine time object that is provided by your programming language.
Use dependency injection: Build your code so that any time reference is passed as a parameter. Then, you have control of the parameters during the tests.

Let’s start with a simple example: a basic booking application.

Let’s say we have JSON API and two messages: a request message, and a confirmation message.
A standard scenario goes like this:

You make a request. You get a response with a token. The resource that is needed to fulfill that request is blocked by the system for 5 minutes.
You confirm a request, identified by the token. If token was issued within 5 minutes, it will be accepted (the resource is still available). If more than 5 minutes have passed, you have to make a new request (the resource was freed. You need to check for its availability again).

Here come the corresponding test scenario:

I make a request. I confirm (immediately) with the token I received. My confirmation is accepted.
I make a request. I wait 3 minutes. I confirm with the token I received. My confirmation is accepted.
I make a request. I wait 6 minutes. I confirm with the token I received. My confirmation is rejected.

How can we get to program these unit tests ? What architecture should we use so that these functionalities remain testable ?

Edit Note: When we shoot a request, the time is stored in a database in a format that looses any information about milliseconds.

EXTRA – but maybe a bit verbose: Here come details about what I found out doing my “homework”:

I Built my feature with a dependency on a time function of my own. My function VirtualDateTime has a static method get_time() that I call where I used to call new DateTime(). This time function allows me to simulate and control what time is “now”, so that I can build tests like: “Set now to 21st jan 2014 16h15. Make a request. Move ahead 3 minutes. Make confirmation.”. This works fine, at the cost of the dependency, and “not so pretty code”.
A little more integrated solution would be to build a myDateTime function of my own that extends DateTime with additional “virtual time” functionnalities (most importantly, set now to what I want). This is making the first solution’s code slightly more elegant (use of new myDateTime instead of new DateTime), but ends up to be very similar: I have to build my feature using my own class, thus creating a dependency.
I have though about hacking the DateTime function, to make it work with my dependency when I need it. Is there any simple and neat way to replace a class by another, though ? (I think I got an answer to that: See namespace below).
In a PHP environment, I read “Runkit” may allow me to do this (hack the DateTime function) dynamically. What is nice is that I would be able to check I am running in test environment before modifying anything about DateTime, and leave it untouched in production [1]. This sounds much safer and cleaner than any manual DateTime hack.
Dependency injection of a clock function in each class that uses time [2]. Is this not overkill ? I see that in some environments it gets very usefull [5]. In this case, I do not like it so much, though.
Remove dependency of time in all the functions [2]. Is this always feasible ? (See more examples below)
Using namespaces [2][3] ? This looks quite good. I could mock DateTime with my own DateTime function that extends DateTime… Use DateTime::setNow(“2014-01-21 16:15:00”), and DateTime::wait(“+3 minutes”). This covers pretty much what I need. What if we use the time() function though ? Or other time functions ? I still have to avoid their use in my original code. Or I would need to make sure any PHP time function I use in my code is overridden in my tests… Is there any library available that would do just this ?
I have been looking for a way to change the system time just for a thread. It seems that there is none [4]. This is a pity: a “simple” PHP function to “Set time to 21st jan 2014 16h15 for this thread” would be a great feature for this kind of tests.
Change the system date and time for the test, using exec(). This can work out if you are not afraid to mess up with other things on the server. And you need to set back the system time after you ran your test. It can do the trick in some situations, but feels quite “hacky”.

This looks like a very standard problem to me. However, I still miss a simple an generic way to deal with it. Maybe I missed something ?
Please share your experience !

NOTE: Here are some other situations where we may have similar testing needs.

Any feature that works with some kind of timeout (e.g. chess game ?)
processing a queue that triggers events at a given time (in the above scenario we could keep going like this: one day before the reservation starts, I want to send a mail to the user with all the details. Test scenario…)
You want to set up a test environment with data collected in the past – you would like to see it like if it was now. [1]

1 https://stackoverflow.com/questions/3271735/simulate-different-server-datetimes-in-php

2 https://stackoverflow.com/questions/4221480/how-to-change-current-time-for-unit-testing-date-functions-in-php

3 http://www.schmengler-se.de/en/2011/03/php-mocking-built-in-functions-like-time-in-unit-tests/

4 https://stackoverflow.com/questions/3923848/change-todays-date-and-time-in-php

5 Unit testing time-bound code

Right now, it sounds like your tests are structured as such:

def test_3_minute_confirm():
    req = make_request()
    sleep(3 * 60)  # Baaaad
    assertTrue(confirm_request(req))

With implementations something along the lines of:

def make_request():
    return { 'request_time': datetime.now(), }

def confirm_request(req):
    return req['request_time'] >= (datetime.now() - timedelta(minutes=5))

Instead, try forgetting about “now” and tell both functions what time to use. If it’s most of the time going to be “now”, you can do one of two main things to keep the calling code simple:

Make time an optional argument that defaults to “now”. In Python (and PHP, now that I reread the question) this is simple – default “time” to None (null in PHP) and set it to “now” if no value was given.
In other languages without default arguments (or in PHP if it becomes unwieldy), it might be easier to pull the guts of the functions out into a helper that does get tested.

For example, the two functions above rewritten in the second version might look like this:

def make_request():
    return make_request_at(datetime.now())

def make_request_at(when):
    return { 'request_time': when, }

def confirm_request(req):
    return confirm_request_at(req, datetime.now())

def confirm_request_at(req, check_time):
    return req['request_time'] >= (check_time - timedelta(minutes=5))

This way, existing parts of the code don’t need to be modified to pass in “now”, while the parts you actually want to test can be tested much more easily:

def test_3_minute_confirm():
    current_time = datetime.now()
    later_time = current_time + timedelta(minutes=3)

    req = make_request_at(current_time)
    assertTrue(confirm_request_at(req, later_time))

Is there any reason you would recommend one of these solutions over another?

Change the system date and time for the test, using exec(). This can work out if you are not afraid to mess up with other things on the server.

Injecting this dependency allows you to manipulate the time in your tests and use the real time in production, without having to write too much extra code.

Nowadays I’m working in ruby, and stubbing time is a lot easier – you simply stub time:

describe 'some time based module' do

  before(:each) do
    @now = Time.now
    allow(Time).to receive(:now) { @now }
  end

  it 'times out after five minutes' do
    subject.do_something

    @now += 5.minutes

    expect { subject.do_another_thing }.to raise TimeoutError
  end
end

Second, if you haven’t already, use UTC for actual timestamps. This will prevent issues with daylight savings and most other clock hiccups.

In the unit test, switch the configured duration to something like 500ms and do your normal test: does it work one way before the timeout and another way after.

If you’re running a dynamic language – then Runkit is just the same type of thing.

For example with Fakes:

[TestMethod]
public void MyDateProvider_ShouldReturnDateAsJanFirst1970()
{
    using (Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create())
    {
        // Arrange: set up a shim to our own date object instead of System.DateTime.Now
        System.Fakes.ShimDateTime.NowGet = () => new DateTime(1970, 1, 1);

        //Act: call the method under test
        var curentDate = MyDateTimeProvider.GetTheCurrentDate();

        //Assert: that we have a moustache and a brown suit.
        Assert.IsTrue(curentDate == new DateTime(1970, 1, 1));
    }
}

public class MyDateTimeProvider
{
    public static DateTime GetTheCurrentDate()
    {
        return DateTime.Now;
    }
}

So when the text method calls return DateTime.Now it will return the time you injected, rather than the real current time.

I explored the namespaces option in PHP. Namespaces give us the opportunity to add some testing behavior to the DateTime function. Thus, our original objects stay untouched.

First, we set up a fake DateTime object in our namespace so that we may, in our tests, manage what time is considered to be “now”.

Then, in our tests, we can manage this time via the static functions DateTime::set_now($my_time) and DateTime::modify_now(“add some time”).

First comes a fake Booking Object. This object is what we want to test – observe the use of new DateTime() without parameters in two places. A short overview:

<?php

namespace BookingNamespace;
/**
* Simulate expiration process with two public methods:
*      - book will return a token
*      - confirm will take a token, and return
*                       -> true if called within 5 minutes of corresponding book call
*                       -> false if more than 5 minutes have passed
*/
class MyBookingObject
{
    ...

    private function init_token($token){ $this->tokens[$token] = new DateTime(); }

    ...

    private function has_timed_out(DateTime $booking_time)
    {
            $now = new DateTime();
            ....
    }
}

namespace BookingNamespace;

/**
 * extend DateTime functionalities with static functions so that we are able 
 * to define what should be considered to be "now"
 *   - set_now allows to define "now"
 *   - modify_now applies the modify function on what is defined as "now"
 *   - reset is for going back to original DateTime behaviour
 *  
 * @author mika
 */
 class DateTime extends DateTime
 {
     private static $fake_time=null;

     ...

     public function __construct($time = "now", DateTimeZone $timezone = null)
     {
          if($this->is_virtual()&&$this->is_relative_date($time))
          {
           ....
          }
          else
            parent::__construct($time, $timezone);
     }
 }

Our tests are then very straightforward. It goes like this with php unit (full version here):

....
require_once "DateTime.class.php"; // Override DateTime in current namespace
....

class MyBookingObjectTest extends PHPUnit_Framework_TestCase
{
    ....

    /**
     * Create test subject before test
     */
    protected function setUp()
    {
        ....
        DateTime::set_now("2014-04-02 16:15");
        ....
    }

   ...

    public function testBookAndConfirmThreeMinutesLater()
    {
        $token = $this->booking_object->book();
        DateTime::modify_now("+3 minutes");
        $this->assertTrue($this->booking_object->confirm($token));
    }

    ...
}

This structure can be easily reproduced wherever I need to manage “now” manually: I only need to include the new DateTime object, and use its static methods in my tests.

Filed under: softwareengineering - @ 07:36

Thẻ: c++, real-time, ruby, unit-testing

Thiết kế website giá rẻ

Danh mục

Real time unit testing – or “how to mock now”

Real time unit testing – or “how to mock now”