You are right, your tests should not verify that the random module is doing its job; a unittest should only test the class itself, not how it interacts with other code (which should be tested separately).

It is of course entirely possible that your code uses random.randint() wrong; or you be calling random.randrange(1, self._sides) instead and your die never throws the highest value, but that’d be a different kind of bug, not one you could catch with a unittest. In that case, your die unit is working as designed, but the design itself was flawed.

In this case, I’d use mocking to replace the randint() function, and only verify that it has been called correctly. Python 3.3 and up comes with the unittest.mock module to handle this type of testing, but you can install the external mock package on older versions to get the exact same functionality

import unittest
try:
    from unittest.mock import patch
except ImportError:
    # < python 3.3
    from mock import patch


@patch('random.randint', return_value=3)
class TestDice(unittest.TestCase):
    def _make_one(self, *args, **kw):
        from die import Die
        return Die(*args, **kw)

    def test_standard_size(self, mocked_randint):
        die = self._make_one()
        result = die.roll()

        mocked_randint.assert_called_with(1, 6)
        self.assertEqual(result, 3)

    def test_custom_size(self, mocked_randint):
        die = self._make_one(sides=42)
        result = die.roll()

        mocked_randint.assert_called_with(1, 42)
        self.assertEqual(result, 3)


if __name__ == '__main__':
    unittest.main()

With mocking, your test is now very simple; there are only 2 cases, really. The default case for a 6-sided die, and the custom sides case.

There are other ways to temporarily replace the randint() function in the global namespace of Die, but the mock module makes this easiest. The @mock.patch decorator here applies to all test methods in the test case; each test method is passed an extra argument, the mocked random.randint() function, so we can test against the mock to see if it it indeed has been called correctly. The return_value argument specifies what is returned from the mock when it is called, so we can verify that the die.roll() method indeed returned the ‘random’ result to us.

I’ve used another Python unittesting best practice here: import the class under test as part of the test. The _make_one method does the importing and instantiation work within a test, so that the test module will still load even if you made a syntax error or other mistake that’ll prevent the original module to import.

This way, if you made a mistake in the module code itself, the tests will still be run; they’ll just fail, telling you about the error in your code.

To be clear, the above tests are simplistic in the extreme. The goal here is not to test that random.randint() has been called with the right arguments, for example. Instead, the goal is to test that the unit is producing the right results given certain inputs, where those inputs include the results of other units not under test. By mocking the random.randint() method you get to take control over just another input to your code.

In real world tests, the actual code in your unit-under-test is going to be more complex; the relationship with inputs passed to the API and how other units are then invoked can be interesting still, and mocking will give you access to intermediate results, as well as let you set the return values for those calls.

For example, in code that authenticates users against a 3rd party OAuth2 service (a multi-stage interaction), you want to test that your code is passing the right data to that 3rd party service, and lets you mock out different error responses that that 3rd party service would return, letting you simulate different scenarios without having to build a full OAuth2 server yourself. Here it is important to test that information from a first response have been handled correctly and have been passed on to a second stage call, so you do want to see that the mocked service is being called correctly.

Martijn’s answer is how you’d do it if you really wanted to run a test that demonstrates that you’re calling random.randint. However, at the risk of being told “that doesn’t answer the question”, I feel this shouldn’t be unit tested at all. Mocking randint is no longer black box testing – you’re specifically showing that certain things are going on in the implementation. Black box testing it isn’t even an option – there is no test you can execute that will prove that the result will never be less than 1 or more than 6.

Can you mock randint? Yes, you can. But what are you proving? That you called it with arguments 1 and sides. What does that mean? You’re back in square one – at the end of the day you’ll end up having to prove – formally or informally – that calling random.randint(1, sides) correctly implements a dice roll.

I’m all for unit testing. They’re fantastic sanity checks and expose the presence of bugs. However, they can never prove their absence, and there’s things that can’t be asserted through testing at all (e.g. that a particular function never throws an exception or always terminates.) In this particular case, I feel there’s very little you stand to gain. For behavior that’s deterministic, unit tests make sense because you actually know what the answer you’re expecting will be.

Fix random seed. For 1, 2, 5 and 12-sided dice, confirm that a few thousand rolls gives results including 1 and N, and not including 0 or N + 1. If by seem freak chance you get a set of random results that don’t cover the expected range, switch to a different seed.

Mocking tools are cool, but just because they allow you to do a thing doesn’t mean that thing should be done. YAGNI applies to test fixtures as much as features.

If you can easily test with unmocked dependencies, you pretty much always should; that way your tests will be focused on reducing defect counts, not just increasing test count. Excess mocking risks creating misleading coverage figures, which in turn can lead to postponing the actual testing to some later phase you perhaps never have time to get round to…

What is a Die if you think about it ? – no more than a wrapper around random. It encapsulates random.randint and relabels it in terms of your application’s own vocabulary : Die.Roll.

I don’t find it relevant to insert another layer of abstraction between Die and random because Die itself is already this layer of indirection between your application and the platform.

If you want canned dice results, just mock Die, don’t mock random.

In general, I don’t unit test my wrapper objects that communicate with external systems, I write integration tests for them. You could write a couple of those for Die but as you pointed out, due to the random nature of the underlying object, they will not be meaningful. In addition, there’s no configuration or network communication involved here so not much to test except a platform call.

=> Considering that Die is only a few trivial lines of code and adds little to no logic compared to random itself, I’d skip testing it in that specific example.

Seeding the random number generator and verifying expected results is NOT, as far as I can see, a valid test. It makes assumptions as to HOW your dice works internally, which is naughty-naughty. The developers of python could change the random number generator, or the die (NOTE: “dice” is plural, “die” is singular. Unless your class implements multiple die rolls in one call, it should probably be called “die”) could use a different random number generator.

Similarly, mocking the random function assumes that the class implementation works exactly as expected. Why might this not be the case? Someone might take control of the default python random number generator, and to avoid that, a future version of your die may fetch several random numbers, or larger random numbers, to mix in more random data. A similar scheme was used by the makers of the FreeBSD operating system, when they suspected the NSA was tampering with the hardware random number generators built into CPUs.

If it were me, I’d run, say, 6000 rolls, tally them, and make sure that each number from 1-6 is rolled between 500 and 1500 times. I would also check that no numbers outside that range are returned. I might also check that, for a second set of 6000 rolls, when ordering the [1..6] in order of frequency, the result is different (this will fail once out of 720 runs, if the numbers are random!). If you want to be thorough, you might find the frequency of numbers following a 1, following a 2, etc; but make sure your sample size is big enough, and you have enough variance. Humans expect random numbers to have fewer patterns than they actually do.

Repeat for a 12 sided, and 2 sided die (6 is the most used, so is the most expected for anyone writing this code).

Finally, I would test to see what happens with a 1-sided die, a 0 sided die, a -1 sided die, a 2.3 sided die, a [1,2,3,4,5,6] sided die, and a “blah”-sided die. Of course, these should all fail; do they fail in a useful way? These should probably fail on creation, not on rolling.

Or, perhaps, you want too handle these differently – perhaps creating a die with [1,2,3,4,5,6] should be acceptable – and perhaps “blah” as well; this might be a die with 4 faces, and each face having a letter on it. The game “Boggle” springs to mind, as does a magic eight ball.

And finally, you might want to contemplate this: http://lh6.ggpht.com/-fAGXwbJbYRM/UJA_31ACOLI/AAAAAAAAAPg/2FxOWzo96KE/s1600-h/random%25255B3%25255D.jpg

The tests you suggest in your question don’t detect a modular arithmetic counter as implementation. And they don’t detect common implementation errors in probability distribution related code like return 1 + (random.randint(1,maxint) % sides). Or a change to the generator that results in 2-dimensional patterns.

If you actually want to verify that you are generating evenly distributed random-appearing numbers you need to check a very wide variety of properties. To do a reasonably good job at that you could run http://www.phy.duke.edu/~rgb/General/dieharder.php on your generated numbers. Or write a similarly complex unit-test suite.

That’s not the fault of unit-testing or TDD, randomness just happens to be a very difficult property to verify. And a popular topic for examples.

At the risk of swimming against the tide, I solved this exact problem a number of years ago using a method not so far mentioned.

My strategy was simply to mock the RNG with one that produces a predictable stream of values spanning the entire space. If (say) side=6 and the RNG produces values from 0 to 5 in sequence I can predict how my class should behave and unit test accordingly.

The rationale is that this tests the logic in this class alone, on the assumption that the RNG will eventually produce each of those values and without testing the RNG itself.

It’s simple, deterministic, reproducible and it does catch bugs. I would use the same strategy again.

The question does not spell out what the tests should be, just what data might be used for testing, given the presence of an RNG. My suggestion is merely to test exhaustively by mocking the RNG. The question of what is worth testing depends on information not provided in the question.

The easiest test of a die-roll is simply to repeat it several hundred-thousand times, and validate that each possible result was hit roughly (1 / number of sides) times. In the case of a 6-sided die, you should see each possible value hit about 16.6% of the time. If any are off by more than a percent, then you have a problem.

Doing it this way avoids allows you to refactor the underlying mechanic of generating a random number easily, and most importantly, without changing the test.