Should I skip unit tests if integration tests cover the same scenarios?

I have a service method called acceptOrDenyJoinRequest that follows a logic similar to this flowchart (green boxes are ignored in code, and the light gray box calls an external service). According to Martin Fowler’s article on the testing pyramid, unit tests should cover all non-trivial code paths, and integration tests should be written for pieces of code that interact with external systems, like databases.

In my case, this method interacts heavily with the database and an external service, so to keep the unit test isolated, I would need to mock/stub these interactions. However, if I write integration tests covering the same scenarios without mocks/stubs, wouldn’t that make the unit tests redundant? Should I focus only on integration tests, or is there a benefit to maintaining both sets of tests even if they cover the same paths?

Is my understanding correct, or am I missing something about the purpose of unit vs. integration testing in this context?

What is an integration test?

and integration tests should be written for pieces of code that interact with external systems, like databases.

Firstly, while this is not the main point of the answer I want to get to, not all integration tests have an IO component to them. At large, integration tests are tests with multiple real components, as opposed to having only one real component under test (hence the “unit” in “unit test”).
It is true to say that all tests that include an IO component are inherently integration tests (since IO means there’s at least two parties communicating with one another), but it’s not correct to conclude that a test without IO is therefore definitely a unit test.

However, even if you disagree with that, we can put that aside for the rest of the answer. The only required distinction to acknowledge is that an “integration” relies on more than one “unit” (in the testing sense).

The name “integration” test reveals the intention of the test: do these multiple components work well together? The focus of the test here is the interaction between components, not the internal logic of the component itself, which is what a unit test is for.

If you want to skip the additional detail I’m going to go into, the above paragraph effectively means “no” to your question. Unit tests cover different ground (implementation of a specific component) than integration tests do (interaction between components). One does not replace the other.

What does a test’s failure tell you?

It is true, however, that when a component is internally faulty, i.e. its unit tests (if they were to exist) are failing, that you’d commonly expect the integration tests to then also fail. The odds of a band playing the song correctly while one of its members are out of tune are astronomically low. This is why many people have tried to shortcut having to write unit tests by claiming that an integration test would fail and thus they don’t also need unit tests.

However, and this is the crux of the issue: a failing integration test does not tell you which component failed, nor why. The only information that you collect from seeing an integration test fail is that not everything is working correctly. It’s not telling you what is actually causing the problem.

Let’s take this to an extreme, for the sake of example. Suppose I have the master integration test which runs the entire application, in every conceivable scenario, and it tells you whether that test passed or failed. In other words, either everything works as expected, or not everything works as expected.

Could you meaningfully use that test as a development aid?

The answer is no, you couldn’t. You don’t get any information as to what part of “the whole application” failed. At best, if the master integration test passed, then you’d known that you were done developing. But while the test was failing, you wouldn’t really know what part of it needed fixing.

While less extreme, the same issue occurs for integration tests. Yes, when then pass, they tend to confirm that the individual components are working as intended. But when they fail, it doesn’t really tell you what failed.

Let’s use the example of a resource creation test: you ask the system to create John Smith as a new person. You receive their generated ID. You then use that ID to fetch the person from the system. This test fails, because “no such person with this ID could be fetched”. Tell me which component failed.

You can’t, because any one of the components could’ve dropped the ball.

Maybe the creation logic did not actually persist the person in the first place.
Maybe the wrong ID was returned, so the “get” logic is looking for a person with a different ID value than was actually created.
Maybe there’s a bug in the fetch logic, e.g. the query is inherently broken, or it’s somehow not using the (correct) ID that it is being given.
Maybe the database loses its state for some weird reason (e.g. it’s load-balanced and you’re looking at a different node before the data between the nodes was synchronized)
Maybe your test forgot to account for eventual consistency, which is part of your design (for ulterior reasons) and it’s perfectly normal that the person isn’t immediately found after creation.

Based on the test failing, you have no idea what went wrong. You know that the given ID does not match the database content, but you do not know which of these two is the wrong one (if not both).

So, to conclude the above example: when an integration test passes, it suggests all of its components also pass their tests. But when an integration test fails, you don’t really get a good idea of what went wrong.

Test should be built to fail

We strike on the deeper crux of the issue: the purpose of a test isn’t to pass, it is to fail. Subsequently, when it fails (as it should), it should tell you everything about the failure that it observed.

A unit test passing is really just a unit test failing to find any kind of failure that it’s actively looking for. While that may sound tautological at first blush, it actually reveals a deeper design philosophy of unit tests: to confirm a state of failure, not to confirm a state of correctness.

A unit test passing does not confirm that the unit is entirely correct (nor does an integration test failing confirm that the entire integration is broken). A unit test exists to indicate that a specific component is broken for a specific reason, and an integration tests exists to confirm that the set of components together is achieving what you expect it to do. Neither of these statements should be negated/inverted.

Revisiting the above example, had you also had unit tests (and let’s assume the unit test suite covers the necessary bases), you could further identify the source of the issue.

If a component’s unit tests were also failing, you’d conclude that something went wrong with the internal implementation of that component. The integration test was irrelevant here, since the unit tests were already telling you that.

If none of the components’ unit tests are failing (again assuming that they cover the bases), but the integration test is still failing, this is an interesting scenario that unit tests alone could not have caught (which is precisely why you need integration tests even if you already have unit tests).
What this suggests is that you have individually correct components, but they’re not interacting with one another correctly. For example, you have a valid nut, and a valid bolt, but the bolt won’t screw onto the nut because of some kind of compatibility issue between them (e.g. because they’re different sizes, or because one is reverse-threaded and the other is not).

Conclusion

Without any tests, you don’t know anything about your application’s behavior.
With only unit tests, you can confirm that the individual components work, but you don’t know if they work well together.
With only integration tests, you can confirm that all the components work well together, but you have no way to identify which component is at fault when not everyone is working together correctly.
With both unit tests and integration tests, you have reasonable confidence that your components do what they should, and together they achieve the behavior you need them to achieve.

Filed under: softwareengineering - @ 08:22

Thẻ: integration-tests, testing, unit-testing

Thiết kế website giá rẻ

Danh mục