I want to know how rigid do we have to stick to the notion of end to end testing, please consider the following explanation:
Basically, when we talk about end to end test we send a request and expect a response, in an end to end test we verify this request and response with a reference payload that we have.
But, consider this scenerio: An api affects two tables, I’d like to verify that if a request fails, the number of rows that are in the second table remain constant. The request fails, and nothing but the status code and error message are returned. Currently there are no apis that return the row count of the second table. Would it be okay to use the service layer to verify the second condition for the second table in the end to end test? Would this violate the notion of an end to end test?
4
I think what you’re realising here is that your definition of an end-to-end test
we send a request and expect a response
is incomplete. Not everything has requests and responses, yet it’s still possible to do end to end tests on systems which don’t have requests and responses. Without wanting to in any way elevate them to the status of “authority”, this definition from Browserstack seems better to me:
End to end testing (E2E testing) refers to a software testing method that involves testing an application’s workflow from beginning to end. This method basically aims to replicate real user scenarios so that the system can be validated for integration and data integrity.
If nothing else, that lets us avoid any domain-specific terms like “request” and “response”. It also illustrates that your idea of looking at the row count is probably wrong: a user doesn’t care about that, because it’s an internal implementation detail. Presumably at some point, something else reads that table and a user (maybe not the same user who made the request) does care – it’s that which you should be testing, not the row count directly.
8
Tests are code. They have requirements. You’ve clearly gathered the requirements for this test and they involve knowing the state of the Data Base. Fine. Create that test.
I want to know how rigid do we have to stick to the notion of end to end testing,
You don’t. Tests come in many shapes and sizes.
Would this violate the notion of an end to end test?
Who cares? These are your needs. The name you give the test is far less important than if it is useful to you.
Now, there are consequences to sneaking behind an API. By using the service layer you’re coupling the test to structure that the API never promised anyone would exist. This test locks down how the service layer works. It demands that it exist. This test will need to be rewritten if the service layer is rewritten. None of that would be true if the test had stuck to the API.
But, like you said, you can’t create this test without doing that. Fine. Just be sure you’re OK with the trouble this causes. Make sure other people understand that trouble. That way they know what it means when this test fails. Don’t hide behind some particular definition of “End to end” testing (there are many). The people who wrote those definitions have never looked at your project.
5