I’ve been trying to develop web applications while at the same time creating the testing. I understand unit testing, I can declare a test method and test specific methods in my application.
But I don’t see what is the best way to use selenium web driver for testing. I tried to test the a specific site for broken links or broken images and I had to use an external http connection library since selenium doesn’t provide header results etc. It is very slow to traverse a whole site. While you can just pass the URL to W3C link checker and you’re done.
Are most developers creating a class for each page and testing individual ui elements since traversing the whole site is a lot more complicated.
What are the benefits selenium brings or ui testing that can’t be tested for with unit testing?
Unit testing is just one of the multiple types of testing. Basically, you’ll find three types of tests:
-
Unit tests.
Each of those tests ensure that a tiny piece of application is working as expected. For example, a unit test will ensure that if, on an e-commerce website, I create an instance of a
Product
class with a price inferior to zero, an exception will be thrown. -
Integration tests.
This is similar to unit tests, but with a larger scope. Once you created the blocks of the application and test them, you start to assemble them into something bigger. Integration tests ensure that those blocks work as expected when assembled.
For example, your product management block was unit tested to prevent creating products with a price inferior to zero, and your administrative interface block was tested to prevent invalid values from being entered by the users, but what about those two blocks interacting one with another?
-
System tests.
The scope of those tests is even larger. While integration tests are more concerned with interfaces between two blocks, system tests are run on the whole system.
For example, what happens when two administrators attempt to modify the product with the same ID, while the primary database goes down for a few seconds, letting the mirror database to handle the operation?
To complicate things a bit, there are additional types of testing, such as:
-
Functional tests.
While the earlier tests are often conducted by developers themselves, functional tests are usually done by QA department to ensure that the product corresponds to the requirements. While unit, integration and system tests are focused on the actual blocks and their interaction, functional tests are focused on requirements, which means that they treat the application as a black box.
For example, if a requirement specifies that the product cannot be purchased more than once per ten seconds by the same user, the test would consist of ordering the product twice in a period of ten seconds and check that the second order failed. Here, the QA tester doesn’t care about how the functionality is implemented and what parts of the app are interacting to prevent the frequent purchase.
There is also security testing, A/B testing, regression testing, acceptance testing, non-functional testing, etc., but let’s focus on system and functional tests above.
A functional test such as:
- Ensure credit card information is remembered by the website.
- Go to the product 123.
- Add it to cart.
- Go to cart.
- Make the one-click purchase.
- Repeat steps 2 to 5, given that the entire test should take less than ten seconds.
-
Ensure the last step fails and the user is informed that:
- The purchase was not made because there is a risk of mistake from the user,
- The IP address was saved in case of abuse,
- The e-mail was sent to inform the user about the risk of abuse,
- The credit card was not debited.
is:
- Impossible to make by hand (unless you have a really fast tester),
- Is just too painful to do by hand anyway (especially in a context of Continuous Integration, where it may need to be done dozens of times per day, among several thousand of other tests),
- Is quite difficult to implement using “ordinary” programming, such as a bunch of HTTP requests and responses checking,
- Would be unmaintainable if implemented using “ordinary” programming.
This is when Selenium is useful. You create your scenario, and Selenium runs it again and again, checking that the test still passes. A similar scenario concerns system testing; although I don’t expect Selenium to be very present here, it’s not excluded that some teams rely on Selenium for this sort of testing.
If your project has requirements, a bunch of Selenium tests will be organized around those requirements, like in the example in my answer.
If not, each of those tests will just be a possible scenario which could get terribly wrong, similarly to what you do when choosing what unit tests to create. Just like in unit tests, you will test some method for input value 3
(ordinary case), 0
(edge case), -1
(exceptional case) and NaN
(exceptional case), Selenium tests can encompass very diverse situations, such as:
-
The user purchases a product. That’s all.
This is also probably the most crucial test one can have for an e-commerce website: if users can’t purchase, it would be slightly problematic.
-
The user adds a product to the cart, waits until the product is not available any longer, and then tries to purchase it.
-
The user adds a product, then removes it, then presses Back, and tries to purchase.
-
The user bruteforces the website by adding every available product with maximum quantities in the shortest possible amount of time, and tries to purchase that.
-
The user starts purchasing a product, but leaves when the transaction is running, and never returns.
2
UI testing is hard. For the most part, it doesn’t lend itself well to unit testing.
Tools like Selenium are designed to fill this gap. For the most part, they do this by automating the user interface, simulating the operations that a user might perform via mouse clicks and typing. But these tests tend to be brittle, and they are generally not comprehensive.
One approach commonly practiced is to push as much of the UI logic as possible out of the View and into a View Model object. View Models can be unit tested, and the UI surface logic is correspondingly simplified for Selenium.
One huge advantage is that in many cases you can use your selenium tests against the production codebase and deployment. They are certainly brittle and non-comprehensive tests in many ways. On the other hand they can actually hit all of the moving parts moving together which is a space traditional unit tests just can’t hit.
You can even extend this out to production uptime, performance monitoring and even load testing — we were a longtime browsermob user [now Neustar Web Performance] and it flat out rocks. On the one hand you could hit the fully configured stack and see what kind of traffic you can take on real-world infrastructure [load testing] and on the other hand you can monitor the uptime — including cruical infrastructure like general network connectivity and DNS resolution as well as app specific 3rd party services.
Also, from a pure development perspective, I’ve used it many times for quick automated validation tests of underlying application changes like major library upgrades. On many platforms just checking if pages can be rendered at all will check a lot of boxes and having something that can mechanically request them is worth it’s weight in gold.