Evolving the Team’s Culture

This might be the most difficult part of implementing testing processes in your organization. Every team reacts to change differently. Some people embrace the opportunity to try something new, and some are perfectly happy in their ways and see no reason to change. I can tell you this: nobody likes a fascist. Forcing people to do things a certain way will likely generate more resistance.

Here are some tips for presenting change in your organization:

• Provide empirical evidence, such as “We caught bug X before delivering to the customer, and it took us 8 hours to patch it. The customer caught bug Y one month after it was delivered, and it took us a week to patch it.” Edit: I see that you mentioned that your client may be concerned with the costs of a test site. See if you can prove that catching bugs early will save the client money.
• Show the developers how new tools work. A lot of us are developers because we think that technology is fun. Try to share new testing tools/processes like new toys with which to play.
• Don’t set your heart on any single solution. Let the team experiment with various technologies. For automated UI testing, you can use TFS, Selenium, manually going through test scripts, or other options. Keeping the door open to different technologies allows the team to find which tools are good fits, and it also encourages a democratic process that promotes a shared ownership of decision making. Encourage open discussions with the development team.
• DO NOT expect everything to change at once. Try to focus on one thing at a time. Here is an example of breaking change into smaller stages:
  1. Write some unit tests for the server code that verify that system-critical functionality works as intended
  2. Create a test VM that duplicates the operational environment of your product
  3. Document use cases that can be used to test the GUI
  4. Combine the server code and the client code into a single repository on TFS so that the most recent release is easily identifiable, testable, and able to be deployed
  5. Start executing use case testing whenever new components are committed (automated or manually)

These are just some example tasks. It may take weeks to get where you want to be; it may take months. How exactly your team adopts testing processes is going to depend on your culture. Last but not least,

• DO NOT give up, but be polite and amiable about it. If you keep suggesting ways to enhance testing processes, the development team might try some of your suggestions on a slow day just out of curiosity.

Technical Solution

I wish I could give you some better information on technical solutions for your situation. I work part-time in a .NET shop, and I can tell you that we use TFS exclusively for continuous integration, i.e. version control for client code (EXTJS w/ ASP.NET MVC 3) and server code (C# w/ Entity Framework), automated UI and unit tests on the release branch, and automated deployment to a test site and a demo site (which will be our production site eventually). As I mentioned above, your specific technical solution(s) will depend largely on your organization’s culture.

Good luck!

You’ve got a number of options, but here’s where I’d start. Mind you, I come from the java world, but a website should be a website, really.

1. Website testing: Selenium is a fantastic tool for scripting tests against websites.
2. Virtualize: You can duplicate your environments in VMware or VirtualBox, and save them. When you want to do your system/integration tests, load the appropriate environment and run the tests.
3. Methodology: Agile Testing: A Practical Guide for Testers and Agile Teams has a lot of good advice on this front. I’d suggest starting with something pretty simple, see how it goes, and revise appropriately.

I think the first question here is what you expect to get out of testing. I use tests as a design tool–code that is easy to test is usually good code–and as a safety net. It sounds like what you want is a safety net.

If you want a safety net, you must have a test environment. You don’t want to run tests in your production environment. For one thing, that means that you are deploying code that you think might be broken into production. For another, you can’t control the data or control access as well. For yet another, that means you are deploying code that you think might broken into production. This is madness, pure and simple, and there are better ways.

The next question is what to test. The team has to have a testing strategy and when something is implemented, there has to be an understanding of what tests need to be written. One reason I’m a fan of Test Driven Development is that tests are written concurrently with code, so the tests aren’t seen as a sort of overhead that everyone has to put up with, and you have some assurance that what needs to be tested is tested because everything is tested. You might consider whether it’s plausible to implement TDD in your shop… is it possible that your fellow developers would consider this if writing and running tests was easy?

Anecdotally, I’ve been able to convince a team to go with TDD and CI simply by pointing out a couple of things: First, we all do TDD all the time anyway. Before you write code, you have intent, and after you write code, you generally check to see if it matches that intent. TDD is the process of putting your intent into a test so that you know that the code you write tomorrow doesn’t violate the intent you had yesterday. Second: it’s possible to avoid having your manager/customer bitch at you for breaking something by simply having a test environment. It removes a lot of hassle from your life when these people aren’t complaining all the time.

If you have time/space to do so, you could consider setting up a proper CI environment that you can demonstrate to the group. Knowing that they can do it is not as cool as knowing that you can do it for them.

As far as implementation details, it sounds like you might benefit from reading “Continuous Integration in .NET” from Manning Publications and maybe this blog post from codebetter on selenium testing with .net. For TDD, check out “Test Driven Development in Microsoft .net” from Microsoft.

I think the biggest hurdle you’ll face (and constantly face for a while) is resistance. Years ago I introduced nightly builds and automated releases internally and stopped developers from checking in compiled binaries and tried to discourage them from moving code directly to client sites. I was surprised how much complaining they did about waiting for code to be tested by QC, reluctance to write any basic kind of unit test and how they continue to skirt around processes and move code manually. It can be discouraging to see how reluctant people can be to migrate towards a process that is actually better for product stability. But you have to stick with your plan. Moving code manually and not checking in changes is madness.

If management has your back you just need to proceed with one thing at a time. First get them to check in all changes to a single repository and have everything built from that repo. Seriously everything should be under version control. Everything should be compiled by a server. Once you get builds rolling then introduce automated testing.

The question is framed with testing help of which I’m still learning but it sounds like you don’t even have a proper build and release process. Only one small part of our product uses any kind of automated testing and that’s our Java stuff. A CI tasks runs and sends an email to the last developer to commit if a CI fails. Beyond that there is no other kind of automated testing.

So build and release is something I do have experience with. I had to write from scratch a build and release process for a shop that would check in compiled binaries and sql scripts to VSS and then push those binaries or scripts manually to clients. I used NAnt w/ NAntContrib for compiling vb6 and .Net and then Ant for our Java stuff. VSS was dumped for SVN. Initially I had CruiseControl.NET for managing CI but recently I switched to Jenkins. CC.Net wasn’t handling CI very well and I switched out of frustration. I used NAnt to deploy the code to our dev and QC areas internally with nightly builds. WiX (Windows Installer XML) to install/update clients.

I came across a book which I’m going to read just to get some ideas, it’s called How Google Tests Software by James Whittaker. I really want to get automated testing in to my company too and with all the other work I have to do it’s an ongoing process.
Good luck to you dude!

(oh and we use JIRA for issue tracking and were previously using Serena TeamTrack. Neither really blows my mind but I think our JIRA stuff isn’t implemented correctly)