How do I get from a highly manual process of development and deploy to continuous integration?

We have a development process which is completely manual. No unit tests, interface tests are manual, and merging and integration are as well. How could we go from this state to implementing continuous integration with full (or at least close to full) automation of build and test? We have a pretty intense development cycle, and are not currently using agile, so switching to agile with CI in one move would be a very complicated and expensive investment. How can we take it slowly, and still moving constantly towards a CI environment?

The approach I’d take depends on your current needs. Since you’re making a process change, it sounds like there’s a problem that you’re trying to solve. What problem(s) you are trying to solve will change the approach that you take. Eventually, to achieve a good continuous integration environment, you’d want to address all of these issues.

Can you build easily? Your build should be executed by a single command to something like a Makefile or an Ant script or a Maven script, depending on your language and target platform. Simply executing this script should do a complete rebuild and execute any tests that you have. If you don’t have this, creating one would quickly make the manual process less painful.

Is your build slow? If your build is slow, I’d look at either starting a regular (nightly?) build or improving your build time as the first step. If you check in to an integration or alpha branch at least once a day, a nightly build would be sufficient. Every night, your build starts and your quality assurance team has a fresh build to work on every morning without downtime in the day waiting for the build to be kicked off manually. If you don’t check in every day, it would probably be better to check to see if there are updates before building. The other option would be to improve build performance so that manual builds take less time to execute. When you move to automation, it will allow you to see the results of the build/test cycle faster.

Is your quality poor? If you have poor quality (unstable software, defects detected after shipping), focus on improving and automating your tests first. Create unit tests that can be automatically run (once you automate your build process) and give instantaneous feedback to developers. Also work on automating integration and acceptance tests as much as possible so they can be executed and provide feedback to the developers and quality assurance teams.

There are probably other questions that you can ask, depending on your individual situation. But take it slow. Implement one aspect of a continuous integration culture at a time – automated builds, automated tests, fast builds, instantaneous feedback, visible results of build/test cycles, and automated deployment – starting with the ones that will add the most value to your team.

As you commented that you’re using PHP – installing Netbeans for PHP might be a good way to go.

It sounds like you’ve got source control which is the first obvious step
The next obvious step is to make sure Doc Blocks are written for each new function and any function you happen to edit – which Netbeans will help automate
Beyond that Netbeans gives you good instructions on how to integrate all the components you need for CI.
Going from PHPUnit
to PhpDocumentor
through to the Hudson (Jenkins) CI server – they’re all bundled in the PHP options for Netbeans.

I suggest using Jenkins which allows you to add build scripts but also commit hooks that run on each commit. You don’t need to configure your entire build on one day but you can make small steps each day.

For example, you can enhance your build script whenever you have time and you can integrate more build reports or unit testing as you go along. In the beginning Jenkins will create an automatic build on each commit and the build will be done within a second because you have not yet specified any work. This assures you that the system is working and you can then start working towards the build system that you want.

Pick one easy goal, get success and then set another goal. IMHO your first goal should be a nightly build. A long time ago we had deployment problems because of sloppy checkins, bad version numbering, unknown dependencies etc and a nightly build solved a lot of those problems. It gave us a single reliable output for testers, installers, deployment etc. Everyone could be confident that version x.y.z was the latest successful build. No more testing old builds.

Creating a nightly build may create some people problems, you will need a strong sponsor to help you. I hit lots of objections such as ‘but that’s not how I like to work’. Of course, but the end users don’t like the bugs in your work. A good sponsor will assist you with objections.

Creating a automated build will uncover other issues like dependency problems. When you start turning over rocks be prepared to be surprised by what you find underneath. Part of the pain of getting to automation/nightly builds/CI is getting everyone onto the same dependencies/libraries. I wish we had maven back then.

Develop a pitch for an automated build process. Try to find bug reports from the deployers or end users that could have been prevented.

First of all, you don’t need to be doing agile to go to a CI deploy environment.

We have a pretty intense development cycle

Well, don’t we all? Just bite the bullet and do it! No more procrastinating, if the team want to do it, then assign someone (or more) to do it.

If management grumble about being a man down for features for a week, come up with some stats around how much time you think moving to CI will save you. How long does it take to do a manual build? how often does a manual build not work and have to be redone? How often does a environment have to be rebuilt because of a poor manual build etc.

Once you are over those hurdles, figure out:

which CI tool, there are lots
what environments you plan to have
what servers you’re going to need for those, and how many
identify all the different disparate “sections” of your system, figure out the dependencies (you probably know that already) and get it into your CI environment.

Start by automating your build. With PHP, this will probably be a shell script. It should do the following things:

If you’re using a database, create a new database instance and populate it with initial data. Don’t be tempted to reuse an existing instance, as that will eventually cause spurious errors. IMO, this is the most important stop to automating a build, regardless of the language; your data is what drives the system.
Check out trunk into a clean directory and apply any configuration changes (for example, pointing to the database instance you just created).
Configure your server to point to this directory and start the server.
Run wget or curl to verify that your homepage is accessible.

This is a basic continuous integration script, although it doesn’t do actual testing. You can run this script with cron, or from a commit hook on your database.

The next step is to add testing. In a perfect world, you’ve separated your business logic from presentation, and could write PHP scripts that load up your modules and exercise them. I’m guessing this isn’t your world.

In which case your first round of tests will probably use wget or curl, along with some other shell programs (grep, perhaps) to verify that you’re getting something close to what you expect. You could also write PHP scripts that exercise your app: deploy them on another server, and invoke from your build scripts. How detailed you want to get is up to you.

For completely exercising your app, you’ll probably want a tool like Selenium.

However you proceed, be sure to mail all of your developers when the build fails. You don’t want to leave the build broken, and the person who broke the build may not be the best person to fix it.

Filed under: softwareengineering - @ 07:06

Thẻ: continuous-integration

Thiết kế website giá rẻ

Danh mục

How do I get from a highly manual process of development and deploy to continuous integration?