It takes as long as it takes...

So, the project I have been managing in one form or another for the last 2 years and 3 months is really close to being done. Although I don't really know how close. Why? It's a web development project. It involves replicating an entire ecommerce site that has been in production and gradually built on and customized for nearly 8 years. It's been hard work, and the team is a bunch of brilliant developers - truly rock stars in this neck of the woods. All of my data says we are 96% done, with just a few bugs to fix - but we've got another round of user testing to do that could turn up some showstoppers and delay us a few more weeks - please, not into 2008.

In January we thought we'd be done in September, worst case. In September, we thought late October was an accurate estimate for completion. In late October, we reset to December 1. Then we slipped it out two weeks, and I think that might finally be it. But I am not sure... because it is a software development project.

Had we told our sponsor and our sponsor's boss in January that it would take all year to finish the project, people would have been crabby. So, we took some pieces out of the schedule - like testing and bug fixing cycles at the end of each release, in order to compress the schedule. Guess what? We ended up using the time at the back end anyway, and the project will take as long as it would have taken had we planned it in at the start. We just deferred the pain.

Understand - each time we refactored the schedule, we took all of our experience to date, got everyone's work estimates and gut feelings and factored these in. But each time, there was more work to do than there was time to get it done by our target date.

There are a lot of good articles and posts on software project management and estimating and why this is so hard. Perhaps software development needs to be treated more like scientific research and less like a project - both are attempting to accomplish objectives that are unknown quantities - any attempts at estimates are just guesses. No one asks a scientist how long it will take to achieve a breakthrough in his or her research, because he or she is doing work that has never been done before. This is the case with many software development projects - the exact project has probably never been done before, so most attempts at estimating effort and time to completion are guesses. Do the best you can with the information you have, and know that it will be done when it's done - or when your sponsor's boss decides it's done.