How do you transition a program from in-development to release?

So I don’t waste much time on writing super clean code at that point because I never know how long something lasts.

Not knowing how long something lasts should never be an excuse for sloppiness – quite the opposite. The cleanest code is IMHO the one which does not come into your way when you have to change something. So my recommendation is: always try to write the cleanest code you can – especially when coding a prototype. Because it will be much easier to adapt it when something has to be changed (which surely will happen).

Don’t get me wrong – my understanding of “the cleanest code” has nothing to do with making code beautiful for the sake of beauty. That is indeed something which can slow you down. In my point of view, clean code is code which is mostly self-explaining (no need to write so much docs – causes speedup), easy to understand (less errors, so less debugging needed – speedup, less time needed to find the correct place to alter – speedup), solves the given problem with the least amount of necessary code (less code to debug – obvious speedup), is DRY (only one place to change when something has to be changed – speedup – and less risk to introduce new bugs by forgetting to change a second place), follows coding standards (less cumbersome things to think about – speedup), uses small, reusable building blocks (which can be reused for many features or even prototypes – speedup), and so on.

I am expected to start testing/final debugging but my gut says I should now somehow clean up and or rewrite stuff to give it proper architecture that makes maintenance etc easier

Doing “cleanup” afterwards never works. Consider you cleanup before you implement a new feature, or when starting to implement it, but not afterwards. For example, whenever you start to touch a method for a feature, and you notice it gets longer than 10 lines, consider to refactor it into smaller methods – immediately, before getting the feature complete. Whenever you detect an existing variable or function name you do not know exactly what it means, find out what it is good for and rename the thing before doing anything else. If you do this regularly, you keep your code at least in a “clean enough” state. And you start saving time – because you need much less time for debugging.

I am in the middle of testing and the bug fix would be a rewrite

… which is the actual proof for what I wrote above: being “dirty” haunts immediately back on you when you start debugging your code and will make you slower.

You can avoid this almost completely if you do the cleanup immediately. Then bug fixes will mostly mean small changes to the code, but never a major architectural change. If you really detect evidence for an architectural improvement during testing, delay it, put it into your issue tracking system, and implement it the next time you have to implement a feature which benefits from that change (before you start with that feature).

This takes some discipline, and some coding experience, of course. It is a similar idea like the idea behind “test driven development”, doing these things beforehand instead of doing them afterwards (TDD can help, too, but what I wrote works even when you do not use TDD). When you do this consequently, you will not need any special “clean-up phase” before releasing.

You have two separate problems, both with the same symptom (sloppy code):

Problem #1: Insufficient requirements control
I don’t mean that your stakeholders change your requirements too frequently, I mean that you’re allowing requirements changes during a bugfix/test cycle. Even the agile methodologies don’t support that; you build, you test, you deliver, you inject new requirements.

Problem #2: You believe the stuff you’re writing is “just for now”
In software development “just for now” code is really extremely rare. You’ve noticed yourself that once you’ve satisfied a user requirement, the rigors of supply and demand make it very difficult to justify going back and re-implementing a “done” feature. So, what to do about it? Always write production code. Functionally for you, that means your estimates to your stakeholders need to be substantially bigger so you’ve got some time to do it right.

Also, please understand that you’re working in the most difficult position as a developer: Read Joel Spolsky’s take on the life of an in-house developer. So, you need to be extra-vigilant if you want to come through with your sanity intact.

It is a common problem – especially when crafting what is essentially a software trial balloon as it were.

There are a number of approaches that can help. Firstly TDD approach can help to reduce the code base to that which is strictly required. If your tests go hand in hand with your code, then you can at least have some confidence that your code behaves as it should.

Take time to refactor as you go. Once you have a prototype and the customer is super-eager to get their hands on it, it is a difficult sell to say you need time to polish what (to them) is complete. I like to check in on a daily basis followed by a refactor check in but YMMV.

Developers who write code quickly are often in demand – we had such a developer in my last department. Every team wanted him because he worked super fast. Once the time came to test and release his code however, the wheels quickly came off. Hard coded stuff, hacks and shortcuts everywhere. His stock soon fell – massively.

Cutting production code from the outset can seem like a drag but depending on your environment, there are many tools that can take the grind out of developing such as Ghostdoc and Stylecop.

It is worth getting in the right development mindset from the outset. You’d be surprised how many back-of-a-fag-packet systems that were supposed to be just stop-gap solutions become cornerstone applications.

Continuously

Speed of development is the main reason to write clean, readable and testable code; it’s not done for beauty, nor other abstract values. Why would I deny that to myself and only do it afterwards for some future programmer?

Sure there might be changes that are mostly cosmetic and therefore not essential; I’d argue that it’s far more useful to have moderately nice code right now, during development, than having a mess right now and hoping to make it perfect later (which, let’s face it, it’s never going to happen, even if you had the time).

You do this by differentiating between “I am just trying this to see how it works” code and “this is headed into the product” code. There are a number of ways to do it.

One is branching or whatever the word is in your source control system. You make a branch for the new report or the new import layout or whatever. If people like it, the job of getting it back into the main branch is a separate, trackable job. It can be assigned to someone and reported on and isn’t expected to just magically happen the day management (or sales) agrees that the feature belongs in the product.

Another is spikes. You don’t do that change in the product. You go off into some separate app, super simple, that exists only for you to have a place to put code. You can be as messy as you like because you’re just exploring the new API or whatever. And again, if you come back and report “yes, we can do that, I’ve figured out how” there is a trackable, reportable, assignable task of writing product-ready code in the product to do what you want.

In both cases, product-ready means readable, neat, following naming standards, with tests, and adhering to your code style and performance targets. In both cases, you make that work visible. I agree that you don’t want to do all that work every time when someone is quite likely to yank the feature back out of the product. But you don’t want to let that work get invisible either. Working in separate copies of the product or in an unrelated product that’s barely more than a test harness allow you to surface the work to make product-ready code once someone decides they want something.

The downside is they can’t decide they want something and ship it (meaning the half-assed, messy, untested, undocumented, possibly slow half-version that you have implemented as a proof of concept) tomorrow. They first time you get pushback on that front, simply ask if you should do it the long (more expensive) way every time just in case, slowing down the path to rejected features. If you ask correctly, you will get a “no”.

Really I think you understand the problem already. The problem is that your coding style is requiring you to do too much rework. The reason it is necessitating too much rework is because (a) it is put together with insufficient foresight and planning and (b) the incremental short term patches regularly put in during development combinatorially increase the complexity of any rework required.

The answer therefore, is to

(a) shift your development style a bit more towards waterfall and a bit less agile. Don’t go all the way though, because classic waterfall has it’s own pitfalls. There is a healthy balance to be had. I know it can be concerning just thinking about stuff for a few days sometimes, like no development is getting done, but you have to trust the process. In engineering you can’t just nail things together and then nail things on top and hope to come out with an elegant solution. If there’s no-one doing architecture and higher level technical design, that means its your job. You have been paying the price of neglecting that job.

(b) try to avoid patching things over. Don’t think long term only when it comes time to do QA. Really you should be testing every little piece you build, all the time, and covering all input cases, those not on the happy-path also. A patch/hack is almost by definition a short term fix, which may well have a long term cost, hits the clients Total Cost of Ownership in the system. Again, the pressure’s on to get code out, so there has to be balance. But try not to put short-term fixes in place, esp. those that tightly couple components that should really be loosely coupled. There will be re-work, so do it EARLY to make it much easier, to avoid the hacks and patches that will mount up over time and become unmanageable.

You write:

Every version is nothing but a prototype. So I don’t waste much time
on writing super clean code at that point because I never know how
long something lasts.
…

Then comes the point where the program is finished and the decision
maker(s) say “that’s it”. I do have a working prototype at this point,
but the code inside is a bit messy from all the back and forth during
the development phase.

A checked-in version can be a “prototype” in that it misses features or some features are not fleshed out, but all the code checked in should be production quality code that does not neccessarily need clean up.

I think you’re postponing you “cleanup” to much.

My rule of thumb is:

• Start with (sub-)feature
• feel free to write incomplete and inclomplete stuff, maybe some c&p to get a feel for what I’m implementing or if I have to scratch the last hour(s) of coding (note that this can go hand in hand with TDD / tests, it’s just that everything is toned down a bit to get quick feedback of the implementation space I’m exploring)
• Sub-feature “works” good enough for now
• Now do the clean up: prior to a SCC commit.
  • Look over the code to see what’s obvious
  • Do a diff vs last commit to review changes and maybe catch some problems
  • Fix stuff I noted down on my scratchpad
• Now I do the commit — this code quality is ready to ship

At this point, the committed code may still contain some workarounds or “technical debt” that it would be nice to clean up, and maybe I’ll clean it up when it’s the natural thing to do for a following sub-feature, but it will be OK if that code is released as is.