I can’t help you stay focused, but in a team you’d quickly fail code reviews. When you fix a bug or implement a feature on a branch and it comes to review, your reviewer will quickly be saying “WTF did you do that for”. I imagine at first you’d be told to stop doing it and the gold plating would be let through “this time”, but if you kept doing it, your reviews would fail and they’d be sent back for the extras to be removed.

This may still apply to refactoring changes. If they’re not necessary to the code change, then they would rightly be rejected as a clean history where you can see what was modified is more important than anything else – if it goes wrong, how else can you see why it went wrong if you’ve cluttered the diffs up. (this is most obvious for refactorings such as changing bracket placements – massive diff changes for no benefit)

Maybe this discipline is what you need though, until you have your changes rejected and sent back for rework, you’ll never quite get the reality of working in branches. Until then, if you are going to do this, then you might as well stop working like this and simply make all changes on big ‘rollup’ branches that contain several fixes, features or similar. Only creating new branches for major work that you might not want to keep, or that would clutter the history of a single development trunk.

EDIT: After reading Neil’s comment I can’t help thinking that maybe the problem is a lack of up-front design. If you’re refactoring all the time because you keep finding things that need changing (and not quick bugfixes) then perhaps the underlying problem is that the codebase does not have a firm plan to follow. IF it is that you keep finding bugs, then perhaps TDD would help you more (and TDD that makes you write tests before code, also helps with design)

I think its pretty common to get sidetracked on smaller tasks. I actually think its a good habit. It lets to get away from a problem that your facing for a bit so you can later look at it with fresh eyes. But everything in moderation. I don’t think you should stop taking on small changes, however I think you need to focus more on developing that feature before you branch off into other things. Break tasks down into manageable pieces, then set a goal before you allow yourself to get sidetracked.

As for refactoring code I might assume you’re methods are too complicated and need to be broken down. You might want to settle for a set of conventions and code design patterns. It might also be worth using a linter before committing, to prevent a lot of problems from even making it into your code base. Monitoring code quality and fixing bugs before implementing features is a good practice, and allows you to refactor before things get out of hand. You shouldn’t be rewriting code that often. Simply put you should be writing better code originally, and though humans are not perfect there are plenty of tools that you can use to aid in this(I’ll list a few at the end).

A good way to manage refactoring is to make sure you have unit tests for the code your refactoring to make sure your changes behave the same as previously. You should also have some kind of task management software which will allow you to see who is working on what, therefore you can make sure you’re not merging over someones work with refactored code. Likely when you work on a team its up to the project manager(who ever assigns tasks) to keep track of everyone’s work and this would entitle making sure tasks aren’t going to interfere with each other.

Some tools you may find useful:

• Pylint To assure code quality before each commit.
• unittest pythons unittest framework
• JIRA project management software for teams.
• Landscape Code Quality and Metrics
• nose unittest made easier

Those sorts of tasks are distracting because they’re hard to get out of your head, so get them out of your head. I keep an interruptions.txt file open that I put stuff like that into, or if it’s small, I add a TODO comment (which I search for and remove before code review). Then instead of it interrupting my main task just to get it out of my head, I can address it in an objective manner when I get to an appropriate place to do so.

When I address it objectively, there might be several outcomes:

• I decide it’s not that important.
• I decide to put it in a user story on the backlog.
• I decide to do it right then.

Deciding to do something right then is not necessarily a bad thing, if the decision was made objectively and not just because it was distracting you. Team members don’t like bad code getting perpetuated any more than lone developers do. Just yesterday, I noted as a defect in a code review that the author failed to refactor code to remove duplication, even though I knew he didn’t create the duplication. Also, certain refactorings actually save you time, because they make it easier to spot defects before they go through a more expensive testing cycle.

You have to apply some judgment to the situation.

If you go off all over the place on dis-related features, that is wrong and not very productive. Also, I once had a co-worker who did that and every checkout from version control was a nightmare for me.

• If it’s dis-related, you should generally log it for later. (I admit, sometimes, I tend to fix it right then but only if it takes seconds or is a serious bug.)

On the other hand, as you are developing a feature you will discover points, that were not necessarily thought of in planning, but that would greatly improve the software. There is nothing wrong with these. As a matter of fact I find that these little refinements developers add that make the difference between mediocre software and amazing software.

However, when you encounter the latter you have three options:

• If it’s simple and you can just do it now, without sidetracking too much, then do it right now.
• If it’s simple but would just complicate things right this moment, put a TODO and fix it before committing next time.
• If it would throw you off track and really is a new User Story, log it for later.

So the key thing to focus is to have a log. This can be a piece of paper or a sophisticated issue tracking system or anything between.

When first implementing this you will notice that the “backlog” in the log increases rapidly. Don’t worry this is only normal. There is a stage in the development process were all your core features are in place and you will start reducing this backlog.

Stay focused.

I have had a similar problem for years, but I seem to have finally broken through. For me, one of the projects I was having this issue with, was a code base that had very poor design, or often a lack of design. Also, I am constantly reading blogs and books about good practices. But I also constantly question everything I read, question my own conclusions and beliefs, and experiment with new ideas in order to improve and learn. So I had a hard time even sticking to my own design decisions or project coding standards. Here are some things I picked up that help me to stay on track, while also being able to experiment, improve, and refactor the code. Many of these sound similar or related. They are. They work together in a synergistic way. To keep the wording simple, I will speak in terms of absolute truths, even though these are actually just things that seem to help me and may also help or inspire you in some way.

• Keep your work interesting. When you find your current task interesting, this helps you to Stay focused. The desire to work on something other than what you know you should be working on is because the thing you are supposed to do seems very boring or will require a lot of tedious work. When that happens, it has nothing to do with the type of feature you working on. It has everything to do with how you go about working on it. If you are bored, you need to work at it from a higher level of abstraction that allows you to solve the solution in a smarter, more elegant, less tedious way. Lift the level of abstraction just high enough that it involves a tiny bit of experimentation, a tiny bit of play, and a tiny bit of learning. Here are some ideas: Write a script to make that database change everywhere it needs to be made instead of making a bunch of manual changes. Write some classes that create an abstraction that you will subsequently use to write the feature you need with a smaller amount of more expressive code. Use, create, or extend a framework that will eventually do a lot of the work for you. Write a reusable script, command, or dev tool extension that does some tiny piece of work for you, or shows you useful information. This will also have the effect of constantly, slowly adding to a body of useful abstractions and tools that you can reuse. In this way, you will constantly increase your productivity the longer you remain on a given project. There are also other ways to keep your work interesting, as you will see.
• Keep your work boring. If you try to raise the level of abstraction too much at once, your mind will get way away from the feature you are actually working on. Figuratively speaking, you are building too high up in the sky and it will take a very long time before you are able to connect your structures to the buildings on the ground. This will help you to balance your need to Keep your work interesting and help you to Stay focused.
• Frequent releases. If you make a practice of frequently releasing, this will help you to reign in your desire to venture off on massive refactorings or experiments, since you know you will need to do a release soon. This will help you to Keep your work boring and Stay focused so that you can do frequent releases. Doing frequent releases will also help to Keep your work interesting as releases are exciting and interesting. You will soon find that you are addicted to releasing. It’s like a drug! You will always be itching to get another release out there.
• Keep your code releasable. Even if you don’t release at every opportunity, always get back to a releasable state as quickly as possible. Make very small changes to the code. This will give you great opportunities to change directions. Making changing directions easy will help you get back on track if you get off track. This will help you to Keep your work boring, do Frequent releases, and Stay focused in order to keep your code releasable. Releasable code is the opposite of imaginary code in that it could be released whenever you want and at which point it can start affecting the real world. Non-imaginary code that can affect the real world is more interesting than imaginary code that is just a dream. Therefore, this also helps to Keep your work interesting and helps you to Stay focused.
• Write high quality code. Make sure code you add is up to your standards. This will prevent you from being tempted to refactor unrelated code, as it will all be pretty well organized most of the time. This will help you to Keep your work boring and Stay focused. Knowing the code you write is high quality will help you to feel better about it. You will not feel like you are actually just wasting your time by digging yourself into a hole that you will just need to dig yourself out of later. That will help to keep your work feeling meaningful. It will Keep your work interesting.
• Keep a list changes you intend to work on. This includes changes to existing features, new features, bug fixes, code quality improvements, development tools, etc. This will help motivate you to Keep your work boring and Stay focused as you know you have a whole list of things that need to be done. Code quality improvements only need to be added to the list for already existing code. New code you add should be up to your standards. One caveat to this is that for code you yourself wrote, that at the time was up to your standards, you may see a way to refactor it to improve it. One problem is you could probably go on doing this forever. If this cannot fit quickly into the context of another item, it should be added to the lists and broken up if necessary. Lately I have been having success using Asana for my lists.
• Prioritize your list. Constantly reorder the list so that the highest priority items are on the top. Add notes that help you conceptualize what the priority of something should be and how to compare it to priority of other items in the list. Having a prioritized list will give you permission to work on refactorings to existing code or experiments when those things are truly of the highest priority. Having a prioritized list will help you to put things into context and bring your emotions in harmony with your thinking as to your actual priorities. When your emotions and thinking are in harmony, this helps you to Stay focused.
• Break up ideas. Each item in the list should usually represent the smallest change possible that you could release and would provide some kind of improvement. This requires you to take ideas for a feature and really dissect them and realize that they are actually multiple features in disguise as a single feature. This helps you to do Frequent releases. and Stay focused. As a bonus this enables you to change directions at almost any time, by reprioritizing the list based on feedback from releases or business changes. When there is not a constant flow of completed items and releases, this means you are not breaking up your ideas enough. When you do have a constant flow of completed items and releases, this helps everyone to see that you have been busy and they can see fruits from your labor. You will become that valuable, productive programmer that everyone wants on their team!
• Utilize transparency. Keep the list available for everyone involved in the project to see, even if you are the only person actually changing the code. Make sure you have the ability to mark items in the list as complete, and that people can see completed items. This transparency will help motivate you to keep your priorities straight. If the majority of items completed are refactorings, other people following the project will start to get upset at you. Just knowing that they see what you have been working on is usually enough to keep you on track.
• Refactor in very small steps. Even though I already mentioned that keeping your code releasable involves making the smallest changes possible and then getting back to releasable code, I thought this was worthy of extra consideration in the context of refactoring. Learning to refactor this way is an art. Refactoring is making changes that are internal to the design of the code that do not affect the behavior of the program. You can to learn to think in terms of preparatory refactorings, and small little in between refactorings that you are only making because of having the goal of making some larger refactoring. A little change is easier to reason upon. This helps you to write it without bugs. If you write it with bugs, this helps you to reason about the bugs, to investigate them, and to fix them. You can make lots of these small changes very quickly. Going slow will help you to go fast, and will help you to safely make larger refactorings. If you eat up more time than expected making small refactorings in pursuit of a larger one, this approach allows you to reprioritize frequently if necessary. You can finish the rest of the larger refactoring later.
• Get ideas out of your head. While you are working on an item from the list, you may think of something you may want to work on at some point that is not already in the list. Instead of trying to remember it, immediately add it to the list. If you try to push it out of your head without adding it to the list, part of your mind will actively work at evaluating how much importance the idea has so that it can remember it later if necessary. So instead of concentrating on the item you are trying to work on, you are divided because part of you is saying “This is not important, just push it aside and stay focused on what is important”, while another part of your mind is stressed out saying “No, that is not necessarily true. This may be very important. I am very concerned that you might forget about this important idea”. Adding it to the list will put your mind at ease, allowing you to truly focus on the item you are working on. Even if you are not sure if it is actually a good idea, or you want to discuss it with someone first, just explain that in the notes.
• Make lists of lists. Eventually the list will become very long, and people looking at it feel overwhelmed and confused. You will start to have a difficult time figuring out how to prioritize things. That is when you need multiple lists. Each list should have a theme that helps you understand the priority that list has in relation to the other lists. One list might be a list of bugs, another a list of changes or features specifically needed by clients. Another a list of ideas for improvements to the software that are not specifically required by any clients. Another may be a list of code quality issues that you want to be addressed at some point. That will make it much easier for you to figure out what is the most important thing for you to work on next.
• Break up lists into subcategories and tags. The bug list could have each item in one of the following categories: critical, important, minor, and very minor. Tag items lists as being for a particular client or part of a particular feature set or sub-project.
• Collaborate with everyone. Make sure that everyone who can see the lists, can add items, notes, categorize, and tag items. Make sure discussion can happen and be seen right along with the items. As I said before, lately I have been having success using Asana for this. Asana can even be configured to send out automatic notifications, keeping everyone engaged. This will eliminate unnecessary email and communication as people can just add something to a list instead of communicating to you about it. If they want to see if an item is complete, they can just look at the lists. This organized, less cluttered, less redundant way of communicating will help you to Stay focused on the item you are working on instead of reading emails, having chats and phone conversations, and then trying to organize all this information in a way that is then only visible to you, or even worse is entirely in your head.