How to modify workflow to allow multiple features per release?

  softwareengineering

This question is inspired by comments on this recent question. In that question I asked about a version tagging workflow. A few commenters suggested it was a little strange that I’m tagging and releasing every new feature that is merged into main. It sounds like it is more typically to collect up a few new features and then make a version bump/release. I’m trying to understand how this workflow would look and how I would modify my current workflow.

Note the context here is a small open source python package. Here’s my current workflow:

  1. Branch a feature branch off of main
  2. Develop code on feature
  3. Start a PR on github feature -> main
  4. Version bump in source code. For me this means updating the version input in pyproject.toml. I’m working on configuring my source so that this is the only place in source code I need to update the version in a hard-coded way.
  5. Update the changelog to reflect the new version and the changes (just one feature/fix usually..)
  6. Once the PR is good (feature looks good and is tested) I merge the PR feature -> main
  7. Then in my local repo I add a tag to the new merge commit with the version number
  8. I build the code into sdist/wheel
  9. I publish the built artifacts to pypi

The last 3 or 4 steps are what I consider my “release” process. I’m starting to learn about github’s “release” feature which I guess would show up as step 10 and be something like: create a github release based on the most recent tag. Copy the changes from CHANGELOG.rst into the github release release notes.

You can see in this workflow that every feature that gets merged into main gets a new tag and is released. This results in e.g. one line change logs per version.

To modify this to allow me to collect multiple feature branches I would do the following:

  • I would skip step 4, that is, I wouldn’t version bump in source code on the feature branch.
  • I would keep step 5 (manually writing the changelog for each feature in the feature branch) but instead of putting a version number at the top of the new change I would put a version placeholder like ?.?.? since I don’t know what all features are going to be collected.
  • Then after enough changes are collected in main I would make a release branch off main where I would do the version bump in source code (step 4 above) and set the version in the change log. Then when this release branch is merged back into main I would tag the merge commit with the new version and build and publish the new version code.

So the idea is that if main started on 1.1.1 it would stay on that version as it collects a few features (and the changelog would read ?.?.?). Then when the release branch happens it would bump to e.g. 1.2.0 and the changelog would read 1.2.0.

Note that I’m baking in a few decisions to this workflow. First, for now, I want a human to manually choose the new version and write the changelog entries. I also want to maintain a practice of never committing directly to main since that seems like sound advice.

The biggest problem with this workflow is that if multiple features are being developed in parallel then if featureA is merged into main while featureB is being worked on, featureB may end up with a merge conflict in the CHANGELOG. One solution would be to write the CHANGELOG in the release branch after all features have been collected. But I prefer the CHANGELOG to get updated within the feature branch because whoever is writing the feature at that time knows best how to describe the details about that change. So otherwise, with the constraints that I want humans writing the changelog and that I want the changes to happen in the feature branches, I don’t see a better strategy.

I know that something like conventional commits could help with this problem, but I’m not totally comfortable using that tool yet for a few reasons*, but I am looking into it.

Questions

I know that the above strategy could work in practice. But I’m trying to understand what best practices are. Or, if not best practices, at least get a sampling of what other projects/teams do for their feature/version bump/release workflows (because I haven’t really seen that many examples, and can’t find many thorough explanations online).

  • Would the above proposed workflow be considered a standard workflow?
  • If the above workflow is non-standard, how could it be modified/improved so that it is more standard.
  • What are examples of other workflows that address (or also don’t address) the issues raised above?
  • Would you consider the changelog issues described above to be a major problem that needs to be addressed? If so how can it be addressed? Am I shooting myself in the foot by wanting humans to write the changelog in the feature branches?

*I’m not comfortable using conventional commits because it’s one more thing devs need to learn and adhere to. Alternatively one person could use squash merges or rebases to write a conventional commit when it’s time for the changelog to be written. By why complicate the git history when you could just manually write the changelog… This is not a question about conventional commits so I don’t want to get too deep into this.

edit: Above I propose collecting change in a changelog section titled ?.?.?. I think a much more standard approach would be to have an unreleased section at the top of the changelog where these changes are collected.

  • versioning
  • workflows
  • release-management

3

Answering your questions below:

  1. Would the above proposed workflow be considered a standard workflow?

    There isn’t a strict “standard” workflow that everyone adheres to, but your workflow is a variation of the “Feature Branch Workflow” and “Gitflow Workflow.” The idea of creating feature branches, merging them into a main or develop branch, and then creating a release branch is a common practice in many teams. However, the specifics, like how you handle versioning and changelogs, can vary from team to team.

  2. If the above workflow is non-standard, how could it be modified/improved so that it is more standard?

    Your workflow is already quite standard, but here are a few suggestions to align it more with common practices:

    • Consider having a develop branch where all features are merged first. Once you’re ready for a release, you can merge develop into main or create a release branch from develop. This way, main always represents the latest stable release.
    • For the changelog conflict issue, one common practice is to have a CHANGELOG directory instead of a single file. Each feature branch can add a new file to this directory detailing its changes. During the release process, these files can be consolidated into the main CHANGELOG.rst file.

  3. What are examples of other workflows that address (or also don’t address) the issues raised above?

    • Trunk Based Development: Developers work in short-lived branches or directly in the trunk, which ensures more frequent integrations and encourages simpler, smaller changes. This doesn’t necessarily solve the changelog issue but does promote more frequent releases.
    • GitHub Flow: It’s a simplified workflow where there’s a main branch, and every new feature or fix is a branch off of main. Once the feature is tested and reviewed, it’s deployed to production and then merged into main. This keeps things simple but might not be suitable for projects that need more structured releases.

  4. Would you consider the changelog issues described above to be a major problem that needs to be addressed? If so how can it be addressed? Am I shooting myself in the foot by wanting humans to write the changelog in the feature branches?

    The changelog issue can be a pain point, especially with multiple parallel feature developments. However, it’s not insurmountable. Having humans write the changelog is a good practice because it ensures that the changes are described in a way that’s meaningful to users. A few solutions:

    • As mentioned earlier, use a CHANGELOG directory. Each feature branch adds a new file to this directory. During the release, these files are consolidated.
    • Use a tool or script to automate the consolidation of changelog entries during the release process.
    • Encourage developers to pull the latest changes from main before finalizing their PRs, which can help in resolving changelog conflicts before they reach the main branch.

Lastly, regarding conventional commits: they’re useful for generating changelogs automatically, but they do require discipline. I am a bit biased here too when it comes to conventional commits, I absolutely love them. But if you feel that the overhead of learning and adhering to the conventional commit format is too much for your team, it’s okay to stick with manual changelogs. The key is to find a balance that works for your project and team.

2

There’s no such thing as a “standard workflow”. There are some models that are commonly used – look at gitflow, GitHub flow, GitLab flow, oneflow, three-flow, and trunk-based development for some examples. I’m sure there are others out there that I’m not aware of or maybe that haven’t been written about. I wouldn’t consider any of these “standards”, though. Without understanding the specific context and problems, it’s hard to say which flows are appropriate or best suited for a particular case.

Personally, once you have a system that is operational, whether that’s a stable published library or a deployed system or however you make your work available for use, I like having release branches. Whether that’s gitflow, trunk-based development with branch for release, or something else, depends on the context. But you can accomplish something very similar with tagging.

When you have a branch for release, you would be able to pick your version identifier and update your changelog and release notes for changes. When you branch from your mainline branch (develop in gitflow, main or trunk in trunk-based development), you effectively create a safe snapshot for your release activities that don’t impede development of the next version. You can run additional, execute longer-running tests and fix any issues (but making sure to back merge any fixes into the mainline), and carry out any release-specific work. Another advantage is that if you keep branches for supported versions around, you’ll always be able to do a patch if your release branches are only for major or minor versions and patches are continued development along that release branch.

But if you don’t have any problems now, I wouldn’t change. As your development model changes – you accept contributions from other developers, the number of developers grows, you want to change the distribution model or support for old versions – you’ll run into various problems. As long as you have a stable mainline, you can change your flow in the future.

If you keep your mainline stable, you’ll be in good shape for future changes to both your product and your delivery processes.

No, mainly because of :

“4. Version bump in source code…”

It’s been said a million times; Version published apps, not source code.

If you want to get the version into the source, then you need some out of process build step that updates the sourcecode when you push to master

3

LEAVE A COMMENT