I’ve never seen anyone estimate the entire product backlog. Typically what happens is that the product backlog is prioritized on an going basis by the Product Owner. Scrum has an ongoing activity known as Backlog Refinement in which the Product Owner and Development Team review Product Backlog Items and add sufficient details to implement the items, estimates, and reorder as necessary (perhaps due to a greater understanding of the complexity or technical dependencies). By Sprint Planning, a good number of Product Backlog Items are refined and the team pulls a sufficient number into the Sprint, based on its priority in the Product Backlog.

The acceptance criteria should be added by the Product Owner before estimation. Since acceptance criteria are based on the business needs (both the functional needs as well as any non-functional or quality attributes associated with the Product Backlog Item), the Product Owner is in the appropriate position to associate these needs with each Product Backlog Item. If there is any additional work that the team feels is necessary to successfully complete the item, they can add it during the discussion of the estimate to ensure that all necessary work is documented and included in the estimate. The team can then include the work necessary to not only complete but also verify and validate the completion of the item in the estimate.

Also, I’ve never seen a “product burndown chart”. The burndown chart is typically for an individual sprint, not for the entire project. On Day 1 of the sprint, the burndown chart reflects the value carried into the sprint. As you implement items from the Sprint Backlog, the chart is updated to reflect the remaining value to implement in the sprint. I don’t see why you can’t have a product burndown chart based on the refined Product Backlog Item, but as Product Backlog Items are added, refined, and removed, the chart will fluctuate – this should happen far more regularly than changing scope as defined in a Sprint Backlog.

Based on your comments in Thomas Owens answer, I would like to add some Points. I agree with his answer!

In the first (few) Sprint(s) it is just not possible to make a good estimate for the project time. Maybe you know the Cone of Uncertainty? It says:

At the beginning of a project, comparatively little is known about the product or work results, and so estimates are subject to large uncertainty.

Especially in Scrum, where you have very big and vague goals/tasks/requirements which will be specified in detail while the project is running.

You probably also know the 4 Keywords: Price, Quality, Time and Features. Standard projects will try to fixate Price, Time and Features. The quality is most likely to suffer this way. Scrum tries to fixate the Quality, by passing the variable to the Features. As you should potentially be able to ship a product after each sprint, features may be dropped or postponed in favour of other, more important ones (from the view of the customer).

Now for the estimation of the project time. You can estimate like Thomas Owens stated – maybe a few more than you expect to finish in a sprint. You may now ‘guess’ the rest of the work by just taking an average amount of Storypoints for the stories. The total amount of Storypoints is very unreliable, but you have your first ‘guess’.

When your next sprint starts, you have estimated another set of stories. Less (or maybe more) tasks are now in your product backlog. Maybe you have already a better idea of what an average story takes. You can update your estimation now. The total amount of storypoints for your product changes. And after the third sprint maybe, you get a good idea of how much it will really take. At least you will see, if its possible to finish all the features with the current setup (based on the velocity).

Edit: forgot to mention: it is maybe better to make the product release chart a BurnUp chart.

You don’t have to estimate the entire backlog. It is ordered so you can estimate the first stories.

If you need to do a release plan and its chart, you will have to estimate enough stories to cover it. So if you want to plan 3 month ahead, then you’ll have to estimate at least 3 months worth of stories.

For the management being able to plan at release level is very important.

Estimating stories is generally more quicker than estimating during the release plan and can be done at any time.

All of these suggestions seem to make a pretty big assumption – that user stories can be prioritized at all without estimations. I’ve never worked in an environment where that is possible – maybe huge companies who have tons of money to burn. To me, it seems like the quickest way to burn through VC money… trying to build the most important / impossible things first without also building some low-hanging fruit that can help trickle in some money.

We use the following method:

Estimate what you know

Refuse to estimate what you don’t and call it a spike

Tell management that at least 1 spike must be performed. The outcome / deliverable of a spike is either an estimation for the story (or stories) the spike was created for or some sort of email, document, powerpoint, etc indicating the lessons learned and the suggested next steps for the next spike.

After 2, 3, or more spikes for a business requirement, it quickly becomes apparent to the entire team and management that this is a risky area full of uncertainty. And, since management doesn’t like things that don’t have estimates, they tend to prioritize spikes… they want their estimates. Either way, it works out well – all of the high-risk and uncertain stories start bubbling to the beginning of the release and so everyone gets a much clearer picture early on about what is realistic and what isn’t.

Even better, leadership learns quickly that spikes are not commitments since they are not estimated and our teams don’t commit to things that aren’t estimated.

The final step – a critical path for various components of the application (UI, Database, Services, etc)… this means spikes and stories. I know this may seem waterfallish and you don’t need to do a full systems engineering critical path, but just a quick “this prevents or blocks this from being worked on”. You need to quickly identify what can be done in parallel, what is blocked, etc. This really helps define release plans, sprint themes, and milestones.

The only time you’d want to estimate the entire backlog is if you are required to produce an estimate for completion of ALL features. That comes up when trying to integrate SCRUM with projects that need a ‘finish’ date for the entire project.

Assuming you don’t need a ‘done’ date for the entire backlog, then I’d just estimate the stuff for the next iteration or two in detail and get on with the development. Since you’ve got a priority to the backlog, you don’t need much more than that.

After prioritising the Product Backlog I always try to estimate all the User Stories as part of the Sprint Planning session for Sprint 0. There have only been a couple of projects when there were simply too many stories for this to be practical in one Sprint Planning session. When this happened, as we were estimating the stories in the order of their priority, when we ran out of time we only left low priority stories un-estimated which we weren’t intending on working on in the first Sprint. We were then able to finish estimating the remainder of the stories in the planning session for the next Sprint – don’t waste valuable development time on estimating, confine this to the timeboxed Sprint Planning sessions.

We use relative estimation in Story Points which we arrive at through Planning Poker. This makes the whole process quick and easy enough that it is almost always achievable to estimate the entire Product Backlog at the outset of the project.

In answer to your second question; if there are specific acceptance criteria for a User Story, yes these should be added when the story is created. If you don’t, as you have identified, it will be hard to estimate not to mention hard to implement correctly! If you have general acceptance criteria which apply to all stories (perhaps a maximum response time for any transaction or a minimum amount of code coverage), then these should be defined in a Definition of Done. This forms a set of quality standards which each team member must hit before they can declare a story complete.

Personally I prefer not to create an overall Product Backlog as the data it represents can be misleading. As new User Stories should be coming up all the time, whilst you’re burning down against the backlog you’re often adding more to it. Showing the number of incomplete User Stories or the total incomplete Story Points over time isn’t necessarily a fair representation of the progress of the project.

And I tend to disagree with the shared sentiment here that entire backlogs should not be estimated or how it can be misleading.

In my experience, if estimating the entire backlog is a chore or misleading, the backlog is too big or the team isn’t good at estimating or the team just doesn’t know, the release is too ambitious, etc. We shoot for 3 month releases (6 sprints).

We also have two definitions of done… the one most people are familiar with and the “validated definition of done”. The one most of you are probably familiar with is pretty lax for us. We need something just done enough to put it out there, get feedback, and learn from it. Typically, this is your product owner “accepting” the story. However, we’ve learned very early on that product owners make poor proxies for users and so we don’t “validate” a story until its been put through the test of real end users. Once we feel comfortable, haven’t received any glaring negative feedback or been through some usability testing, we’ll mark it “validated”. Once validated, this means we should shore it up with some automated UI testing, quality controls, etc. We want to etch this user story in stone – our product owner said it met the business requirement and our users don’t have a problem with it either.

I’ve heard of other cases where even if the users don’t have a problem with it, it might never be validated so it actually gets put into an “scrapped code” repository. If it won’t help the app sell, users never use it, users don’t necessarily love it… why spend the resources to shore it up, maintain it, etc.