Agile Forecasting... WITHOUT estimates?
Vložit
- čas přidán 28. 05. 2024
- It's almost a given that we take Estimates, mix them up in a certain way, and produce a Forecast. But what if we could Forecast WITHOUT Estimating?
= = = = = = = = = = = =
New for 2024: my best-ever training:
"How Your Agile Teams Can Achieve Predictability and Productivity WITHOUT Burnout"
→ www.developmentthatpays.com/w...
= = = = = = = = = = = =
Grab your FREE Agile Forecasting Cheat Sheet: pages.developmentthatpays.com...
Learn more about Vasco Duarte and #NoEstimates: czcams.com/users/results?searc...
0:00 Intro
0:30 Catch up with the series
0:44 Story Points Forecast method 1
3:34 Story Points Forecast method 2
5:50 Story Points Forecast method 2½
6:23 The fundamental flaw
7:03 Vasco Duarte and #NoEstimates
9:11 Story Points data
9:40 NoEstimates data
10:08 NoEstimates Forecast method 1
10:52 NoEstimates Forecast method 2
11:31 NoEstimates Forecast method 2½
12:02 The big reveal
-------------------
152. Agile Forecasting... WITHOUT estimates?
#AgileForecasting #NoEstimates #DevelopmentThatPays
If you thought it was necessary to estimate in order to forecast, well, you are not alone. It's almost taken for granted here in our Agile world, that we start with estimates, mix them up in a certain way, and produce a forecast. It's just what we do. But what if we could forecast without estimating Welcome to Development That Pays, my name is Gary Straughan, and welcome back to this short series on estimates and estimating. If you've been following along, you already know that I prefer you didn't estimate, but if we don't estimate, how on earth can we forecast, how indeed We'll get to that in due course, but let's start with common practice. Common practice is to forecast via velocity. Velocity, as I'm sure you know, is the average number of Story Points delivered per sprint. Now, already you can tell I'm talking about Scrum. However, although some of the terms I'm gonna be using are Scrum specific, the general approach is not. I'll say more about that in a moment. As I was saying, velocity is the total number of Story Points delivered per sprint, meaning that if we can figure out the total number of Story Points in what it is that we need to build, then we can figure out how many sprints it's going to take. For example, if we have 1000 Story Points in total, and a velocity of 100 Story Points for sprint, divide one by the other, we're looking at 10 sprints. If we're doing two week sprints, that's 20 weeks. So far, so simple. I guess the only tricky bit is coming up with a value for velocity, and the way I was taught to do that is by plotting a graph. Sprints along the bottom, Story Points on the side, and I'm gonna plot the total number of stories delivered in each sprint. The next job is to come up with a line of best fit, and there are a couple of things to bear in mind while doing so. Firstly, what we're looking for, especially in the context of forecasting, is stability. That's something we're unlikely to get with a brand new team, or a team that's had major personnel changes. It's usually gonna take a few sprints for things to settle down a bit, and for that reason, I'm gonna ignore the first three sprints. Secondly, what we're looking for here is a horizontal line, this would be velocity decreasing over time, luckily, that's relatively rare, this is velocity increasing over time, despite what we'd wish for, this is extremely rare. So yes, a horizontal line, and I'm gonna fix it right around here. Looks like our velocity is, well, would you believe it 100 Story Points per sprint. You know I like my nice round numbers. And now we have a value for velocity, we're ready to make a forecast. Let's say we have 1000 Story Points in the backlog, divide one number by the other, that's a forecast of 10 sprints, or 20 weeks if we're doing two week sprints. As I said earlier, some of the terms here might be Scrum specific, but the approach is not. What we're plotting here is the number of Story Points delivered in each sprint, but we could just as easily plot Story Points delivered each week. So if you're not doing Scrum, but you are estimating, you could use this approach, and the same goes for the approach I'm about to show you. But just because you could, doesn't mean you should. I'll say more about that later. There's another way of forecasting that we can do right on the graph, no arithmetic required. This axis doesn't change, still sprints, or weeks if you're not doing Scrum, but I'm gonna change this one from Story Points to cumulative Story Points. Exactly the same data, just presented in a s
• Agile Forecasting... W... - Jak na to + styl
I've been a Scrum practitioner for about 2 years now. Estimates was a complex concept to grasp, and even to this day, we tend to make mistakes when assigning story points. This seems to be way easier! I will ponder this and see how I can integrate it into my current practice. Thanks.
Way easier indeed!
Glad you found the video useful, and I hope you're able to put this into practice. Let me know how you get on. 👍
There is one very important confounder to take into consideration : this comparison is done on a dataset produced by estimating teams. It is imaginable that the act of estimating leads to a different story slicing, and a different number of stories, than for a no-estimating approach. E.g. because the team has a rule to always split stories until each story has 5 points or less. In that case, a non-estimating approach might not lead to the same amount of slicing, and possibly a less accurate prediction (or more accurate). To draw a more confident conclusion, this comparison should also be done on data produced by non-estimating teams, and see if the accuracy of the forecast is still this accurate. (PS I'm against estimating, but we should always be our own harshest critics ;))
Ah, interesting. I confess I hadn't considered it from that angle.
Annoyingly, there's no equivalent "natural experiment" here - at least not one that I can see. (Putting that another way: an estimating team produces a count of stories as a natural byproduct; a #noEstimates team does _not_ produce estimates as a natural by-product.)
@@Developmentthatpays that's very true! The comparison that can increase the confidence is : non-estimating forecast with estimating data VS non-estimating forecast with non-estimating data, and see if the accuracy is consistent.
Yet again, taking complex ideas and concepts making them accessible and engaging. And moreover, providing some options for us in the real world. Thanks again G!
Thank you, Chay. That's very nice of you to say so 👍
Lean thinking suggests that the only valuable time for a team is planning how to do the work or doing the work. Given this method reduces or removes the planning aspect, it could increase productivity significantly. I’m going to crunch the story numbers and see if it jives. Very interesting 🤔
Did you d any crunching. Would love to know!
I still believe forecasting without estimates relies on the stories being somewhat well written and of "similar" size, if I were to suggest this on projects I have recently worked on, I suspect it would be all over the place as we had stories ranging from copy changes to what you would likely term as epics. In this case the issue isn't with estimating (or not), I think its with the stories themselves.
I think I'm with you on this one: although Vasco Duarte's work suggests that relative Story Size isn't as important a factor as we may think, my guess is that _similar_ story sizes - especially similar _small_ story sizes - are likely to result in a more accurate forecast.
If you're working in sprints, you do have somewhat of an upper bound on the size of stories, which helps to limit the variability.
Vasco speaks about value vs effort in process of forecasting. Estimation requires effort, that can be put in way more valueable way. Especialy when it comes to backlog estimation. The thing is, you never know if thing is small or big technically, when you only have story. ofc if story sounds like FRD - maybe you can estimate it.
For example my 2 scrum/kanban B2B in-house teams works with users that does not know how to express their actuall need/problem, though them being "process owners with high field exp". So when I look into another user made solution/haliucination and extract actual need - the implemented solution may and most probably will differ. What we know about story that most probably there is an original issue that user wants to be addressed. For me forecasting/projecting requires way less effort, and it is easier to measure how long is my backlog and if I will need some outsource to join us for few months to decrease my backlog age (accelerate metrics being my main performance indicator) - thats all I get from it and thats all I do see valueable in long term development planning. Rest of my planning is "Now" and "Not Now". When? When it will be first thing on the list. And when it gets to the top we still estimate story, but we value as Dan North says "fits in your head" approach to time and task size. If I need to slice it to tasks horizonatly or vertically I do it, yet I do it in a manner it does not expand backlog and treat it as single story. We do not waste time for estimation for unforseeable future (i.e. 2 month), as some backlog issues will not even reach priority before they will be cancelled due to shift or merge during grooming.
Of course users hated this approach at first. My "business" require dates, because they are "business", they are lean and do not get how production can not be measured in KPI. It took a while to explain them that unique problems are never resolved in same manner, and that internal team is not a service, but RnD working for company's best interest. And as for sprint planning - we always plan 100+% capacity instead of velocity and we think that everything above 75% is great result.
Sooo much good stuff here,
@Raimondas Baranovskis. Think I agree with every word of it 👍👍
@Michael Mealey True. (But that limit is WAY too high IMO.)
Glad you’re back with new and interesting Content!! Thank you! Can’t wait for more.
It's great to be back. I'm working on the next episode right now 🚀
Love it ❤ anything that helps us move away from using story points for forecasting is a good thing in my opinion. My preference is cycle time and Monte Carlo Simulation, it’s easy and quick, no estimation needed 😊
I need to take a look at Monte Carlo Simulation 👍
I love the idea that people could actually save time and add value instead!
Indeed! What's not to like!
Interesting, this ties in with recent experience. Asked the team to slice stories to be able to complete in 3 days, which effectively standardised story size. Then using throughput was able to assist with improved forecasting, saving a lot of time not story point estimating
"Asked the team to slice stories to be able to complete in 3 days, which effectively standardised story size".
Very useful, thank you!
Really glad you liked it! 👍
An interesting thought from a math/statistics point of view, maybe it helps to convince people:
If I understood #noestimates correctly the idea for the conversion from time or story points estimation to story count based estimation is pretty much to normalize the story points of each story by the estimate no of story points of all stories. If you set all stories to 1 point, you will have an error of 0 for the average story (by definition of the standard error, obviously). Each individual story will have have an error according to the deviation from its own story points to the average story points in all stories.
For the next iteration you take the highest prio items, if we assume that the prio and the standard error of a story are not correlated (which sounds reasonable to me) the expected error is 0.
Now there is another component: There is a high chance that the story points estimation is being further away from the truth than putting every story on 1 point is.
Does that explanation makes sense? Maybe I am missing something. Maybe this explanation could even be extended.
Think the logic is sound... but I think it might obscure the key difference between the two approaches:
- The Story Point reflects what we THINK will happen (on the future)
- The Story Count reflects what ACTUALLY happened (in the past)
It makes sense that, over time, using the number of imaginary story points or the number of stories completed in a sprint should become pretty close.
The main reason I tend to get teams to estimate is to spur discussion and knowledge sharing by teams estimating together. It takes time, commitment, a will to take it seriously and not all teams are willing to invest in it. When you end up with a single person who "happens to know about the specific thing in question" being told to simply give a number and you testers refuse to weigh in on something heavy on front end develolment because "they dont work with that" those benefits break down.
Completely agree with you about the value of "estimating to spur discussion". But there are other ways to get shared understanding and buy-in. (As explored in this video: czcams.com/video/K6PqofeqoCc/video.html )
You nailed it hard, Gary. Excellent explanation and contribution towards #NoEstimation.
Thank you, Kiran. You just made my day!
This is fantastic! Thank you so much! I missed your videos, it has been some time.
Also, where is my t-shirt? lol
You're right: it's been a while!
Oh... the t-shirt... 😳
I like that this seems to be an alternative empirical way to make longer term forecasting easy for a scrum team working on one product. It would eliminate the need to estimate all stories in the whole backlog which is not useful as the bottom of the backlog doesn't have the detail to estimate with any kind of certainty. This is certainly worth an experiment to try it out! And thank you Gary for putting the time in to produce this video clip so well. It was a joy to watch.
I wonder if in parallel to this new method for long term forecasting, there is still a need for the team to estimate the top 20 stories in the backlog, enabling the team to get an evidence based velocity from the last three sprints. (or more if the team is stable)
That evidence based velocity in combination with estimates for the top 20 stories helps the team to see the limit of the amount of work they should take into their next sprint. And estimating also supports the backlog refinement discussions to help the team and the PO understand the top 20 stories in their backlog well enough.
Knowing the evidence based velocity based on story points is also a quite powerful tool for the PO to help business stakeholders understand what the team can sustainably do per sprint.
So far in this series, I've covered *estimates (ep* isode #1), estimating (#2) and _fore_ casting (this one). But there's something missing. *And you found it!*.
I guess the key question is "How can you select the correct about of Stories for the Sprint Backlog if you don't have estimates?"
I don't have a full answer, but here are my thoughts so far:
My experience is that Sprint Backlogs are rarely - if ever - the right size. (This is partly by design: we'd rather pick too many than too few.) So the bar (of accuracy) isn't very high.
Without estimates, we'd be forced to count stories. I.e., instead of (say) 200 Story Points of work, we'd select X Stories/work items. Whether this is workable or not depends on the value of X:
- If the team tends to work on big Stories, then X is going to be small AND highly variable from one Sprint to the next.
- If the Stories are smaller (think Story Slicing), then X is larger and less variable - meaning that selecting a count of Stories for the Backlog is more likely to work.
As with Forecasting, it might be possible to put the above to the test using historical data.
--
Would *LOVE* to hear your thoughts on the above: it would be VERY helpful for the _next_ episode.
Gary
A clear explanation, well organised and at a good pace, thank you :). At 10:32 you assume a total of 100 stories in the backlog. What are you basing that on? Also curious to know why (09:20) you don't use the fibonacci sequence and why the story point value is included in the couloured cards but not in the matrix?
Good content. Well presented and precise. Thank you for that. But there are also simply too many ads. Specially middle of the video, they kill the experience!
That's useful feedback: I'll see if I can cut down on the ads.
Hi.
Thank for the good explanation!
I am curious, how can we know if then item is too big to fit into sprint and then we have to split the item in an incremental way with no estimation? For example, in my team 13 point case is a red flag, by our experience, or we have to split it or increase collaboration to deliver it on time, meaning the 2 weeks sprint. 20 points or more have to be split.
So this analysis in the video was done converting already sized items to non estimates, after things happened, it was done on items in which that analysis on it’s size, I mention above, had been done previously due to sizing them. So maybe that is why the difference in the forecast is not big with both methods?
So how do you handle that conversation on splitting or increasing collaboration in big items with no estimates approach?
In scrum the items have to be small enough to fit into a sprint and the increment should be usable and testable. Also there is a commitment on a sprint goal.
Is there any experience on that?
The approach depends on work items fitting EASILY into a Sprint. Meaning that a BIG Story is one that takes 3 or 4 days. Getting to that point is easier said than done: it takes a bit of technique (slicing heuristics), a bit of creativity, and a bit of practice. (I plan to do a video on this in the not-too-distant future.)
Great video Gary @Developmentthatpays, I've just started following you and enjoy your videos so keep up the good work!
Something that does stick in my mind with this video however, is that the Product Owner still needs to create/refine tickets which are small and achievable. That's based on experience and still involves developers and QA for their acceptance. Do you think this effectively means we are doing more of an 'informal' estimation where the team would be happy to take on the tickets (because they are small enough) without just adding the number on at the end?! Or to re-phrase, we still need to follow almost the exact same process, without documenting a number against the ticket?
If that's the case, I'm struggling to see the overall benefit. Apart from the time saved playing (!) planning poker, instead we subconsciously agree that a ticket is a 5 or under (for instance), but we still have to spend that to reach that initial agreement.
To summarise, what actual benefits is this aimed at, based on my points above?
Cheers,
GM
Awesome question. Two things:
1) In the context of *forecasting* , the difference is stark. Estimating the *entire backlog* (as required for a Story Point forecast) is *massively wasteful* :
- Estimating work that never makes it to a Sprint Backlog
- Forgotten conversations leading to items being re-estimated
- (Sure there are more)
2) As for a side-by-side comparison - _Estimating meeting vs. Story Slicing meeting_ - you'll find a more complete discussion in this episode: czcams.com/video/K6PqofeqoCc/video.html
Remember to grab your copy of the *Forecasting Cheat Sheet* : pages.developmentthatpays.com/forecasting
I don't think it was happenstance that the story count was more accurate than the story points. I think what happened is that as the teams got better with Agile and breaking down stories, they most likely worked to make stories the similar size. That way they ended up having a fairly equal number of stories in each sprint. The question is, without story points, would they be able to initially determine how many stories to pull into a sprint? I think not, but if they didn't use story points to estimate story size and say "that's too big, we need to break it down", they wouldn't be able to determine the number of stories to pull into a sprint and thus end up not having a fairly equal number of stories for each sprint.
I'll be addressing the issue of selecting work for the Sprint Backlog in the next episode.
(I'll respond separately re. moving to smaller stories over time.)
Really interesting. Think I have to watch this a few more times.
Also what does repo mean? (Data set?)
I think made a mistake using the word "repo". It stands for "repository" and is usually applied to _code_ repositories. I guess Jira is a repository of sorts, but I probably should have just said "Jira" a bunch of times :)
@@Developmentthatpays no makes total sense. Great video, excellent production. Thank you!
Really glad you liked it. 👍
Thanks for the videos, Gary, really interesting and thought provoking stuff.
I have a question about the stories we are forecasting here. Within an Agile framework we are encouraged to only refine stories as they are prioritised to the top of the product backlog, i.e. don't bother refining a story/feature/epic that we are unlikley to develop for another 3 months, because this effort might be wasted should the PO decide to drop it altogether. Let's assume for a sec that this is a big piece of work that would take the whole team 4 or 5 sprints to deliver, i.e. if you were to split it and then story point it, it would be around 300-400 points in total. I have also noted that the range of stories you use in your example is from 1 to 40 points. So if we are asked to include this epic in a forecast then how do we deal with this? Do we need to refine these epics to keep them down to a reasonable size, e.g. roughly less than 40 points, but without actually story pointing them?
This is what I find so fascinating about Vasco Duarte's work/research: what you've described is just one of a *million* scenarios where the makeup of the backlog seems likely to impact the accuracy of a forecast; one of a million scenarios where it *_feels_* like we need more *detail* in the form of Story Point estimates - and/or we need to impose limitations on the makeup of the backlog (e.g., limiting the Story Point range). But the empirical evidence suggests that such measures will do little to improve the accuracy of the forecast.
@@Developmentthatpays Thanks Gary. So I think what you (or rather Vasco) are suggesting is that you "count" ALL stories as 1 story no matter how big and we need not breakdown even the big epics ... the still cound as just one story. Right?
@@makemeagile - I don't know if Vasco made any allowance for Epics.
I am going to try it and tell you the result in 6 month =)
Vasco's discovery is not so much a discovery, as it is a result from common statistical principles. As example an average 'storypointet' project would tend to a story point distribution that falls into Pareto (80-20) distribution. A project with too many complicated stories (breaking away from pareto to like a non-pareto 60-40) would in the end be taken into discussion as there are too many stories that are too complicated and need division to manageable pieces. What Vasco's method can encourages is to decomplexify the stories in the start or in the process of the project, aka. we learn as we go.
But this No Estimate approach requires you to refine the whole 'project' into small User Stories at the start - isn't that waterfall?
Surprisingly, it doesn't require that at all. All that's required is a more or less stable "rate" of break-down. (And, for that matter, a more or less stable rate of... just about everything!)
Aren't story points an "estimate" of complexity or effort that the team agrees on? If that's true, then we are using estimates to forecast.
Yes, Story Points (or T-shirt sizes, or time) are estimates.
Logically, points and velocity just doesn't add up. Doing 5 stories of 3 points each is not the same as 3 of 5 points, nor 1 of 15. It's just not the same amount of work shared by the team members. So velocity of 15 tells me nothing about how much can be done, unless all your stories are always the same size and all team members are equal in their proficiency.
"oing 5 stories of 3 points each is not the same as 3 of 5 points, nor 1 of 15"