In the last two Certified Scrum Product Owner courses that I taught, people have asked, “What’s next?” A beautiful question indeed. My answer included a path to Certified Scrum Professional that Carlton Nettleton and I have developed, a series of advanced courses that Applied Frameworks offers, and the Scaled Agile Product Management course that we offer.
Let’s go much deeper in how people in the role of Product Owner and people in the software Product Management profession should think about levels of training and knowledge acquisition. I thank Luke Hohmann for the following structure that he attributes to Meiler Page-Jones.
1. Innocent. You have not been exposed to a given area of knowledge and are unaware of its existence. In other words, you have absolutely no plans associated with the topic in your cognitive library, your preexisting set of solutions and experiences.
2. Aware. You have been exposed to an area of knowledge (such as a new technique to organize your product backlog), perhaps by reading an article, and can see its relevance, but have not yet applied or used it. Your cognitive library may have one or two plans regarding the body of knowledge. These plans are rudimentary at best. You are still unable to use it for any useful purpose.
3. Apprentice. You have had some formal training in the structures, processes, and outcomes associated with a topic, perhaps through a two or three day workshop. You have begun the task of creating and storing plans in your cognitive library. At this stage of learning, structures tend to be viewed as absolute, not to be violated. You can produce simple outcomes for well-defined problems, but require the assistance of more expert individuals to solve ill-defined or new problems.
4. Practitioner. You are able to accomplish moderately difficult tasks without assistance. Your cognitive library is fairly well developed, but you must still rely on experts to accomplish very complex tasks.
5. Journeyman. You regularly use the body of knowledge in your work, and begin to question and/or modify structures to suit your needs. At this stage your cognitive library is reasonably large. You begin to apply existing plans in novel ways. Individuals at levels 2 through 4 seek your guidance.
6. Master. You have mastered the body of knowledge, and can effectively apply it in many different situations. Your cognitive library is quite well developed. It contains plans enabling you to solve well-known problems quickly and easily. You are adept at applying plans in novel ways. You can easily adapt or invent appropriate structures to aid in problem solving.
7. Expert. With substantial expertise, you move beyond the master stage by extending the collective body of knowledge through lectures, writing articles and/or books, or applying the knowledge in new problem domains. The difference between a master and an expert is subtle, but important. Both possess extensive cognitive libraries, but the expert works at externalizing their library in a form suitable for use by others.
All of us at Applied Frameworks focus on how to assist you on your journey to the Expert level of product management in each of the frameworks you need to succeed and excel at your job.
We will help you assess where you are now and your path to the next level. We absolutely value your input and specific feedback as we work through our minimum viable product to create a service that delivers value.
Jump to Apprentice level with Luke and me at our Certified Scrum Product Owner course in Palo Alto next week.
1. Leverage two trainers for the price of one! Luke and I teaching together will be awesome!
2. Obtain two certifications at once!
a. Scrum Alliance Certified Product Owner
b. Innovation Games® Yellow Belt Use games to learn!
3. Get Luke’s book!
4. Give away, yes, we'll GIVE YOU a subscription to Innovation Games® online!
Sign up here!
All the best,
Seth Lane is a little boy who was born without an immune system. He needs our earnest hopes and healing energy today. If you didn’t catch the compelling video of little Seth on AZ Central, you’ll want to.The Axosoft team sending warm thoughts to Seth from sunny Scottsdale, AZ!
Seth sits in his hospital bed, hooked up to tubes, puffy from medication and bruised; yet somehow, when he looks into the camera to tell us what he loves, he seems buoyant! He loves Fireman Sam from Paw Patrol and the color yellow. Despite his circumstances, 5 year-old Seth is reaching out and asking us for a little support in the form of his favorite color, yellow. Yellow makes him happy, cheers him up, and gives him strength, all of which he will need before, during and after his upcoming (second) bone marrow transplant.
So… put on your yellow TODAY, take a photo of yourself, and share it on Seth’s Facebook page, on Twitter with #WearYellowForSeth or on Instagram. Also, his family set up an account for everyone interested in helping with his increasing medical expenses. Axosoft just gave, maybe your company can ban together and give too, at http://www.gofundme.com/SupportSeth
If you need a little more inspiration to give Seth your healing energy, Dr. Deepak Chopra says, “There are healing forces in nature that science is only beginning to understand.”
As part of our recently announced partnership with Code for America, Rallyers took a leadership role in supporting the annual CodeAcross event here in Boulder, where Rally is headquartered. On a recent snowy Saturday, Code for Boulder—the local all-volunteer Brigade—hosted “Crafting Civic Tech: The Housing Edition.”
Veteran Rally Agilists, company founder Ryan Martens, and Rally coach Ronica Roth led the group of more than 50 community members on a six-hour design thinking journey. The goal: identify ways to improve and increase civic engagement among community members and local government, with a focus on the issue of housing.
Ronica Roth and Ryan Martens explaining the design thinking approachDigging Into The Topic of Housing
The City of Boulder sums up the area’s housing situation as “Setting + Culture + Opportunities + Quality of Life = High Demand for Housing.” To create a “collaborative community conversation,” the City is teaming with Code for America to increase two-way dialogue with residents and encourage citizens to take a more active role in key local issues."The city takes transparency and community collaboration seriously. Communication in the twenty-first century is changing and we want to harness the entrepreneurial spirit of our community—and be part of that change." Jane Brautigam, City Manager
Because the timing of the annual CodeAcross event aligned well with the first few weeks of the project between the City and Code for America, the local Brigade designed the gathering to broaden and deepen the conversation between community members, then share insights and ideas with local government.Crafting Problem Statements from Empathy Interviews
The Stanford University Institute of Design created a design thinking workbook to lead people through the process of gaining empathy as a critical first step toward creating solutions that solve real human or business needs. Without addressing problems that people want or need solved, products and services are destined to require “push” from the creator rather than “pull” from the user. Participants started the day by practicing empathy interviews on each other. Afterwards, several pairs shared with the whole group the problem statements they crafted after interviewing each other.“Listening is a critical skill to develop—it was a great setting to practice listening.” Dina Robin, participant
Next we heard from Becky Boone, the Code For America fellow who is conducting an in-depth exploration into the needs and challenges of improving community engagement in Boulder. Becky has done numerous empathy interviews within government as well as the community. As a result, she crafted problem statements on the topic of housing. Here’s a small sampling, written in the design thinking format: [Name] needs a way to [user’s need]. Unexpectedly, in his/her world, [insight].
- Hugh needs a way to weigh in on housing issues in Boulder. Unexpectedly, in his world, he does not have a computer or internet access.
- Jake needs a way to feel like his opinion matters. Unexpectedly, in his world, he hasn’t seen public opinion matter when the City Council makes a decision.
- Tam needs a way to feel like her presence in Boulder matters. Unexpectedly, in her world, she feels her contributions to the local economy aren’t valued, hence her opinions aren’t valued.
- Bao needs a way to know who to talk to when he wants to talk about housing. Unexpectedly, in his world, he has no awareness of the current structure in place.
- Darryl needs a way to feel like his demographic is valued in Boulder. Unexpectedly, in his world, people in power have spoken dismissively of what he has to offer.
- Maxwell needs a way to allow change in his neighborhood. Unexpectedly, in his world, people are squashing ideas for the city, eliminating possibilities in his neighborhood.
Teams ranging from four to eight people formed around the “problem statements” that attendees had interest in exploring further. For two hours, they talked through potential solutions, continuing to use the design thinking workbook as a guide.
“The Code for Boulder Civic Tech Forum was excellent because we were encouraged to stay in the place of inquiry, rather than racing superficially to find the solution. We worked in small teams with other community members who care deeply about an issue. I am focused on senior housing issues and am thrilled that two very concrete and needed tech solutions arose from our discussions today.” Neshama Abraham, participant
To wrap up the day, each team shared their ideas for solutions with the whole group.
Then participants were encouraged to rate the ideas and solutions that were generated. Each person was encouraged to place two dots next to the idea(s) they believed were most important. Red dots indicated “hot” ideas and blue dots indicated what may be a “quick win.”
The top vote-earning idea was to create a two-way dialogue with citizens, where they could click an online map of current development projects, learn about those of interest, and leave comments that would become part of the public record related to the location. What came to be known as “idea #1” received 12 votes total: 7 blue and 5 red.Ideas Into Taking Action
The week following the event, both the City Council and Housing Committee received briefings from Becky Boone, and both groups asked to stay informed of how these ideas progressed within the Code for Boulder Brigade.
Brigade members decided at their subsequent meeting to work toward making idea #1 a reality. Because of the bonds formed during the CodeAcross event, there is now open dialogue between the city and this all-volunteer civic tech group to access data sets that have previously not been available to the public or not in a format usable by the latest technology. City employees are attending the Brigade meetups and listening to the requests of how to improve what they offer to residents.
How did the event live up to its goal to identify ways to improve and increase civic engagement among community members and with local government? We’re proud of our “wins”:
- Over 50 community members worked together for a day of active listening, ideation, and problem-solving. They experienced the design thinking methodology, starting with empathy listening to craft problem statements, before jumping to possible solutions: an approach that they can use in other contexts.
- Three city leaders and one City Council member joined us throughout the day. They heard firsthand from residents and participated in a collaborative problem-solving process.
- Three city employees have gotten involved in the Brigade’s new initiative that came out of CodeAcross. The result is not only increased dialogue, but the opening of additional data sets and ongoing project collaboration between civic tech volunteers and local government.
- The Brigade now has a solid queue of potential projects from other ideas the came out the event, all of which were generated by community members.
Watch a 90-second overview of the day, produced by the City of Boulder’s Channel 8.
Rally’s partnership with Code for America’s Brigade program will continue to bring opportunities to apply approaches like design thinking, Lean Startup, and Agile team processes to the civic tech movement. Our next event? We’re bringing civic tech to RallyON 2015 in Phoenix, June 15-17. Stay tuned for details in the coming months.Geri Mitchell-Brown
While there are many books and much research on organizational development, this system view combined with some validated learning over time is a powerful way to look at organizational challenges as a coach/consultant.
Let’s take a closer look to define these areas then apply some validated learning from my own experience.
Business Outcomes – the outcomes desired from the business strategy selected
Org Structure – the structure of power and authority to facilitate decision making
Incentive Systems – rewards for individual and group performance
Work Systems – how people get work done in the organization
Collaboration Systems – systems to overcome the friction to collaboration introduced by the org structure
People Systems – hiring, firing, development, HR systems – both tactical and strategicValidated Learning (observations and experiences over time)
- Business outcomes are required to think about the other dimensions; and interestingly, in my experience even some top leaders can struggle to articulate these, so it may require some elicitation and dialogue. I like to use the pithy term “operationalize strategy” when discussing this topic.
- Incentive systems usually mirror org structure fairly closely.
- The org structure will help determine both work systems and collaboration systems; however, collaboration systems have a stronger relationship because they must overcome the friction introduced by the structure itself.
- Incentive systems and people systems strongly impact everything else except strategy.
- People tend to focus first on org structure and work systems because they are the most visible, tangible, and even “fun” to work with.
- Each organization design decision made will impact the other dimensions so as the design is created, the entire system must be reevaluated.
- Organizations are typically good at people systems when it comes to tactical training and development, but more powerful levers are hiring, firing, and strategic training needs.
- The most common constraint on change involves incentive systems.
What observations and experiences do you have using a systems perspective to view organizational challenges? Has the use of a systems perspective helped overcome these challenges? Leave your comment below so that we can get the conversation started.
The post Operationalizing Strategy with a Systems Perspective appeared first on LeadingAgile.
Choosing the right agile metric to measure agile success is really simple, right? I wish that were the case, but in reality choosing the correct agile metric can be a little tricky.
So, how do you get the most out of your agile metrics? I reviewed the 9th annual State of Agile survey, which compiles insights from nearly 4,000 respondents, to find out how agile practitioners are measuring the success of their agile initiatives.
#1 On-Time Delivery
According to the State of Agile survey, 58% of the respondents* said they measured the success of their agile initiatives by on-time delivery.
With agile, our schedule is fixed and our scope is flexed. What does that mean for on-time? Well, time just happens, so theoretically, we are always on time. But, on-time is generally measured in context with the expectations about what will be delivered. To measure and have visibility of what is being delivered, we may look to the out-of-the box metrics of the burndown or the burnup.
For instance, in this VersionOne burndown chart you can see progress as the team heads toward an expected end date.
This burnup chart, on the other hand, allows you to see the trend of getting stuff done, as well as the impact of scope changes.
#2 Product Quality
A total of 48% of the respondents to the survey said they measured the success of their agile initiatives through product quality.
Quality is often measured in multiple ways, including looking at the customer satisfaction, revenue growth, and the technical aspects of testing conducted throughout the development life cycle. With agile software development teams, we’ll look at our velocity of completing working software with quality built in. We tightly couple continuous testing and inspection throughout the lifecycle of the development, so we’ll constantly be monitoring testing trends as well as constantly inspecting build and code health.
For instance, in this testing trend chart you can see the cumulative progress around testing activities. Ultimately you want to see all green, but a large amount of red along the line might reflect some issues in the code base or process.
#3 Customer/User Satisfaction
The survey found that 44% of respondents measured the success of their agile initiatives by customer or user satisfaction.
As with all these benefits, there are multiple ways to measure the outcomes. In the case of customer/user satisfaction, these include looking at the Net Promoter score, sales figures, number of support calls vs. number of features delivered in a time period, or usage statistics of product or site capabilities.
#4 Business Value
Approximately 44% of the respondents to the State of Agile survey stated that they measured the success of their agile initiatives by business value.
And several of the principles of the Agile Manifesto recognize the importance of delivering business value. Measuring business value is very explicit when we know that there’s a contract for work to complete or a compliance need and fines if we don’t finish the work. On the other hand, sometimes measuring value is prospective or speculative in the sense that the market inputs drive decisions and the value is often a best guess. Having a business value score applied to the features to be delivered can measure value.
Here’s a sample epic cumulative flow chart based on value. This helps you see the delivery of anticipated business value as features and other large stories are completed.
#5 Product Scope (Features, Requirements)
Another 39% of the respondents answered that they measured the success of their agile initiatives with product scope.
Setting a goal around what to get done over the next three months, then tracking status, and getting it completed is hugely rewarding. Actually having real-time feedback as to the progress of work is valuable to everyone on the team, from the engineers to the program managers. With agile software development projects, you can always rely on the burndown charts, or just visualize the progress of the cards moving from left-to-right on the project kanban board.
Here’s an epicboard in VersionOne that helps the team track and visualize the progress of features at a program level. Or if you’re using Scaled Agile Framework® (SAFe®), you can see progress at the release train level.
#6 Project Visibility
Project visibility was the measure of choice for 30% of respondents to the survey.
One of the best ways to build trust is transparency. That means having the plans out in the open and making progress visible to all. Sharing progress at multiple dimensions provides the different stakeholders with information that makes sense from their point of view. Metrics that show feature or overall progress against a targeted plan can provide great insights.
In this chart you can visualize progress on a feature and known work. The red diamond represents what was actually anticipated, so it’s easy to gauge whether you are above or under the anticipated scope.
The other reason visibility is important is because we need to have alignment among internal teams so they can best manage their work in relation to component or service dependencies.
Understanding the impact of one team’s work on another team is critical. By looking at the dependency chart below, it’s easy to identify the stories at risk.
According to the State of Agile survey, 29% of the respondents said they measured the success of their agile initiatives through productivity.
The concept of productivity in an agile world is a measure of outcomes, not output. So looking at burnup for a product or based on value is hugely impactful. Simply looking at a burnup of count of stories or features over time is a great way to understand how much the team is actually delivering.
Approximately 25% of the respondents from the survey said they measured the success of their agile initiatives by predictability.
A predominant metric used to assess predictability is velocity trend. For a three- to four-month period, this shows how much work has been completed at a sustainable pace on average. A velocity that wildly fluctuates might reflect a team that is changing, work that is unpredictable, or simply a team that is still getting used to defining work small enough to complete in an iteration.
A velocity trend chart like the one below not only helps you see performance, but also gives you visibility into whether or not the team’s output is at a predictable state – as this one shows.
You can always try to assess velocity based simply on the count of story cards completed every week. This is usually the best indicator of predictability.
#9 Process Improvement
Another 23% of the respondents said they measured the success of their agile initiatives by process improvement.
A core tenet of all lean and agile mindsets is continuous improvement – constantly getting better. But how do you know you are getting better unless you are measuring the outcomes? There are all the metrics above that help, but there’s also the extremely helpful cumulative flow chart which shows how well work is flowing through the lifecycle.
With this team level cumulative flow chart, you can see where bottlenecks or slowdowns may exist.
Also, there’s cycle time – which helps us with planning and predictability. Cycle time is a great metric to view over time to see if process tweaks and adjustments are having an impact on productivity.
For instance, in this cycle time report, you can see the level of variability and performance across the various estimated pieces of work.
#10 Don’t Know
Just 11% of the State of Agile survey respondents said they didn’t know! Well, if you don’t know the benefits, try to start looking at the metrics above. You’ll see improvements in delivered value, better quality around what is produced, a more predictable cadence, and ultimately happier customers.
These results show that there isn’t just a single metric that everyone uses. Different organizations, types of management, and teams need different metrics.
Not sure which metric is right for you? Check out the 9th annual State of Agile survey to learn more about what nearly 4,000 of your peers are doing.
What metric do you use to measure your agile initiatives’ success?
*Respondents were able to make multiple selections.
State of Agile is a trademark of VersionOne, Inc.
In many of my transitioning to agile clients, the managers want to know when the project will be done. Or, they want to know how much the project will cost. (I have a new book about this, Predicting the Unpredictable: Pragmatic Approaches to Estimating Cost or Schedule.)
Managers ask for estimates because they want to know something about their ability to recognize revenue this year. How many projects can they release? What is the projected effect on revenue; customer acquisition and retention; and on service revenue (training, support, all that kind of service). We pay managers big bucks so they can project out for “a while” and plan for the business.
You need to know this in your life, too. If you are an employee, you know approximately how much money you will make in a year. You might make more if you get a bonus. You might make less if you get laid off. But, you have an idea, which allows you to budget for mortgages, vacations, and kid’s braces.
Remember, in waterfall, there was no benefit until the end of the project. You couldn’t realize any benefit from a project until it was complete: not revenue, not capitalization, not any effect on what customers saw. Nothing.
When you use agile, you have options if you can release early. Remember the potential for release frequency?
If you are agile, you don’t need to estimate a lot to tell them when they can first receive value from your work. You can capitalize software early. Your customers can see the benefits early. You might be able to acquire more customers early.
Agile changes the benefits equation for projects.
Agile is about the ability to change. We see this at the team level clearly. When the team finishes a feature, the team is ready for the next feature. It doesn’t matter if you work in flow or timeboxes, you can change the features either for the next feature (flow) or at the next timebox. You can change what the team does.
Agile is most successful when teams finish features every day (or more often). The faster you finish a feature, the faster the team gets feedback on the feature. The more flexibility the product owners has to update/change the backlog for the team (either for flow or the next timebox). The teams do have to complete their work on a feature in a reasonable amount of time. If your cycle time gets too high, no one sees the flow of features. If you don’t get to done at the end of the iteration, you don’t get the benefit of agile. Teams need to learn how to get to done quickly on small features, so they can demo and get feedback on their progress.
What does this fast delivery/fast feedback cycle do for senior managers?
It allows senior managers to change their questions. Instead of “When will this be done?” or “How much will it cost?” senior managers can ask, “When will I see the first bit of value? Can we turn that value into revenue? When can we capitalize the work?”
Those questions change the way teams and senior management work together.
When teams do agile/lean, and they have a constant flow of features, managers don’t need “assurances” or “commitments to estimates” from the teams. Instead, the team estimates a much smaller chunk of work–time to first delivery value.
You might not know precisely when you can deliver that first value. But, as soon as the team works together if they understand agile/lean, they can create a reasonable estimate. They can update that estimate if necessary.
What else can teams do?
- Work to a target. If the teams and the product owners know that management has a desired release date, they can work to it. Teams can track their feature flow through their systems, understanding their cycle time. They can use timeboxes for focus. They can measure how close to done they are with a product backlog burnup chart.
- Demo as you proceed. Always demo to the product owners. Depending on the pressure for revenue/whatever, ask the senior managers to participate in the demo. That way, they can see the product’s progress as you add more features.
- Keep the backlog item size small. It doesn’t matter how much is in the backlog if the size of every item is small. The smaller the backlog items, the easier it is for teams to estimate. It’s also easier for teams to maintain a flow of features into the ever-evolving system. Who knows? You might be done earlier than you expect.
With agile, you don’t have to set the strategy for a year, fund the projects, and expect that the projects will complete within that year. A year is a long time in almost every market. Managers might want the ability to change their strategy, and still get to a first “delivery of value” date.
Our metrics need to change. Cost to release or time to release is inadequate. They are inadequate because we can change the backlog at any time.
Instead, consider these metrics:
- Time to release value: How long will it take us to release something for revenue? (The smaller the number, the better.)
- Frequency of release: How often can we release? (The higher the frequency, the better.)
- Run rate (What the team costs per unit time)
- When you capitalize software. I will admit too much ignorance here to provide you guidance.
I have other measurement suggestions for programs in Organizing An Agile Program, Part 5: Measurements That Might Mean Something to a Program.
It’s not about #noestimates. It’s about which estimates your managers need. Managers have a fiduciary responsibility to the organization. You have the responsibility to release often, at least internally. The more often you release, the fewer time/cost estimates your managers need. The more your managers can take advantage of capitalizing software and what the software can do for the organization and the customers.
Your managers need estimates. And, they need to change the estimates they request. It’s all about your organization’s transition to agile.
The world’s moving faster than ever, and as a partner to some of the most innovative companies in the world, Rally is at the forefront of change. We bring our world-class SaaS platform and the most experienced coaches and consultants in the industry to help customers navigate today’s challenges — including nimble competitors, changing markets, increasing regulation and faster cycles of innovation. And like you, we too must adapt to change.
Rally has been a pioneer in Agile approaches, with Agile development software that revolutionizes the way companies across the globe accelerate software delivery. We have delivered countless scaled Agile solutions and led large-scale transformations across thousands of people and hundreds of teams, helping our customers to align software development with strategic business objectives and leverage agility across their organizations. Building agility is mission-critical in organizations today. It’s improving the way we work, learn, respond to change, innovate and win in the market.
Just as Rally has evolved, our brand must evolve to better reflect and communicate our expertise in helping organizations worldwide succeed on their Agile journeys. As a result, you may notice that Rally looks a little bit different. We’re introducing a new logo that modernizes our look while staying connected to our Agile roots.
We have a new tagline, Unleash What’s Next, designed to better communicate what Rally software and services help customers achieve.
You’ll also see our name as Rally instead of Rally Software, because we are more than a software company: we’re a partner to our customers, helping them succeed on their path to agility.
Learn more about our refreshed brand, and how we work with our customers to help make them more successful, efficient, and productive, in this short video.
Our customers, partners, and employees have played a key role in shaping who Rally is today, and for that I thank all of you. Our employees continue to be the heart behind everything Rally does — I’m grateful to you, the Rally team, for taking this journey.
We’re excited about the future and look forward to continuing to innovate with you to step beyond the possibilities of today.Tim Miller
A new report by the help desk research and analysis consultancy Software Advice shows that a majority of companies are looking to improve or expand their help desk offerings. Among companies who plan to change their spending, 84% stated their intention to increase investment in help desk software. The majority cited their need to add new features to keep up with the increasing service expectations of customers, as the most common reason for increasing their investment.
This increase in investment is further fueled by a desire to improve the customer experience in order to stay ahead of the competition; even more so than actual product innovation. This is a big plus for customers! It’s not just a push to increase automated processes or replace humans with software to cut costs and increase margins, but a conscious effort to improve customer satisfaction!
Axosoft has certainly got on board with making our customers the focus of our business decisions. Even our product focuses on servicing our customers’ customers with features like the Customer Portal (now that’s meta). Axosoft customers also have the ability to integrate their help desk features with their development tools for project management, bug tracking, and documentation. Not to mention they can create even more robust solutions by integrating Axosoft with other well established CRM tools like Salesforce and Zendesk.
Software Advice reports that the most frequently used help desk functionalities are ticket management, reporting/analytics tools and live chat integration. These are trends we’ve picked up on and built into Axosoft. We’ve also seen that as more and more customers are preferring instant chat support, integrating Axosoft with a live chat solution like Pure Chat helps provide a superior customer experience. Our Customer Success and Sales teams personally use this live chat solution along with Axosoft to better serve our customers.
Our goals at Axosoft are broad: improving our overall customer experience. In order to evaluate the best help desk solutions for your company, it’s important to identify your business goals and how you can measure the impact of a system. You should also consider your own scope of use, integration requirements, and preferred deployment model (cloud or on-premise). Take a look at Software Advice’s 2015 Help Desk Report for more data that may help you make an informed purchasing decision.
What solutions are you investing in for 2015 and beyond? Let us know in the comments section below!
Risk is a big topic, but generally for beginning agile teams (or any team) it can be calculated in a minimalist way as dependencies. To me, dependencies represent a binary probability. Either they are resolved or not.
When paired with value, risk becomes super important to look at because it can tell you how much of that value is in danger. Specifically, we can use it to make informed decisions about investment.Regarding value
For now, I will take a look at value through the lens of Net Present Value (NPV) because the program I am currently coaching is already using it. I am not advocating for NPV, just meeting the program where they currently stand. Boiling it way down… Net Present Value is used in capital budgeting to represent the profitability of an investment over time. In agile, this might be an investment in an epic or a feature or a capability, etc. You also might use story points as a super simple starting place until you figure out what value means in your organization. Net Present Value and a few other valuation methods can be seen here.
Risk-adjusted Net Present Value (rNPV) has been used in pharmaceuticals to determine if a capital investment is worth pursuing. The basic reason for the use of this method is that new medications are expensive… Expensive expensive. Like $2.5 billion expensive to get one for which the market approves. There is a high probability that drug trials won’t work out and the drug won’t make it to market on time or at all. For that reason, risk of failure is assessed and applied to the valuation. That can lead to investment decisions in large capital projects.How About Doing That In The Small?
With proper breakdown of strategy, I encourage organizations to do some form of analysis for the next three months at a tactical level to make sure that they inform themselves of the level of risk so they can determine the likelihood that they will realize a return on their investment. The risk exposure is smaller than NPV is usually concerned with and we may be willing to take on a larger risk profile, but the problem is the same. If you have a relatively significant investment, you want a reasonable shot at realizing your return.Looping in Agile
Take a look at programs and portfolios. Sure there are plenty of risks that can prevent projects or features from succeeding, but to keep it simple, I am focusing on the binary dependency. It is done or not done. Therefore, the probability of the risk being removed is 50%. Continuing the math for a bit, for each dependency on the outcome we target, we have a factorial of possibilities… or in English:
If we have two dependencies, we can have four possible outcomes.
- Resolved | Resolved
- Resolved | Failed
- Failed | Resolved
- Failed | Failed
That makes the probability 25% that we will have an outcome of Resolved | Resolved.
If we have three dependencies you can have 8 outcomes and thus a 12.5% probability and so forth and so on. So now forget about NPV. We don’t care. We need a way to make good investment decisions for things that will actually come true. We also need this method to be lightweight.
Given 100 points are in a release and we have 2 dependencies that need to happen in order for us to release the features. We have a 25% probability that we will release 100 points. Your risk to point value that you can count on is 25 points assuming that all points are blocked by those dependencies. Granted, that’s not a value assessment, it’s a point assessment. But that’s an ok place to start.So Now What?
With this new information, there is a definitive incentive to remove risk early. Really early in the release. As a matter of fact, I want relatively new teams that are just starting out to plan a release that has an 80% certainty of delivering it’s value. That means that before the release, we need to remove most of the dependencies. I have been toying with capturing this as a risk burndown. Here’s a quick wireframe of it.
In this figure, we are burning down the percent of the release that is impacted by risk. Like I said, I like relatively new programs that hit 80% of their value delivery. We can use the percent impacted to drive risk points down to below 20% before the start. It’s a pretty simple, good way to look at the probability that programs and portfolios are making responsible commitments and have the math to back it up. Speaking of math… this stuff can get a ton more complex. A great goal is to keep it simple so we can all relate. I know this is a little off math wise, but it’s enough to inform and get people started toward developing reliable release plans and roadmaps that mean something.
With the constant pressure in the non-profit sector to do more and create more impact, while at the same time continuing to be good steward of resources, it’s become imperative that nonprofits develop a business sense that isn’t typically associated with our sector. Working with Rally has given us a huge advantage in developing team processes and skillsets. Rocky Mountain MicroFinance Institute (RMMFI) isn’t just fulfilling a great mission; we’re well on our way to becoming an efficient and productive agile team, capable of delivering outstanding results.
"We strongly believe that being a nonprofit is no excuse for not operating based on solid business principles. Adopting ‘the agile way’ has reshaped and strengthened the manner in which we live this organizational value."
RMMFI has always taken a non-traditional approach to the business planning process in our work to help low-income and otherwise marginalized individuals launch and grow micro-enterprises. Day in and day out, we preach simplicity, focus, and the experiential learning process to our aspiring entrepreneurs. But sometimes—especially in times of high growth—we have trouble swallowing our own medicine. Through RMMFI’s partnership with Rally, we’ve had the opportunity to learn from a number of experts and gain exposure to valuable agile principles that have, in many ways, come to define our approach to work planning and project management.
One of the things we’re most excited about is our shift toward a more intentional, “agile-inspired” organizational planning process. In 2015, RMMFI changed the way we defined our annual plan to be more focused on the priorities of the organization while being realistic about our capacity to get the work done. Our goal is to increase the visibility of the work being done across the organization, increase collaboration among team members, and put a system in place to continually reassess the workload for changes in priority and scope. This shift has or will affect all phases of planning across the organization, from the printed Annual Plan we share with stakeholders to the individual sticky notes hanging on a team member’s Kanban board.
In recent weeks, RMMFI has been working with our coach, Rachel Weston Rowell, to develop the project planning abilities of the team. In a matter of weeks, RMMFI will begin implementing a sprint planning method with the hopes of making the workload more approachable, making deliverables more valuable, and making our staff more organized and happy. In addition, Rachel led us through a work planning exercise to determine problem areas in our client recruitment process. This exercise helped the team identify blockages in our “value stream” (pictured below) and helped us define improvement projects that would allow us to get to better results—such as more client applications in the door with the same or less effort/strain on the part of staff.
“Working with RMMFI has been an incredibly rewarding experience. Being able to take the agile and lean concepts that we leverage both inside Rally and in our work with our customers, and bring them into the nonprofit space means we get to impact an even larger community. I’ve seen RMMFI take huge steps in transforming their processes and practices, how they collaborate, and how they steer their business. Nonprofits are hungry for the same things that for-profit business are – to be more effective at what they do and how they engage, all in service to their mission.”
In the last three years, Rally’s experts have helped us develop time management trainings for our clients, facilitated strategic planning sessions for our Board of Directors, and assisted staff with the development of process improvement activities that are now in regular use. Additionally, through a grant funded by the Rally For Impact Foundation, our leadership team has taken the Leading Collaborative Meetings course from Agile University. By taking that class we better understand how to work collaboratively across teams and projects as our organization grows. Nearly every interaction with the Rally team becomes a learning opportunity in facilitation methods—a skillset that’s very valuable in the delivery of our core client-facing work.
Part of RMMFI’s mission is to create for our clients a network of expertise, and provide them with access to business knowledge that doesn’t naturally exist in their world. In many ways, Rally has played that exact same role for RMMFI. Instead of being stuck at a certain level of impact or performance, we’re now able to use some of these practices to do more and create more: maximizing our time and resources on what matters most. The better we become at actually doing the work, the more opportunity there is for low-income or otherwise marginalized individuals to turn their ideas into a business that works—a business that has the potential to transform a household, a neighborhood, and ultimately, an entire community.
Graduates of RMMFI Class #11, who all got a little dose of agile planning as part of our Boot Camp curriculumBrendan Landry
We invite you to join us at RallyON 2015, June 15-17 in Phoenix, Arizona.
RallyON brings you the best thinking, strategies, and practices to help you capitalize on the new pace of change. Find out how today’s leaders are adapting to the future of work and building organizations that are fast, lean, and nimble. Come to learn, connect, innovate, and be inspired.
- Join your peers at sessions and workshops to discuss your toughest challenges — and solutions.
- Get your product or IT delivery engine humming at scale through visibility and planning into your process.
- Learn how to create an unstoppable delivery engine, where coordinated teams operate with agility and flow at scale using the coolest new collaboration tools.
- Discover how to surface risks and dependencies, and get the best results, from Agile release planning and quarterly business steering.
- Hear stories from your peers about applying Lean, Agile, and DevOps practices to eliminate silos and extend agility and flow throughout your teams.
Visit the RallyON site to learn more and register!Rally Software
I think I’ve found myself (somewhat accidentally) at the beginning of a series of posts called ‘debates I find useless… let’s move on’. The latest round of discussion that seems to have spiked (at least for me) this week is the whole ‘to estimate or not to estimate’ conversation. The answer to this question clearly falls into the ‘it depends’ category, so if we are having an argument that involves any kind of absolute, we are probably wasting our time.
Even in a domain like commercial, non-governmental, software product development… the one LeadingAgile plays in most of the time… there is seldom any one, single way to do anything. I do believe that in this domain most estimates are functionally useless… but understanding what is estimateable and what isn’t… and more importantly what makes things un-estimateable and why they are un-estimateable… I find to be a way more useful conversation.
If we decide not to estimate, we better have a credible response to the question… when will you be done and what will I get for my money… because asking someone to spend a bucket of cash on the promise they might get something when the bucket runs out… is usually pretty much a non-starter. There are of course exceptions, but for most companies, the answer to this question is pretty important so we’ll need an alternative approach for solving this problem.
Almost always when someone calls my company… they aren’t really looking for advice on how to innovate or how to build the right product, that hasn’t historically been our brand… they are looking for help using agile to make and meet commitments, get product into market earlier, improve quality, or reduce costs. That’s typically been our sweet spot when it comes to introducing agile into large complex organizations and solving it requires more than just better estimating.
What’s the Problem with Estimating?
The companies that call us want to know how much they need to spend to get a particular outcome. They want to be able to make and meet commitments. They want to be able to manage customer expectations.
To have that conversation, you have to first start looking at why organizations aren’t predictable. Most of the time it’s not so much that they can’t estimate, it’s that they have way too many things going on at one time, they have way too many dependencies, and way too many non-instantly available resources. They believe in optimizing for individual production capacity, which causes them to matrix people across multiple initiatives, which just further exacerbates all the aforementioned problems.
Even once you get past the alignment issues, and you reduce much of the waste getting in the way of delivery, quite often companies don’t really know exactly what they want to build. They don’t really know exactly how they are going to build it. They might not even know who exactly is going to do the work (when they need to pull together the estimate) and we all know that we can see wild swings in throughput and productivity between any given developer on the team.
Even if companies can remove the waste, eliminate bottlenecks, and such… even if they know exactly what to build, how they are going to build it, and have a highly consistent stable of software engineers… developers able to deliver against estimates in a reliable way… quite often the code bases they are working in aren’t covered in tests, have a ton of technical debt and defects, and aren’t generally architected in a way that lends itself to stable delivery throughput.
Is Estimating the Right Problem to Solve?
Here is my take… having established our context and domain… I think that asking how to do better estimates is the wrong question to ask. I don’t think we really have an estimating problem as much as we have an organizational alignment problem, we have an investment strategy problem, and we definitely have a risk management problem. As companies, we are placing critical dollars on investments that have a very low probability of paying off… and we are relying on flawed estimates to mitigate that risk.
That is the problem that is killing us.
We want to use estimates to reduce uncertainty and end up increasing uncertainty.
We want to use estimates to reduce risk and end up increasing risk.
We want to believe that with enough up front planning we can know exactly what we need to build and how we are going to build it. That with enough historical information… or enough up-front analysis… we can determine how long the work is going to take. We want to believe that all developers are the same and that every developer can do everything in the estimate at the same rate as any other developer.
In practice, in my experience, this does’t work.
All it does is shift the perceived risk from the business to the development team and everyone loses.
So, What is the Right Problem to Solve?
We have an interesting paradox to deal with here. This is irrational… but make no mistake… this is the current reality in most software businesses today.
We live in a world where requirements are uncertain, technology is rapidly evolving, people are unpredictable… a world where technology is poorly architected, changes result in unintended consequences, and defects are rampant… AND we have to be able to make and meet commitments with some level of assurance that we can actually solve the business problem within the time and cost allocated to the project.
We figure this out or our companies fail.
Any solution that doesn’t answer the questions of when will we be done and what will we get for our money is a non-starter in most organizations.
#noestimates is a non-starter in most companies.
To solve for this irrationality, there is a ton of thinking that has to change, but at the highest level, you have to tackle this problem on two fundamental dimensions…
1) Do everything you can to optimize for throughput and stable delivery capacity, and…
2) Focus on budgeting and constraints rather than estimates.
What does this mean? Let me explain.
Optimize for Throughput and Stable Delivery Capacity
Well… given the uncertainty of requirements, technology, and people… you have to optimize the systems of delivery to eliminate as much of that variability as possible.
To me, that means creating complete cross functional teams, teams aligned toward a single set of products, features, or business capabilities. You have to give those teams as much clarity as possible regarding the problem they are trying to solve. They need to be given the tools necessary, and be held accountable for producing a measurable, working tested increment of product on regular intervals. This allows us a sampling frequency to assess progress.
To me, this is where agile really comes in. Well formed agile teams, working against a known backlog, and having the ability to produce a working tested increment of software at the end of every iteration will begin to stabilize delivery throughput and become predictable delivering on regular intervals. In a really stable, well formed team, it is often possible to estimate the backlog and establish a velocity against the backlog.
But even with well formed agile teams and stable delivery throughput… there is still variability in both the requirements space and the solutions space. In our problem domain, this isn’t going to be solved for easily, so what do you do?
Establish Budgets and Constraints Over Estimates
This is where I think the notion of estimates get’s us in trouble. Often we don’t know exactly what to build, or even how we are going to build it, and believe it or not… in our domain that can be a good thing. If our domain is uncertain, we don’t want to pretend that uncertainty doesn’t exist, or worse, force a level of certainty that isn’t good for our product, or for our company, or that forces us to make early decisions that need to be deferred to when we have more information.
Given though that business won’t allow software production to be totally open ended… what do we do?
We generally recommend that folks look out across their product roadmap, consider where they need to be with their product in the next 6, 9, or 12 months, and do a very high level estimate on what they think it will take to get there. At this level of abstraction, you don’t simply consider what you believe it will cost to do the work, but also what you are willing to invest to get the return on investment you are expecting to get.
At this level of abstraction, it doesn’t have to be exactly right, just directionally correct.
Once you have a high level estimate, stop calling it an estimate. It is a budget. It is a constraint.
As you begin the process of elaborating the requirements, and maturing your emerging understanding of the product you are building, you are no longer asking yourself ‘what are the requirements’ or ‘how how am I going to build this’… you are asking yourself what solution can we develop that can be delivered for the time and cost constraints that the business has asked me to develop within. This is a VERY powerful thinking tool for constraining product development and meeting objectives.
You are adapting the requirements in real time to solve the business problem, and adapting the solution to something that can be built within the constraints that have been established. You invest much of your energy into requirements decomposition, getting the MVP done as early as possible, getting feedback from the delivery teams, aggressively managing the backlog to the constraints, and adjusting to meet business goals. We make tradeoffs at all levels to meet business outcomes.
This is the essence of agile risk management IMO.
Mitigating Risk and Managing Uncertainty
Just to drive a couple of these points home… in the commercial, non-governmental, software product development space… many organizations are not organized well, technology is not architected well, we are making critical high risk investments all the time, competition is fierce and speed to market is essential, and we are dealing with crushing uncertainty in requirements, technology, and people… and given these constraints, sometimes stuff isn’t knowable.
That said, not estimating isn’t an option. Not constraining development isn’t an option. The good news is that in the presence of a stable delivery organization… in the form of complete cross-functional agile teams… teams working against a known backlog and able to produce a working tested increment of product on regular intervals… and in the context of an investment strategy that is based on high-level estimates which quickly become budgets and constraints… we can begin to deliver on plan.
We can now start evaluating progress against our assumptions and actively manage risk to make sure we are optimizing our chances of being successful.
We can begin to enumerate the backlog, making relative size estimates as we go. Since the teams are stable, we can begin to correlate the estimates with what it actually cost to build them. We can use past performance data around the accuracy of the estimates to anticipate the accuracy of the future estimates. The further out we have the backlog, the better we can get at planning forward. Because we know our progress, we can see where we are at against our business objectives.
When things change, we have clear line of sight to our top level business objectives, we can assess how the changes will impact what we are trying to accomplish, and if the emerging size of our backlog is going to impact our ability to meet those objectives within the time and cost constraints that we have established. When we learn that things are getting too big, we can either take something out, agree to spend more and go longer, or we can kill the project.
Sometimes we have bitten off more than we can chew. Sometimes the investment we want to make isn’t really knowable until we get in and start building it. Sometimes we get started, and then we realize we are screwed, and our only viable option is to kill the project, cut our losses, and be thankful that we spent as little money as possible to figure out we were running a fools errand. Nothing (in any of this) guarantees success.
We just need to know when to get out as early as possible.
Separating Knowable Stuff and Unknowable Stuff
I think that much of the stuff that we think is unknowable is often more knowable than we think. Just because we don’t know exactly what to build, doesn’t mean we know nothing about what to build. Just because the technology is a mess, doesn’t mean that it’s such a mess that we can’t work with it at some level of abstraction. Just because developer throughput varies, doesn’t mean that we can’t make some planning level assumptions that are reasonably accurate.
That said, I have worked with teams that are inventing new mathematical algorithms for solving problems that have never been solved before. I’ve had conversations with developers where the estimate could be anywhere between 2 hours and 2 months… and maybe even the problem is unsolvable with the team we have available to solve it. Honestly, the team legitimately doesn’t know, and no amount of analysis is going to help them know. They just have to try.
All I want to do is acknowledge these kinds of problems exist, but they aren’t every problem. The approach I’ve taken for problems like these is to isolate them from the stuff that we are better able to understand and deliver. Now… we have a business decision to make… does the product have value even if the problem never gets solved? Is it still worth investing in? Is it worth spending money given the risk that this aspect of the solution could never be solved? Can I afford to assume this business risk?
A big part of our challenge is that we are placing big bets on unknowable things, we are making hard commitments on things that aren’t sufficiently understood. We have to get better at isolating R&D from the stuff that is more well understood product development. If we are going to invest in R&D, the payoff better be big enough that the risk of failure is worth it. We can’t pretend the uncertainty isn’t there and we can’t estimate that uncertainty away.
To have a rational conversation about what to estimate or how to estimate, we have to look at the nature of the problem we are trying to solve, the nature of the organization that is chartered to solve it, the investment and risk profile of what we are trying to build. We need to assess our tolerances for uncertainty. We don’t want to pretend we can estimate things we can’t estimate, but we also want to recognize that some things under some conditions can be estimated.
My experience is that there is a ton which we can do with organizational alignment, program and portfolio management, investment rationalization, risk management, budgeting and constraints, limiting WIP and focusing on flow, that can dramatically help companies get better at making and meeting commitments… and yes, even better at estimating.
Likewise, I think there are problems that don’t lend themselves to estimating at all, where the real problem isn’t estimating or not estimating, but the allocation of essential dollars to high risk endeavors… which are masquerading as software product development… but profile more like experimental R&D.
You just have to know what business you’re in, build your organization to accommodate those realities, estimate and control what can be estimated and controlled, and stop demanding those things which can’t be estimated or controlled conform to your sense of reality… unless you are really to invest in more advanced project modeling than most organizations I’m aware of have the capability to perform.
NOTE: Just realized this is the 600th post on LeadingAgile. That’s kind of a cool milestone. Thanks for being around and keeping up ;-)
The post Estimates or #noestimates… It’s All a Matter of Context appeared first on LeadingAgile.
Bobtuse can be wildly informative