Measuring Predictability and Agility during PI Execution

If you want to reach higher levels of Agile maturity, it is important to identify and address the underlying issues that may be hindering your progress. Applying objective value-based metrics, Flow Load and Flow Predictability, enables continuous improvement every quarter.

Using Flow Metrics across the organisation has become nearly synonymous with Agile Ways of Working and SAFe. Provided with good aggregate data from across teams showing Program or Portfolio progress, business owners, Lean-Agile Leaders, Product Manager/Owners, System Architects and Release Train Engineers can have wonderfully compelling conversations.


Tips to Fostering Continuous Improvement

1. Commit to what’s realistic

How much work is currently in process? What is the historical capacity of the Value Stream (Agile Release Train)? What is your current capacity allocation to various types of work in the system?

“Stop starting, and start finishing”

Use a Flow Distribution metric to determine optimal attention on what needs to be done. The health of the value stream requires a balance of key work item types: new features, defects, risks and debts.  Adjust the capacity allocation policy of these four value stream ‘Flow items’ to reflect a percentage of effort-time to each.  Finding the sweet spot takes analysis and experimentation. 



2: Decide where to distribute work in the next System Increment (PI)

Program predictability is boat-anchored by defects. Defects drag capacity away from new feature development.  Too often, focusing on new features are done at the detriment of reducing technical debts.

To reduce defects in the long term, ensure a consistent percentage of your Flow Distribution addresses tech debt and maintenance.   If the defect rate in the metric climbs, increase the distribution of debt payoffs.



3: Seek out Neglected WIP

The program seems to be in a reactionary state; only the most urgent work gets attention.  This is typically a symptom of too much work in process.  Perhaps due to items being injected into the Sprint or the PI without due prioritisation.  

Not treating this situation will quickly erode confidence by business owners because the program is not able to complete work as predicted.

Ask “What is the percentage of work that exceeds the capacity of the ART?”  Flow Load provides the appropriate visual metric needed to watch for neglected WIP in the system.



4: Find System Bottlenecks

Value Streams in a Lean-Agile context are a ‘pull-system’ As a result bottlenecks occur just prior to or in the steps immediately following the buildup.  For instance work is being injected unplanned into the program or team backlogs.  Or items are slow to exit the backlog. This could mean Acceptance in one form or another is blocked.

  • Study the ART’s Flow Time or Flow Load in context with Flow Distribution. Are they steadily increasing at one or more steps?
  • Experiment by reducing new WIP limits or scope reduction before anything else.



5: Focus the ART Sync for resolving dependencies

Whatever your cadence is for Product Management/Product Owners and Scrum Masters (ART Sync), ensure the agenda includes time for resolution of dependencies.  Let a visual display of the ROAM and Program Boards fuel the sense of urgency.

Teams on the train can develop conflicting priorities during PI execution.  Use the Flow Metrics in the tips above to enable discussion on objective data, refocusing everyone about shared priorities of the value stream.

Here is a scenario framework to focus ART Sync (PO sync & Scrum of Scrums) in the span of 1 Sprint:

Week 1

  • Purpose: 
    • to get visibility into how well the Agile Program (ART) is progressing toward meeting its quarterly (PI Objectives)
    • To assess any scope adjustment
    • To create strategy draft plan for the upcoming Sprint
  • Inspect
    • Product status checkout issues tracking tool dashboard with PI Objective progress and CI/CD test results
  • Adapt
    • ART Goal for next Sprint, Feature Backlog, Sprint Goals draft plan


Between Weeks 1 & 2

  • Dependencies and new blockers discovered from Week 1 are worked out
  • Enablers are planned to resolve blockers
  • Backlog Refinements at the team level include time to analyse relevance and feasibility of Sprint Goals. And then lock them in.


Week 2

  • Purpose
    • To prepare strategy and PI Objective for upcoming PI
    • To validate strategy draft plan from Week 1 PO Sync
  • Inspect
    • Product status checkout issues tracking tool dashboard with PI Objective progress and CI/CD test results
    • Team Sprint Goals forecasted during Week 1 PO Sync
  • Adapt
    • PI Objective and Feature Backlog forecast


6: Retrospect for predictability and agility

At the end of the quarter empower the program’s Retrospective with value-based metrics representing the ART’s predictability as a team of teams.  

  • Ensure both quantitative and qualitative measurements are part of the Retrospective. These are indicative of the value delivered.  
  • Follow up the Retro with a facilitated problem-solving workshop that includes a 5 Why’s structure.
  • To stimulate innovative thinking, try introducing false constraints when the program has decided on the actual problem to be solved.  For example:

“If you could resolve this in an email…”

“If you could stop development for a month to resolve this….”

“If you could resolve this in 1 day…”.

I have used all three of these in a single session to great success.

Leverage these tips throughout your Program Increment, or planned quarterly work cycle.  Relentless improvement is driven by objective metrics, retrospection, and discipline practice when scaling Agility.