Early Warning Signs That Your Agile Metrics Are Not Set Right!

Setting up the metrics that help the Scrum Team can indeed be a very difficult job and, in this article, I want to write about some smells that you may observe. These smells are an early warning system or indications which tell you that your metric friend work needs some tweaking.

• Metrics That Take Time To Prepare If a Scrum Team needs time to prepare a set of metrics that usually is a Red flag indicating that the metrics are not coming from one place and need some collation effort. Such a collation effort does not add any specific value and should be discouraged. Also there is always and risk of such manually calculated metrics maybe based on somebody’s perception rather than an universally accepted single source of truth. Where possible it is a better idea to automate the Metrics calculation that way the team’s effort is not vested in calculating the metric.

• Teams Seem Hassled About Metrics / Teams cannot explain rationale Metrics are meant to help the Scrum Teams continually improve their performance. This purpose cannot be achieved if the teams are hassled about the additional burden metrics collection and tracking puts on them. If the Scrum Team is unhappy about some Metrics that they have to track or capture, it usually indicates that either the team does not understand why they are being asked to capture certain data or how the data is going to be used. In such a case being transparent with the Scrum Team about the rationale behind asking for this data helps the team to align better with the ask. Where possible enabling automated data capture also reduces the overhead the Scrum Team has to bear and that can further reduce their resistance.

• Metrics That Are Judgy /Random R-Y-G windows Purpose of metrics should be to help the script to continuously improve their performance. Often the Metrics are structured not as a way to improve the performance continually but to create arbitrary service level agreements which are tried to agile contacts. Many times such metrics have been created with artificial designed acceptable tolerance limits or random Red-Amber-Green ranges rather than tolerance limits which Scrum Teams can understand. In such a scenario Scrum Team might focus on representing data in such a way that their own performance falls into the desirable limit. This might defeat the very purpose of metrics which is to find out ways to improve not to judge a Scrum Teams’ performance.

• Metrics that create barriers Purpose of metrics is to enable the Scrum Team to inspect and adapt in such a way to continually improve its performance. This becomes difficult to achieve when there are Metrics which set team members against each other. Most efficient tester/ the developer with zero defects against his or her name/ star of the month may be some examples of such individual metrics. In such cases it is possible that such metrics create a hindrance for the Scrum Team to work as a single unit and encourage the team members to work as independent players since they want to protect their own metric level. We have to take similar care while we try to pitch 1 Scrum Team against another especially when these teams are working on the same product goal and the product backlog. Such Metrics that compare one team against the another may discourage collaboration among the teams which in the end might hamper the value delivered to the customer

The post Early Warning Signs That Your Agile Metrics Are Not Set Right! appeared first on World Of Agile.

 Figure 1 – Defining Your Metrics Framework

Setting up the metrics that helps Scrum Team can indeed be a very difficult job and, in this article, I will explore some things to keep in mind while you are setting up your Metrics

• Metrics Are Not About Keeping Things Green – In my experience I have seen quite a lot of Scrum Teams where the focus of metrics reporting is to show how what we are doing is working or in other words we are all green- this usually stems from metrics that are defined in agile contracts and are tied with payment for services rendered. This makes the Scrum Team focus on ensuring that there always seen green. In such a case the focus cannot be on what we need to do in order to improve. I usually guide the teams not to have these service level agreements in their Agile Contracts – I rather focus on having metrics that are measure the right things like value of work delivered. 

• Focus on End-to-End Metrics – It is usually a better idea to have metrics which measure the value end to end across the value chain rather than measuring value at interim points. To give an example, it is better to have metrics that capture into and time required for a feature rather than tracking time for a specific task like coding or testing. Another example can be, number of zero defects features delivered is a better Metrics than number of defects identified during testing phase. When defined in this manners Metrics encourage Scrum Team to work as a unit. It becomes a challenge to define such metrics in a multi-vendor scenario. 

• Measure Only What You Want to Improve – Metrics Should NOT be cast in stone – Often, I see very complicated and detailed metric structures that texts up a lot of time and energy for the Scrum Team to create and maintain the data. These metrics frameworks often are collecting data for it sake without a concrete plan of how the data will be utilised in order to help the Scrum Team to be more effective and efficient. Such metric frameworks make it difficult for the Scrum Teams to derive meaningful in inferences out of huge amount of data being collected. I have seen that it instead works better to have a smaller number of metrics that the Scrum Team has to track. This way the team can actually use the data being captured. Some of the most effective organisation and Scrum Teams are known to choose their focus for the upcoming period of 6 months to an year and then decide the metrics that will help them achieve their goals for the said period. This also aligns with the Scrum Value of Focus

• Avoid metrics that Compare one team or one person against another – Agile believes into collaboration. For a Scrum Team working collaboratively is the best way to ensure frequent delivery of valuable software. If the metrics are structured to measure an individual performance or productivity it may sometimes lead to people working against each other to protect their own Metrics level. It is a better idea to measure value delivered as the whole team. The same principal can be applied to creating a competition between the teams- for large products requiring many teams it can hamper the collaboration amongst Scrum Teams. It works better if the teams are measured against their own last performance and encourage to have a continuous improvement in their way of working rather than comparing against other teams.

• Trends Need To Be Monitored – NOT individual numbers – Sometimes a particular metric may show a bad value. This can happen because of multiple transient reasons and it may not be worth it to really worry about an isolated value or reading in a metric. It is much more useful to monitor the trends and analyze if the Scrum Team is going in the right direction.

The post Defining Your Metrics Framework appeared first on World Of Agile.

In the article before, we have discussed many metrics that can be used by the Product Owners or the Developers in order to help them generate value at a regular frequency and in a efficient as well as effective manner. However, one accountability that we have not yet discussed is the master of the scrum, our Scrum Master. Does this mean that Scrum Master does not have any metrics ? Or that the Scrum Masters performance can not be quanitified at all?

If that has to be true, then how will the Scrum Masters improve their own way of working and ensure more effective as well as efficient outcomes for the whole Scrum Team? 

In order to decide what metrics, the Scrum Master can track and use we need to consider the key accountabilities of Scrum Master as explained in the Scrum Guide. The Scrum Guide says, master is accountable to ensure that 

• The team continuously improves its effectiveness
• Developers are able to create high value increment that meet definition of done
• And the Scrum Team implements scrum the way it is defined.

Considering the about 3 asks from the Scrum Master, Scrum Master needs to track the trends in other metrics so that the Scrum Master can help Developers or the Product Owners to help improve their own ways of working.

Also, the Scrum Master is a coach for the Scrum Team and at such needs to ensure that the team is self-managed where the team is empowered, enabled and capable of taking decisions independently without excessive support from the Scrum Master. Scrum Master also ensures that the Scrum Team is implementing scrum the way defined in the guide and to do that scram master may track how the various Scrum Events and are the rules stated in the guide are being followed by the team not only in the words but also in the spirit that they are meant.

When considered in this perspective, it is the Scrum Master who is helping the Scrum Team and the rest of the organisation to create a metric framework that can act as a backbone which supports the Scrum Team to continually improve its way of working leading to better value delivered more frequently.

Below I have given some sample Metrics that can help the Scrum Master.

1. Number of times Scrum Master has to be the interface for the developers – with PO /With Organization
2. Team Independence
3. Over All team effectiveness – trends in other metrics
4. Metrics which are found useful by Scrum Team / Metrics which are found cumbersome

The post Agile Metrics and The Scrum Master appeared first on World Of Agile.

Within the Scrum Team the Product Owner and the Developers often have different focus and different perceptions on what do they mean by value or how to improve the effectiveness of their own work and that of the entire Scrum Team.

This means Developers and product on us need different set of metrics in order to be really effective and efficient.  A common anti pattern often seen in Scrum is to have a metrics framework that is generic and not tailored for the individual accountabilities. In my experience it is worthwhile to spend time and energy into deciding specific metrics for both these accountabilities.

Each team needs to define Metrics that help them inspect and adapt. Below I have given some metrics Product Owner as well as Developers can use

Measuring Value –

A Product Owner is tasked with maximizing value of the product and the most important metric from a Product Owners’ perspective will be those which has centred around the net value or the end value delivered to the final customer. Developers. on the other hand will see value of their work reflected into the quality of outcomes that they have achieved.

The table below shows some sample Metrics that can help the Product Owner and Developers measure “value”

Table 1 Scrum Metrics Around Value

Measuring Frequency –

Agile manifesto states the importance of delivering value early and in a continuous or frequent manner. 

To measure the frequency, a Product Owner will like to measure the frequency at which the Scrum Team is able to put the features or the work into hands of end user. However, Developers will want to measure the frequency at which they are able to turn out usable and useful increments. 

The table below shows some sample Metrics that can help the Product Owner and Developers measure “Frequency”

Table 2 Scrum Metrics Around Frequency

Measuring Day to Day Work

The Product Owner plays a critical role in order for the Scrum Team to be efficient and effective. Product Owner has to support the Developers and work as an integral part of the Scrum Team so that the increment created is of best possible value. Developers will want to ensure how they work together collaboratively and support each other while creating the increment. 

The table below shows some sample Metrics that can help the Product Owner and Developers measure “Day to Day effectiveness”

Table 3 Scrum Metrics Around Working Effeicently

Following are some of the articles on Agile Metrics

• Agile Metrics – Friend or foe
• What metrics should we focus on in Agile projects?
• Some sample Agile metrics
• Agile Metrics and Scrum Master
• Defining your own agile metrics framework
• Early warning signals of incorrect choice of metrics for your agile projects
• Velocity

The post Some Sample Agile Metrics appeared first on World Of Agile.

Can we use Traditional metrics as-is in Agile?

Figure 2 Traditional Metrics do not fit in Scrum

Traditional Metrics usually focus on measuring various aspects of planned versus actual. The idea is to track any variances against the detailed plan. They also measure quantum or size  of scope delivery. 

Intention of these metrics is to measure and showcase the efficiency of the delivery be it the schedule or cost adherence, Scope delivered or quality of the delivery.

Some of the usual metrics companies track are

1. Schedule Variance or Cost Variance
2. Size of work packets (KLOC / No of use cases / No of test cases etc)
3. Scope adherence – how we track Scope variance or rework and how we reduce the rework
4. Productivity  

The above metrics are set to track if we are delivering as per agreed plan. – But in Scrum and Agile the plans are changing often as we believe that while planning is indeed valuable – plans need to be updated often. Then these metrics to track against a plan do not seem relevant when plans are being updated very often.

Another key point missing in the above metrics is the focus on value – Agile Manifesto says that as agilists our highest priority is to deliver Valuable Software. The metrics listed above measure only “what work the team did” they do not measure “what Value was generated” – For a Scrum Team to ensure their effectiveness and efficiency we need metrics that can quantify the Value.

Do we need Metrics in Agile Projects?

We saw in earlier section that traditional metrics do not help when working in an Agile manner. The question then comes to mind is, do you really need metrics while working in Agile?

The answer is a definite yes – When done well, Metrics provide us with transparency by giving quantified information that does not depend upon individual perception. This quantified information helps us to inspect the work we are doing and the value we are delivering. The inspection then allows us to adapt our way of working to improve the value we deliver.

In other words, metrics help us to work in an empirical manner.

Figure 3 Why do we need Metrics in Agile?

What should we measure while working in Scrum?

In my experience there are 3 major dimensions where metrics provide us basis or the transparency to inspect and adapt

1. Value – Are we delivering Value? Scrum focuses on Value and our Product Owner is tasked in maximizing value of product. We need to have metrics that quantify and capture the value we have created. 
2. Frequency – Are we delivering value frequently enough so that we help our customers have competitive advantage? – Agile Manifesto tells us that we believe in continuous delivery of valuable products. 
3. Day to Day Metrics – These are the metrics which helps the Scrum Team to inspect and then adapt their way of working so that they become more effective and efficient.

The post Metrics in Agile Projects – What Should We Focus On? appeared first on World Of Agile.

As part of the Introduction to Scaled Agile Framework series, this article with cover the Scaling Conetx and Some Basic Concepts like where scaling fits and how it works.

Let’s first understand what Scaling is?

As a word, Scaling means there are multiple teams working on a Product (Value Stream). 

Scaling Agile refers to the process where the established Agile methods (Scrum/Kanban) are applied to other layers of organization.

Look at the Matrix below to understand the layers of the organization. 

In any Project organization the ratio of Product and Teams is:

• 1 Team working on 1 Product
• 1 Team working on Many Products
• Many Teams working on 1 Product
• Many Teams working on Many Products 

The Matrix above shows the context of Product v/s Teams.

• When we define a Product – here we define an Independent Value Stream which makes sense from a market perspective.
• When we talk about teams, here we are talking about Development Team of Scrum which is 3-9 team members. This team excludes the Product Owner and Scrum Master.
• Scrum Only defines “One Team – One Product” Context. Scrum does not define the other contexts. Scaling is out of scope of Scrum.
• “Many Products – Many Team” context is defined by portfolio management, which is out of scope for most of existing scrum frameworks – SAFe is one of the most prevalent scaling framework that helps with
• “One Team – Many Product” quadrant is for Staff Pools who do heterogeneous type of work. Generally involved with ticketing systems in organizations. Kanban may work in this quadrant and the focus is here on resolving bottlenecks and getting heterogeneous work done without bottlenecks.

Context of Scaling and De-Scaling

When we think of larger products, we can think about descaling – that is, dividing the larger products first into smaller units of independent value streams (Products).

For the above product, the Scrum Context is that for each product (an independent value stream) there has to be a single product backlog, a single product owner and a single development team. Like the below diagram, each team works as an individual Scrum Team with their own “descaled” Product Backlog. There will have to be some considerations made to co-ordinate among the various dependent product backlogs. The diagram shows how the descaled products work.- Spotify is an example of framework which works on the principle of descaling

Context of Product Backlog and Product Owner

There is always a one to one relationship between a Product and Product Backlog and Product Owner. The context of the product backlog is shown below

The following structure where one is NOT OK. The accountability on the Product will go down if the following is implemented. It is better to descale the product instead of having a structure as shown below:

Context of a Product Owner and Development Team in Scaling

A Development team will always look up to Only one Product Owner.

A Development Team looking up to multiple PO means, the accountability is not central and conflicting decisions may come to the Development Team.

The post Scaling Context – Some Basics Concepts appeared first on World Of Agile.

Common Misconceptions

Most people feel that every Sprint MUST make an Increment which HAS to be sent to production and given to the end-users for using. This is actually not true. Sometimes, it is not possible to produce Incremental “production-ready” outcome every Sprint

What really is the outcome of the Sprint?

Scrum defines the “OUTCOME” of the Sprint as an Increment which is “USABLE”. Usability is associated with getting feedback and not necessarily “Go-Live”

Recommendations

• Sometimes, in a short sprint it may not be possible to create an “Go-Live-Ready” product. However, every Sprint must result in a deliverable where feedback can be received. The feedback may happen from internal stakeholders and not necessarily the end-users
• Sprint should be considered separate from “Release”. While “Release” is not a Scrum term, most teams use the word “Release” to describe “Go-Live-Ready” outcome. Generally, it is observed that multiple Sprint outcomes can be combined together to create a “Release”. However, this is also not a necessary condition. Sometimes, teams may create multiple “Go-Lives” within the Sprint itself. It is important to understand that feedback is necessary every Sprint so that we know as a Scrum Team that we are going to achieve the Product Goal (Long term Goal). The intent of Sprint should be to optimize the probability of achieving the long-term objective and the feedback received should be used to optimize this probability. Now, the feedback may come from internal stakeholders or end-users. Important part is that feedback should come every Sprint.
• While this article says that it is not necessary to create a “Go-Live-Ready” outcome every Sprint, that does not mean that we create “Documentation” as output. We CANNOT be doing a “Requirements Sprint” which creates a “Requirement Document” as an output. We CANNOT be doing a “Coding Sprint” to create a “Code” as output. If we create outputs such as Documents then what we end up doing is “Outputs” (which are of no value) instead of “Outcomes”. “Outputs” do not necessarily result in “Outcomes” which are of business value or feedback-able value. “Outputs” do not necessarily result in feedback where we can check progress towards the Product Goal.

Conclusions

Sprint creates an Incremental outcome which is called Increment. The Increment should get the Scrum Team feedback, which will optimize the probability of meeting the Product Goal. The Incremental outcome may be a “Go-Live-Ready” outcome many times within a Sprint or sometimes over multiple Sprints. The Increment should never be an output which is not a valuable outcome.

The post Myth about every Sprint Output resulting in a go-live to production appeared first on World Of Agile.

Common Misconception

Most people treat Sprint Review as a “gate check” forum to release the product into production. Thus, the Sprint Review ends up being a formal meeting with gate-checking checklists to go through so that the product can be sent to production.

What really is a Sprint Review?

• Sprint Review is an informal forum to seek feedback from the Stakeholders
• Intention of the Sprint Review is for the Scrum Team to find out if they are on track to meet the Product Goal, and if not, inspect and adapt the Product Backlog for future Sprints
• Scrum de-links the “go-live” or “delivery to stakeholders” and “seeking feedback”. Sprint Review’s intent is to seek feedback. The product may be delivered to the stakeholders before or after the Sprint Review

Recommendations

• Sprint Review should be a informal discussion between Product Owner, Stakeholders and Developers to discuss improvements in the product and increase the probability of reaching the Product Goal
• Sprint Review should be more like a working session instead of formalized presentations and checklists to go through
• It is better to create a separate gate check forum other than the Sprint Review. If any gate checking is done in Sprint Review, the real intent of getting feedback may be lost

Conclusions

Sprint Review is not a gate checking forum. It is a working session where Scrum Team and Stakeholders collaborate on how better to achieve the Product Goal

The post Myth – Sprint Review Being a Gate to Releasing Value appeared first on World Of Agile.

Common Misconception

A Product Owner is considered a “Client” or “Customer”. So, most people feel, how can you invite a “Client” for an internal forum to discuss “what went well, what did not?”

Recommendations

• Most Product Owners indeed behave like a “Client” or a “Customer” instead of a Scrum Team Member. Therefore, the Developers do not consider the Product Owner a part of them and consider him/her as an outsider. The Product Owner should be first part of the Scrum Team and then be a “Customer” or “Client”
• Product Owners often ridicule the teams or shout at their teams for not understanding the requirements properly. Product Owners often escalate against the teams to the Line Managers of the Developers. This creates a divide between the Product Owner and the Developers. The Developers then don’t feel comfortable about opening up in front of the Product Owner. The Product Owner’s job should be to get the Developers to feel comfortable and put the “fear-factor” to rest. This means, the Product Owner should be empathetic, calm, show patience and be looked at as a Leader instead of a manager.
• Sprint Retrospective is a forum to see what went well and what did not go well. This increases the effectiveness of the next Sprint. If the Product Owner is not there in the Sprint Retrospective, then effectiveness improvement will be looked at only from the perspective of the Developers. The Developers are doing the technical work for the Product itself. So if the Product Owner is absent in a Sprint Retrospective, the effectiveness improvement cannot be done from the Product perspective
• Sprint Retrospective is a good forum to discuss the DoD (Quality Measure for next Sprint). If the Product Owner does not attend, the Sprint Planning of the next Sprint may be an issue since DoD is not finalized.

Conclusions

As a part of the Scrum Team, the Product Owner is a mandatory participant of the Sprint Retrospective. Sprint Retrospective is an excellent forum to thrash out any opinion differences and plan for improvements and everyone from the Scrum Team must participate.

The post Myth – Product Owner should not be invited in a Sprint Retrospective appeared first on World Of Agile.

Common Misconceptions and Negative Implications

·        It is a common practice for a Product Owner to come up with a set of requirements in the Sprint Planning and disappear during the Sprint. Then appear directly at the Sprint Review forum and conduct the review of the Developers work during the Sprint Review.

·        By doing this, an important opportunity of giving feedback continuously is lost. The feedback becomes a big-bang-end-of-the-sprint activity

·        Since Product Owner is seeing the product first time during the Sprint Review, it can become difficult to take feedback from stakeholders

·        If Product Owner brings along the stakeholders to do a Sprint Review, the Product Owner ends up acting like the “Other party” and does not build confidence in the team doing the work

• When the Product Owner sees the newly increment only at review time, Sprint Review becomes a gate that holds back value from reaching the users. When the Product Owner sees the increment during Sprint an opportunity is created to release value earlier

Recommendations

·        The Product Owner owns the product. Therefore, it is important for the Product Owner to participate in the product development continuously and constantly provide feedback.

·        The Product Owner has to build confidence with the Developers. Developers being left alone is not a great idea. The Product Owner should not behave like an outsider and should not sit on the other side of the table. Once the Product Owner wears the “Product Owner Hat”, he/she is first part of the Scrum Team and then part of the Customer or Client or Stakeholder team

·        Developers must not fear the Product Owner. Feedback is not a cause for worry. Feedback is good. The earlier you receive the feedback, the better it is for the Product

·        Typical steps in the Sprint Review that are seen useful by many teams could be as below

o   The Product Owner should invite the relevant stakeholders for a Sprint Review

o   During the Sprint Review forum, the Product Owner should sit with the Developers

o   The Product Owner should thank the Developers in front of the stakeholders for the work done

o   The Product Owner should take ownership of the product during the Sprint Review and collaborate with stakeholders to make it a working session

o   During the Sprint Review, the feedback should be from Stakeholders and not from the PO. The PO feedback is assumed to have been taken already during the Sprint

o   The stakeholders and Product Owner should think about the next steps based on market place conditions and make adjustments to the Product Backlog

Conclusions

·        Sprint Review should be a working session between entire Scrum Team and Stakeholders

·        Product Owner should be reviewing the product with stakeholders with help of the Developers

·        Product Owner should have provided feedback to the developers continuously throughout the Sprint and should not be seeing the product first time during the Sprint Review

The post Myth – Sprint Review being the review of the Developers done by Product Owner appeared first on World Of Agile.