A Dozen Ways to Fail at Scrum

Posted on November 11, 2015
by Doug Klugh

Scrum provides a process framework to help realize the benefits of Agile principles.  The value of Scrum has been demonstrated many times, on numerous projects, throughout various industries.  It is a fairly simple and straightforward set of practices and guidelines that will (usually) result in greater adaptability to change, improved productivity, high quality products, and happier customers (over waterfall methods).  But as simple and straightforward as these methods may be, many organizations struggle to realize the benefits of Agile.  Through my thirteen years of facilitating Agile projects, here are the top twelve reasons (I have observed) that organizations fail at adopting Scrum.

Fear of Commitment

Every day it seems more people are touting the benefits of Agile.  Certainly, there is often a lot to be gained by adopting Agile practices, but if you’re merely cherry-picking the practices that are easiest to adopt, or ones that best fit your organization, you will most likely miss the boat (of value) entirely.  Agile principles were derived from decades of observations of what worked in successful projects and of what did not work in ones that failed.  And the practice of applying these principles to product development has been refined for more than just the past ten years, since Agile borrows lessons learned from Lean principles, such as Kanban (which dates back to the mid-twentieth century).

If you want to realize all of the potential that Agile offers, you can’t go half-way and then throw up your arms when it doesn’t fulfill expectations.  And don’t expect Agile practices to solve your problems.  Agile is a very good tool at highlighting issues in your software processes, but once those issues are made visible through Agile methods, it will be up to the people in your organization to solve them.  So if you’re going to “do Agile”, then make the commitment to adopt all of the principles identified in the Agile Manifesto.

Lack of Discipline

Successful Agile projects require interaction and collaboration.  Not just between the development team and the customer(s), but among each other.  This is not facilitated through functional teams, who simply accept artifacts from one team, add value, and then hand them off to another team.  Teams must be cross-functional.  So if your organization has a separate validation, performance, localization, configuration management, or installer team, then get ready to reorganize if you want to be successful with Agile.  All (required) software engineering disciplines must be completed within an iteration.  A user story must be entirely developed, from cradle to grave, from requirements to deployment, within an iteration.  Once a story is done, it is done.  If you’re not able to complete that scope within an iteration, then your stories are too large or your iterations are too short.  It is also important to develop just enough, just in time.  Only write as many requirements as can be completely designed (within an iteration); only design what can be completely implemented (within an iteration); only implement what can be completely tested and validated (within an iteration); and so on.  And don’t do this weeks or months ahead, do it just in time; within minutes or hours of when it is needed.  This will enable you to quickly adapt to changes in the market, in the requirements, or in the heads of the stakeholders.

Difficulty in Extending Functionality

Being able to deliver multiple slices of functionality within a sprint requires more than effective planning and collaboration.  Success depends greatly on being able to easily extend existing functionality and supporting infrastructure.  If your solution consists of tightly coupled code, you will be challenged to meet your sprint commitment before the team even gets started.  Your software must be written in a manner that facilitates extensibility.  Employing sound engineering methods, such as single responsibility principle, open/closed principle, Liskov substitution, interface segregation, and dependency inversion will make designing and implementing extending functionality much easier, much quicker, and with higher quality.  This is essential in utilizing an iterative approach to product development.  More times than not, these principles are not applied and too much time and effort is spent trying to extend poorly written, tightly coupled code.

One Person Fulfilling Multiple Roles

Scrum defines three roles used to facilitate Agile practices.  The Product Owner represents the interests of all project stakeholders, ensures the team is pursuing a common vision, owns the product backlog, and ensures good ROI.  The ScrumMaster leads the implementation of the Scrum process, teaches scrum values and practices to everyone involved in the project, ensures that everyone follows its rules and practices, and removes impediments to the success of the scrum team.  The Scrum Team decides how many items on the product backlog it will commit to completing in a given sprint and how to deliver those items.  The Scrum Team must then deliver on those commitments.

These roles work very well in putting Agile principles into practice.  However, none of these roles are effective if they are combined with another role by the same person.  As an example, the Product Owner role should never be fulfilled by the same person fulfilling the ScrumMaster role.  If your process is working well, there should be some healthy conflict between your Product Owner and Scrum Master.  While the Product Owner is working to ensure that the team is working as quickly as possible, to deliver the highest value, with the lowest cost, the Scrum Master needs to ensure that the team is following the defined process, fulfilling all required engineering disciplines, and working at a sustainable pace.  Often times, these objectives can be in conflict with each other.  But it is the consensus between these two roles that ensures the best outcome.  You also would not want your Product Owner to also be a member of the Scrum Team.  Another responsibility of the Product Owner is to accept or reject work products delivered at the end of the sprint.  If the Product Owner is also a contributing member of the team, it would be difficult (at best) to be objective in determining if a work product is acceptable for release.  And if a ScrumMaster is a team member, that person would have a difficult time identifying flaws in a process in which he/she is an active participant.  The ScrumMaster would also have a more difficult time removing impediments as a team member.  To realize the full potential of Scrum, you must keep these roles separate.

Unable to meet sprint commitments

Working in sprints offers advantages in several ways.  It allows teams to work uninterrupted on a small piece of functionality and development that functionality fully.  When this functionality is completed at the end of the sprint, the team will showcase what was built to elicit feedback from project stakeholders (to be incorporated in successive sprints).  And showing working software to customers on a predictable basis is the best way to build trust.  When the customer can actually see the realization of what was promised at the beginning of the sprint, it’s hard not to trust a team when it says it can deliver something within a sprint.  Of course, this trust will dissipate very quickly when the team fails to deliver on their commitment made at the beginning of the sprint; especially for experienced Scrum teams.

Once a Scrum team has five or more sprints under their belt, they should have a pretty good idea as to their velocity; assuming that the team composition and capacity remain (at least) fairly consistent.  This velocity should enable the team to maintain a sustainable pace of development over the long term and consistently deliver a certain number of points each sprint.  One thing that will interfere with their ability to consistently deliver a certain amount of scope is if the ScrumMaster allows disruptions to occur outside of the team.  When executing an Agile project, there must be an agreement between the development team and other project stakeholders that the team will commit to delivering what they promised, but any changes to scope or requirements (or anything else) must wait until the following sprint.  This will allow the team to go heads-down in development without having to worry about outside distractions or interruptions.  Not only will this allow the team to meet its commitment, but it will also help improve productivity and product quality.

Groups of Individuals

Agility comes in many forms.  Being able to adapt your process quickly is one method of being agile.  This requires teamwork.  Not just a group of individual engineers, or inflexible specialists, working on their own little piece, but a group of professionals working together and making sacrifices to meet the goal of the team.  Much too often, development teams have people who only work on certain types of tasks.  It is certainly okay to have people with specialized skills; in fact, it is often needed throughout product development.  But that does not mean those specialists should only work on their specialty.  The ability to quickly respond to change requires team members who are willing to work on any tasks that are needed at that time; tasks that will add value to the product when it is needed.  It is very common for immature Scrum teams to have team members who only do implementation, validation, or configuration management.  To be truly agile, all team members must be willing to fill the gaps as they occur.  Lessons in Lean Development have taught us, if we want to achieve high work flow, we must shift resources to alleviate bottlenecks whenever they occur.  This means that team members who traditionally only do architecture or implementation must be willing to work on design, validation, or deployment tasks any time they are needed.  They need to get out of the habit of only choosing tasks in their specialty.  While it is important to have members with specific domain knowledge, it is equally important to have team members who are cross-functional.

Lack of Self-Organization

The value of group intelligence has been documented many times and in a wide variety of circumstances.  Two heads are better than one… as are seven.  Numerous studies have also shown that self-managing teams are more productive and are better at problem solving than are teams led by an individual.  If left to self-organize and make decisions as to how to best resolve issues, self-managed teams tend to rely on the collective intelligence and experience of the group, as opposed to that of a team leader.  This cooperative approach not only makes the team more productive, but also makes them better at identifying opportunities and employing methods for improvement.  Over time, the team will begin functioning as one unit, as opposed to a collection of people following an individual.

Ineffective Sprint Reviews

Sprint Reviews are more than just meetings to review what was accomplished during a sprint.  They are golden opportunities to put your empirical process into play.  Most organizations adopt Agile practices to improve their agility - their ability to quickly and effectively adapt to change.  Sprint Reviews facilitate that ability by giving all project stakeholders the chance to see first-hand what was development and the opportunity to provide feedback, criticisms, or suggestions.  This feedback is not only welcome, it is encouraged.  It helps to guide product development by making small changes to the release plan between every sprint.  It also helps to mitigate risks early.  The more eyeballs you have focused on your evolving product, and the earlier this is done, the more effective you’ll be at catching and correcting issues.  And the earlier issues are caught, the easier, quicker, and cheaper they will be to fix.  Too many teams use Sprint Reviews as a meeting to review completed stories, tasks, and defects and to make updates in their process management tools.  This type of agenda will just about guarantee that you will miss out on one of the most valuable practices that Agile offers.

Therefore, it is critical that the Sprint Reviews be all about showcasing working software.  And be sure to publicize these Sprint Reviews well enough that all project stakeholders are aware of them.  Everyone should be invited to attend and participate.  Everyone and anyone should be made to feel comfortable enough to give feedback, without fear of being criticized themselves.  This is what makes a successful product: incorporating ideas from a diverse group of people who have a stake in the product’s success.

Integrating Waterfall Milestones

Agile development methods are designed to maximize value to the customer.  By continuously focusing on the needs of the customer, the success of the project is greatly enhanced.  This is not the case in waterfall lifecycles.  Waterfall methods are task-oriented, while Agile methods are value-oriented.  Because of these conflicting principles, it is counter-productive to attempt to align an Agile development lifecycle with waterfall milestones.  The plan of Agile is to deliver value to the customer early and often, while easily and effectively adapting to change.  The plan of waterfall is to complete a predefined set of tasks on-time and within budget which are used to deliver a product which conforms to a set of requirements defined near the onset of the project.  Milestones, such as Alpha and Beta, are used in waterfall lifecycles to help measure progress.  But using Agile methods, progress is measured from working software.  If a waterfall project is 30% done, you will have absolutely no working software.  If an Agile project is 30% done, you will have 30% of the product features completely finished, working, and deliverable to your customer.  Attempting to integrate these two lifecycles does not allow you to realize the benefits from iterative (Agile) development, let alone make sense.  And if your organization is attempting to measure the progress of an Agile project by using waterfall milestones, then it is missing the boat entirely.  This behavior leads to nothing more than superficial adoption of Agile methods.  Don’t disrupt Agile practices by attempting to integrate waterfall principles.

Building Processes Around Tools

Scrum has been carefully planned to help product development teams put Agile principles into practice.  As a process framework, Scrum provides methods, guidelines, and best practices to help realize the benefits of Agile values and supporting principles.  By defining your processes based on this framework, they will be best suited to facilitate your adoption of Agile.  However, if you define your processes around a specific tool, such as a lifecycle management tool, you will be limited to the vision of those who developed that tool.  Even if they had your specific industry and domain in mind, it is very unlikely they addressed all of the issues your organization will face while developing and delivering your products.  Your processes need to be designed, by your organization, to serve its goals and objectives.  No one else can do that, especially not a development team who has never interacted with your organization.  Unless the tool is developed in house, no tool will ever fulfill all aspects of your process; even then, it is doubtful.  Not to mention what happens if you change tools.  If your processes are to serve you well, they should never be built around your tools.

Lack of Training

Once your organization is sold on Agile, education is critical to ensure that stakeholders understand their roles and responsibilities, and how to execute Agile methods in a way that will realize maximum value.  It is also important that they understand the Agile principles and how their practice can improve your products, along with customer satisfaction.  Even if everyone has experience “doing Agile”, you cannot assume they have the necessary, or correct, foundation for being successful with Agile projects.  To ensure that everyone is on the same playing field and has (at least) similar expectations of what Agile practices will offer, lineup training from someone who has the experience (preferably within your industry) and who has a proven track record of success within a similar (if not the same) domain.  This training should be tailored to your organization by an experienced Agile coach or trainer who has invested time in understanding your organization.  Establish a foundation with the basics, then after your teams are effectively applying those practices, move on to more advanced training.  It may be prudent to follow up with basic training after the first year to ensure that new stakeholders understand the foundation and expectations of the organization as a whole.

Do-It-Yourself Agile

On paper, Scrum is a pretty simple framework.  Read a couple books, attend a user group meeting here and there, follow a couple blogs, and you’re good to go, right?  Well, while Scrum may be simple, it is not necessarily easy.  Putting Scrum into practice, and getting good at it, can actually be quite challenging.  If it was only a matter of apply methods documented in a book, it probably would be pretty easy.  But it’s the human factor that makes Scrum so hard.  You can learn a lot by studying case studies in Scrum, but no two projects are the alike.  And unless you have the experience of struggling to resolve issues, coaching and motivating individuals, addressing criticisms, and defending the principles of Agile, you will find success to be quite elusive.  To succeed in with Agile, especially starting out, you really need to have an experienced Agile coach.  This person can help define a product roadmap, make sure that roles and responsibilities are well understood, help avoid common impediments, and ensure that you’re realizing value as you put Agile principles into practice.  Your coach should come from outside of your organization.  While it’s helps for this person to have domain knowledge of your product, it helps even more to have a coach with a discriminate view of your organization.  If you’re committed to doing Agile, then commit to doing it right. Please share your comments and experiences on other factors that can impede an organization from succeeding with Scrum.