When the Manifesto for Agile Software Development was first published, it was not a process, but a set of guiding principles. Two of the core, foundational principles of agile development are reflection and continuous improvement. By creating tight feedback loops of iteration, analyzing the meta-process of product development, and constantly experimenting in order to improve communication and efficiency, these early agile teams were able to efficiently deliver high quality software while being able to quickly adapt to the shifting priorities of the business.
As more and more people drank the Kool-Aid, and Scrum became mainstream, many teams adopted the dogma without the principles. They treated Scrum as a rigid process in and of itself, rather than an approach to developing a process that works for the team. Many teams have adopted Scrum as a way to measure and maximize productivity, effectively turning software development into an assembly line process. I like to call this the Mini-Waterfall model. By focusing all of your energy on estimation, burn-down charts, and velocity, while ignoring the first principles of reflection, experimentation, and continuous improvement, you have effectively become the opposite of agile.
The Assembly Line Model
The Assembly Line, or Mini-Waterfall model, focuses all of its energy on the estimation, measurement, and control of velocity. In this assembly line, the story point is the unit of production, and all systems and processes are designed to maximize productivity. In this model, a significant amount of overhead is invested in estimating tasks, logging work, developing burn-down charts and sprint reports, and providing status in the daily standups. To try new things and experiment with the process would introduce risk into the production line. To maximize feature output, there is little room to invest in automation, paying down technical debt, learning new technologies, improving documentation, or experimenting with the development process. If this sounds like you, and you aren’t meaningfully investing in continuous improvement, you’re doing Scrum all wrong.
The Role of the Product Owner
The Product Owner is not the customer of the Scrum team, she is an integral member of the Scrum team. The Product Owner’s job is to clearly articulate and communicate the product vision and mission, relative priorities, and, most importantly, the customer voice. In successful Scrum teams, the Product Owner physically sits with the team every day and is constantly working to convey the vision in high fidelity to the engineers, so that they can gain a clear understanding of the big picture and can make more intuitive product decisions. Conversely, the engineers should be constantly explaining the technical aspects of the product so that the Product Owner can gain a clear understanding of technological capabilities, limitations, and feasibility. If the product owner is treated as a customer, it prevents a strong bond of trust from forming, and creates a gap of understanding and communication.
The Role of the Scrum Master
In Assembly Line Scrum teams, the Scrum Master is simply a project manager, whose job it is to provide status reports, assist with backlog grooming, do capacity planning, and so on. In successful Scrum teams, the Scrum Master is a coach and a mediator, whose job it is to ensure the team is tightly integrated, clearly communicating, truly seeking to understand the problem, raising issues, committing to the principles, and driving experimentation to continuously improve. The Scrum Master is responsible for ensuring the team is being agile with a lowercase “a” instead of a capital “A”, and following the agile principles instead of the Dogma of Scrum.
Sprint Planning
The purpose of the sprint planning ceremony is to ensure that all high priority tasks are clearly defined and actionable, that clear success criteria are understood by the entire team, and to commit to a scope of work that will not be broken or altered during the course of the sprint.By this time, the product owner should have already clearly defined and prioritized the backlog, so that the engineers can quickly and efficiently estimate story points and determine the clarity and feasibility of each task.
The purpose of poker planning is not to commit to a hard and fast measure of productivity, but to assess how well defined and thought out are the tasks. The story point is not really a measure of effort, it is a measure of complexity. If a story is scored as a 3, then all of the variables are known, and it is a simple matter of churning out the work. If a story is an 8, however, there are many dependencies, unknowns, poorly defined criteria, or risk to break other parts of the code. The purpose of sprint planning is to identify stories that need to be broken down, further defined, or require additional research to fully understand.
The Spike
The spike is a critical tool that many Scrum teams overlook when they are maximizing for productivity. If the sprint schedule is rigid, meaning that one sprint is expected to begin immediately after the completion of another, it leaves no time for additional research or clarification. If, during a sprint planning meeting, a task scores too highly, or has too many unknowns, a spike should be initiated and the sprint kickoff should be postponed until the spike is completed. One caveat to this is if the task is not critical to the sprint, the spike can be treated as the task for the current sprint, and the feature itself can be postponed to the following sprint.
During the spike, an engineer may research a new technology, review some bit of legacy code, create a prototype, or otherwise clarify unknowns and reduce complexity. Once the spike has been completed, a true commitment can be made and the story is now ready for action. The fundamental premise here is that there are periods of rest and thought baked in between sprints. The sprint is not kicked off on a particular date, it is kicked off when there is a clear understanding and commitment to a scope of work.
The Daily Standup
In Assembly Line Scrum, the daily standup is used to gather and up-to-the-minute status update so that the Scrum Master and Product Owner can update their status reports. This does nothing to improve the ability of the team to execute mid-cycle. In successful Scrum teams, the standup is reserved for communicating roadblocks.
If an engineer is missing a critical dependency or piece of information, is unclear about the objective or success criteria of a story, or is otherwise blocked, it is the team’s duty to help clear those roadblocks. If the engineer has no roadblocks and has a clear direction for the day, he or she can simply cede their time. The status can be easily observed in the sprint board itself. This is how you keep a stand up to 15 minutes and physically standing, by focusing on helping the engineers succeed, rather than trying to get up-to-the-minute status.
The Retrospective
The retrospective is the single most important ceremony in the entire development process. If you are not performing a retrospective in earnest at the end of every sprint, where you identify what is working and not working in your process, and brainstorm and commit to trying new things, you are completely disregarding the principles of agile software development.
As an agile development coach, I have found too many teams who either skip the retrospective altogether, or they make it just another opportunity to discuss the features of the product. With some clients, they simply use it as an opportunity to bitch and moan about people and things that cause them grief, without committing to any solutions.
The standard model for driving the retrospective is Stop, Start, Continue. This is a way of breaking down and analyzing the communication and delivery of work product through the pipeline. It’s an opportunity to identify problems in the process that resulted in inefficiency or waste, and to devise an experimental method to address them. Here is an example dialogue:
“In the last sprint we tried to complete this 8 story-point task, but when we started developing we realized it depended on some unknown technology. It took us longer than expected to come up to speed on this new technology, which caused our timeline to slip. As a result, we didn’t complete tasks X, Y, and Z, and we were forced to squeeze our QA cycle to complete the sprint on time.”
“To prevent this in the future, we need to ensure that highly scored tasks are fully researched and clarified, broken up into smaller tasks if possible, and we must give ourselves some unstructured time before kicking off a sprint to research these unknowns.”
“We should stop forcing sprints to begin immediately after completing the previous sprint, and committing to tasks that are too complex. We should startsetting the expectation that the sprint planning meeting will probably be followed by one or two spikes, and a subsequent ad-hoc sprint planning meeting be held once all of the unknowns are accounted for. Only then will we truly be able to commit to the sprint and execute effectively.”
In this model, Stop means to identify a process problem and an opportunity for continuous improvement, Start means to devise an experiment to address that problem, and Continue means to validate that the experiment was a success and should be incorporated in the process going forward. This is the foundation of agile software development.
Always Be Shipping
Another critical, yet often overlooked, principal of Scrum is that you should always be in a position to ship to production at a moment’s notice. Many teams overlook this by simply not pushing to production at the end of every sprint, and lumping several sprints into large releases. This is wrong for a number of reasons.
To be in a ready-to-ship state, a number of things have to be in place:
- You must be able to test and deploy quickly, without interrupting the development cycle. This requires a solid test automation and continuous deployment pipeline.
- Your tasks need to be completely production ready by the time they are checked in and the story is closed. This requires the stories to be designed in such a way that they are broken up into atomic, useful features.
- It requires a full suite of test automation to ensure that deployment won’t create regression issues.
- It requires the architecture to be designed in such a way that there aren’t so many tightly coupled interdependencies, so that changes can be quickly and confidently deployed without creating breakage and firefighting.
The more complexity you introduce, the larger your releases, and the more monolithic your architecture, the more difficult and time-consuming it will be to rapidly push changes. This results not only in increased overhead for deployment, but it also breaks the critical feedback loops. By being unable to rapidly iterate and gather feedback, you are likely to invest too much in the wrong solution, and increase the amount of rework and technical debt later on. The Always Be Shipping principle must be woven into the fabric of your architecture and development process.
Celebration
One of the biggest failures of Assembly Line Scrum teams is that they turn sprints into marathons that never end. The endless cycle of sprint after sprint in an effort to maximize productivity can be soul-crushing. In successful Scrum teams, they take any and every opportunity to celebrate their successes, champion their customers’ successes, reward those who went above and beyond, and provide perspective on how much the team has improved over time. This can be an elaborate ceremony on special occasions, or as simple as a team lunch or happy hour after every sprint. It is critical not to overlook this period of rest and appreciation, for it provides the passion and energy that will be needed for the next sprint.
Conclusion
Hold up a mirror, look deep within your own heart, and acknowledge which of these teams you identify with more. Share this article with your team, and discuss it in your next retrospective. Identify all the ways you follow the first principles, and all the ways you subscribe to dogma. Don’t be afraid, the first step to continuous improvement is admitting you have a problem.
-Eric Weiss, Fractional CTO, product strategy consultant, agile coach, startup advisor. Connect with him on his website.
Interested in reading more? Check out How to advocate for your ideas.