As is the case with many of the people who actually know what they’re talking about, I’ve come to shudder when I hear the word “Agile,” at least until I can figure whether the person who uttered the word actually knows what they’ve just said. More often than not, they don’t.
The word Agile has taken on a meaning that the folks who coined the term at Snowbird wouldn’t recognize—a rigid process, mindlessly followed by a small part of an organization that isn’t the least bit agile (lower case). The word agile means nimble, flexible, adaptable, quick. If you aren’t all of those things, you’re not agile, whether or not you’re using some “Agile” technique. Agile is a frame of mind and a culture, not a process. You can implement Scrum perfectly, and not be in the least bit agile.
What’s important, then, is agility, not Agile™. I’ll be writing a lot about that in this blog, but this post lays the groundwork.
I’ve found in my teaching that the best way to really get agility is to start with the basics and work towards the practices (and to actually do it, of course). This article is a curriculum of sorts that will get you up to speed on those basics.
There are a boatload of books on Agile, and many are worth reading, but I’ve tried to keep this list short. I focus on the essential background that you need to truly understand what you’re doing. The list is, nonetheless, a tad longer than I’d like. There is no one source that covers everything, however.
If you’re interested in my take on the state of Agile, watch my Death of Agile keynote.
I'm not a big fan of Scrum, but it is popular. Scrum is defined in the The Scrum Guide. There's a lot of pseudo-Scrum out there, and a lot of things that people imagine are part of Scrum which are not. For example, stories, burn-down charts, and stand-up meetings are not part of Scrum. Project managers are not mentioned in the Guide so are not part of Scrum—agile teams are self managing. If you want to know what Scrum really is, read the Guide.
Agile Culture and The Workplace
Agile is not a process. It's a culture and a philosophy that you can apply in many ways, with many different processes. Given its cultural nature, the biggest difference between fully Agile shops and traditional ones are the ways that they treat people. To paraphrase a friend of mine, traditional organizations assume that the workers are conscripts, agile shops assume they're volunteers.
Agile workplaces, then, are completely and utterly different from traditional workplaces. There are no people telling other people what to do, there are very shallow hierarchies and many fewer departments, internal systems are based on trust, not control. The idea of accountability is gone, replaced by responsibility. You're not in Kansas any more, and this is definitely not your father's company.
So, you cannot become agile simply by training a few people in Engineering. Everything has to change.
Spotify, with almost 1000 programmers working in three countries on two continents is a poster child for a functional Agile workplace. They're also proof that Agile scales just fine within having to use an elaborate prescriptive framework like SAFe (which I think of a way for a big corporation to pretend to be agile while maintaining the status quo—it's a sham). Spotify is the ultimate answer to the that-can't-possibly-work crowd. If you think it won't work, they're probably doing it, and it works. For example, the "squads" (teams) don't have a budget. There's a big pot of money, and if a squad needs something, they just buy it. They don't ask for permission first (which would slow them down—if you need the tool/training/resource now, you need it now!). A dozen of them traveled from London to NYC so they could interact face to face with another team. They didn't ask permission. They just went. Working that way saved them vast amounts of time, as compared to the mistrust-based system of permissions that you see in traditional organizations.
So, Spotify provides us with a great example of what an Agile organization looks like. Here are some resources:
This book is the most important one that I'll discuss. If you read nothing else, read Drive. Daniel Pink talks ostensibly about what motivates people, but a functional agile shop is full of motivated people who want to go to work every day and finish the day satisfied. Turnover in fully agile shops is very low.
Drive talks about how to do that by following three principles: Autonomy, Mastery, and Purpose. Spotify's culture is built on those principles, and they work spectacularly well. Without a Drive-style culture, you can't really be fully agile.
Drive is actually a popularization of Deci and Ryan's work on Self-Determination Theory. If you want to look at the original research I recommend their Handbook of Self-Determination Research.
Spotify's internal workings first came to everyone's attention with this white paper (from 2012). They've evolved a bit since then—they're an agile organization after all, and if you can't apply the word to your internal processes and structure, you're not particularly agile—but the basic structure of the company is still the same as the one described here. The main structural thing they've added is a triumvirate of managers (for lack of a better word) that guide each the tribes. These folks are not managers in the traditional sense of assigning tasks and managing budget and time, but they are in charge of the direction that the development is taking and coordination both within the tribe and between tribes, however.
You can't understand Agile if you don't understand Lean. Many agile practices like short cycles, non-negotiable quality, regular retrospectives, pulling work from a "backlog," come from Lean. So do important cultural values like trusting the people who do the work to decide how to do the work, continuous process improvement, and the notions of servant leadership. It's essential that you know Lean in order to understand both why we do what we do and how we do it.
The four essential books on Lean for the programmer are:
This book desribes the Theory of Constraints, from which various Lean practices derive. It talks about workflow, what makes it fail, and how to optimize it. Sounds boring, but the book is written in novel form, and is quite readable. Understanding the theory of constraints makes a lot of agile thinking make sense. For example, the notion of a multi-disciplinary team solves problems associated with bottlenecks and handoff inefficiencies. There's also a graphic-novel version, which is condensed, but not bad if you're pressed for time.
The Phoenix Project takes the ideas layed out in The Goal and applies them to the software world. The focus is on DevOps (think: automated and continuous deployment driven by the development process), not software development per se, but pretty much everything here applies outside of "operations." The book does gloss over a lot of the details covered by Goldratt, so it can't hurt to read The Goal first. You might also want to augment this book with Humble and Farley's Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation , which has a lot of detailed, nuts-and-bolts advice, but is short on the big picture.
The Poppendieck's have written a lot of books, and they're all worth reading. I think this one is the best practical introduction to Lean, as applied to software development.
This book is about neither Lean (in the manufacturing sense) nor Startups, but it's essential reading nonetheless. Reis lays out an agile-friendly approach to business planning that centers around customer interaction, short cycles, experiments, and making business decisions based on the results of those experiments. Lean manufacturing is a development process (and philosophy); lean startup is a business process (and philosophy). You need both.
I should also mention two practical books: Marcus Hammarberg and Joakim Sundeno's Kanban in Action, is a good nuts-and-bolts practical guide to implementing a Lean process (i.e Kanban) in a software development shop. It's not a complete introduction to Lean thinking by any means, but it's a great practical introduction to a Lean/Agile process. Also, David J. Anderson's seminal Kanban: Successful Evolutionary Change for Your Technology Business is the first book to suggest applying Lean thinking (Kanban) to software development. His discussion of introducing Lean at Microsoft is priceless.
Finally, I'd be remiss if I didn't list Taiichi Ohno's Toyota Production System: Beyond Large-Scale Production. Ono invented Lean. This book is about making cars (which is why it's not in the main list), and is really about how to run a lean business, but it's a great read. In the more-essential category is Mike Rother's Toyota Kata, which is a discussion of why Toyota succeeds, not how, but also contains practical advice.
Planning and Process
I'm a little reluctant to list books about process, though I did list a couple of Kanban books, earlier.
First of all, Agile is a organization-wide philosophy and culture, not a process. Agile and Scrum are not the same thing, for example, and Scrum is neither the only nor the best approach to agility at the engineering level. Moreover, none of the processes (Scrum in particular) are complete. You can implement them perfectly and not be in the least bit agile. Finally, you cannot achieve agility by implementing some process "in a bubble" (an existing engineering team doing Scrum, for example). True agility typically requires a massive cultural shift in the organization as a whole, and probably a major reorganization and a major rethinking of things like governance practices. Reading the books, though, you'd never get any of that.
I'm also not a big fan of proscribed process in general. You can't be rigid about what you're doing and be agile at the same time. It's best for the team to get some training, then working with an experienced coach, develop a process that they think will work for them. I do that sort of work and enjoy it immensely (get in touch!). The process is one of education (literally, "to lead from"). You can't successfully impose process from outside.
If you don't have adequate experience in house or can't find a competent consultant/coach, and you want a well-documented place to start your agile journey, then implementing a documented process (like Scrum or XP) is a good start. Just bear in mind that something you read about (or take a class in) is just a start.
I'm deliberately not listing books on the big so-called "enterprise" processes (e.g. SAFe). To my mind, these big processes are actively destructive. They are ways for expensive consultants to make a lot of money working for big corporations that don't understand agility, and probably never will. Don't waste your time on them. In any event, Spotify is proof that you don't need an over-elaborate pseudo-agile process like SAFe to scale agile.
This book isn't about process, per se, but the so-called story is at the heart of agile planning. A story is a simple narrative that describes an end user in some domain-level role going through some domain-level process to achieve a domain-level, and valuable, outcome. It does not describe a computer program. All agile processes require you to organize your stories by value to decide what to work on next, and Story mapping is the best way that I know to do that.
Extreme Programming (XP) was the first Agile process that anybody ever heard of. Scrum, which actually predates XP, was quite obscure at the time, and XP is actually a much more complete process than Scrum, which describes itself as a "framework" for a process. This book describes XP, but more importantly, describes the philosophy and culture that underlies it. Most Scrum implementations use XP to do day-to-day work. Read this book, even if you never plan to implement XP, so that you can understand Agile thinking.
If you do opt to go the Scrum route, Cohn's book is probably the best description of how to do Scrum successfully. Its advice is useful even to people who aren't using Scrum.
- We do a lot of visual thinking in Agile. The best books on creating visual representations are Edward Tufte's series: The Visual Display of Quantitative Information, Envisioning Information, Visual Explanations, and Beautiful Evidence. I'd buy all four.
- Tom Coens and Mary Jenkins, Abolishing Performance Appraisals: Why They Backfire and What to Do Instead. Individual performance appraisals do a lot of damage with no up-side whatever.
So that’s my list. If you have a favorite book of your own, tell the rest of us about it by leaving a comment.