Skip to content

The Problem With Design

Somehow, some people have gotten the idea that design is somehow inimical to Agile. That’s not true. It’s always helpful to think about things before you do them. The question is not whether or not you should think in advance, it’s how far in advance should the thinking occur. Hours? Days? Months?

The problem with shorter periods is that you may get something wrong and have to redo it. Nonetheless, the most Agile shops tend to design in short (a few hours max) bursts literally just before implementing. However, they use design principles that can handle incremental changes (domain-focused designs, for example). They create the simplest design that works for the story at hand, then incrementally modify that design to accommodate the next story. Incremental design is a discipline in it’s own right. You don’t just go with whatever pops into your head first. You can easily learn how to do it, however. (Take one of my workshops! :-)).

The (considerable) plus side of this approach is that you have the minimal (and simplest) effective design. That simplicity speeds up current development and maintenance, and accelerates the speed of changes. Down-the-line payback is considerable.

The problems with doing your design too early far outweigh the problem of occasional refactoring in an incremental approach.

First, the odds of getting something wrong if you’re thinking too far in advance are very high. It’s just not possible to plan in advance for everything. You learn by releasing code to actual customers and getting feedback. Too-far-in-advance designs can’t incorporate that knowledge, so are never optimal (or at least I’ve never seen an optimal one). Put another way, you’ll have to refactor these larger designs, too, because they’re never right. Unlike an incremental design, that refactoring is likely to be much more difficult. The benefits of too-far-in-advance designs are usually illusory.

You can try to overcome the refactoring problem with a “flexible” design that can accommodate whatever you throw at it. Unfortunately, those designs are inherently large and complex, and they easily take three or four times longer to implement than a simple solution. You are effectively guessing about how the design might evolve, but knowing that your guesses are guesses, you end up planning for many eventualities that never happen, and waste time building code that’s never used.

The best case in this scenario is that unnecessary work pushes out your release date by 2-4x. The cost of that delay is way larger than the cost of the extra work, and the extra time required for development is typically longer than the time required to do rewrites in the incremental-design scenario. You haven’t really gained much.

The next problem is sunk cost. A large complex design, once in place, becomes politically impossible to change, even if it’s fundamentally wrong. Consequently, people put up with it taking two or three times longer than necessary to make any required changes because they don’t want to change the underlying architecture that causes that delay. That’s a problem that lives with you for the rest of the life of the program. Everything is harder. All changes are slower. In a worst-case scenario, that can make agility impossible. Changes become so hard that you just don’t make them.

The final problem is the sheer scope of a larger design, and the fact that the pieces typically interact with one another. If you need to make a change in a corner of the design, it’s often the case that that change ripples out to the rest of the system, again making small changes very expensive. People often solve that problem with kluges that attempt to adapt a part of the design only, but those kludges accrete over time until you have an unmaintainable mess.

So, in general, it’s not a good idea to design too early. Big, up-front designs rarely give you the benefit you imagine. Instead, learn how to design incrementally, and do that a the last responsible moment.

2 Comments

  1. Matt J. on May 1, 2019 at 8:08 pm

    Wouldn’t your final generalization be more accurate if you said, “it’s not a good idea to design too much too early? I have long considered the process of good design as being analogous to the opening of a game of Go: sure, you do not know what the board is really going to look like after 20 more moves, but you know that if you plan strategically, the stones that seem weak, wandering and lost in the wilderness during the opening will spring to life later when you need them to in the game.

    Similarly in software design, if you make the right decisions early on, it becomes much easier to accommodate changes even wider than ‘refactoring’ changes when the need arises later on. But also similarly, some decisions should not be made during the opening, they must be put off until the middle game.

    But please note that this “right decision early on” cannot be and must not be an example of the ‘flexibility’ you mention. The flexibility you mention that causes only trouble for the project sounds like (for example) adding extra code for more generality instead of finding more generality by getting rid of special cases. Planning for more generality by eliminating special cases is good at any time in the development cycle, planning for more generality by adding code to handle cases (whether truly special or not) is an activity that belongs later in the process — if at all.

    Finally, I have to admit that although I called the above the way I have long considered design to be, I have rarely seen it work so smoothly. Usually, the political pressures you mention intervene and ruin it. Or worse, yet, the rest of the team is bound and determined to insist on an interpretation of Agile principles that, as you say at the start, deny that design is “inimical to Agile”.

    • Allen Holub on June 6, 2019 at 3:26 pm

      Hi Matt, My only reservation with your “not too much too early” is that, in the wrong hands, that could still lead to too much :-). It’s surprising how little you need to get started. I’m assuming, though, that the people getting started are all architects at some level (and if they aren’t, the the Architect isn’t doing his/her primary job—education). People who are just flailing around will make bad decisions. So, maybe there is a first step, but it’s not designing, it’s learning how to design.

Leave a Comment