The Goals of a Software Organization
I have a lot of crazy ideas about how software engineering teams should be organized and managed. I say crazy because every time I share my thoughts with others, they tell me as much. The thing is, I have had a lot of success by applying these principles, so even if they’re crazy, maybe there’s something to them. I was recently motivated to try to enumerate these ideas in a logical structure, and as I set out to do so, it occurred to me that I could share them publicly. Selfishly, that can only serve to sharpen and improve them. Therefore my hope is that this will be the first in a series of posts on how to organize and manage a successful software team. I hope you’ll join me for the journey.
Before diving into specifics, a little bit of context is in order. I have worked in software for 26 years. During that time, I have seen many different approaches to organizing and managing people. Around a decade ago, I was reluctantly pulled into management. As the company I was in grew, I found myself “leading” a team of ~120 people in 6 different countries. I vividly remember sitting in my office one day wondering, “What do people imagine I do in this position?” What could it mean to “run” an organization this diverse and distributed? I would ultimately learn that my understanding of leadership was fundamentally wrong, but my frustration with it primed me for what was next.
Seeing the futility of middle management drove me headlong into a tech startup that was experimenting with a managerless system known as Holacracy. Entertaining the idea that an organization might be able to function without managers really opens your mind. You begin to see that this notion of traditional hierarchy and organization that we all take for granted might have some fundamental flaws. This experience is what ultimately created my appetite to dig deep on the abstract topics of managing and organizing. As my time there ran its course, I was hungry to be able to apply the principles I had learned in a new environment.
That opportunity came at my current company in the form of an open-minded CEO and a CTO that understood that a software leader needs to care just as much about the organization as he/she does about the technology. I was off to the races. I’ve been at it for close to two years now, and while ultimately others would have to be the judge, I find this to be the healthiest, most mature software organization I’ve had the privilege to be a part of. Luckily, I’m surrounded by talented, open-minded people that have honed these principles with me to make the organization what it is today. At this point, I’m motivated to try to capture these principles in writing. It’s good for clarity within the organization, and maybe it can help others replicate what we have.
As I dove in to try to bring order to this jumble of ideas that was guiding the organization, it began to feel like I just had a bunch of principles that my intuition and experience told me were good, but they were really unmoored because they weren’t explicitly driving toward anything. If I say that “having good culture” is an important principle, a reasonable person might ask, “Why?” There are a lot of good answers to that question, but I realized I was ignoring it altogether. When I took a step back and simply asked, what are all of these principles driving towards? A coherent and simple set of goals emerged, so that seemed like the best place to start this series of posts. What goals do we want our organizations to strive towards?
Scalable
Let’s start with scalable. By scalable, I mean that we want the organization to exhibit an economy of scale¹ which is to say that we want it to become more economical/efficient as it grows. Economies of scale are often achieved in manufacturing: as a factory produces more widgets the incremental cost to produce each additional widget decreases. Ironically, this is not how “knowledge work” typically scales. The bigger a business gets (as in number of employees), the less efficient it gets. This is known as a diseconomy of scale.
It seems tragic that as we grow a business with the intent to better execute its mission, we end up only getting fractionally additional productivity. By contrast, biological systems, or more generally non-hierarchical systems, tend to exhibit economies of scale. Not to tip my hand too much, but you’ll find a biological thread throughout my thinking: nature has a lot to teach us regarding organization.
Wouldn’t it be great to buck the trend and have a software business that gets more efficient as it grows?
Antifragile
Any good engineer knows the importance of robustness — the ability to recover from failure or, more academically, the ability to tolerate perturbations. Sounds good, but there’s actually something even better: antifragility. Antifragile systems aren’t just robust, they actually get stronger under stress. Many biological systems exhibit this behavior. For example, bones get stronger when placed under repeated stress (Wolff’s Law).
Think of the things that keep you awake at night as an organizational leader. If Kate doesn’t get her bit of work done on time, the entire project plan will be thrown off. Or maybe, Rob seems a little disconnected these days. Is he looking for another job? If he leaves, we’ll be done for. Or how about, we’ve thought through the ins and outs of this framework for weeks. What if the assumptions we made while we were planning don’t hold constant? Wouldn’t it be great if you didn’t have to constantly be worrying about stuff like that? An antifragile organization would naturally adapt to these kinds of events and build up an ability to handle them effectively — not just endure them.
Nimble
A nimble organization is one that can easily and quickly adapt to changing circumstances. It isn’t exactly orthogonal to being antifragile, but so far, I’m convinced it stands alone as an organizational goal.
An example might help. When I started with my current employer, we had the all too common problem of not finishing a lot of what we started. When engineers encounter this, they conclude that management and ownership are just too darned ADHD: they can’t make up their mind and stick to something. Meanwhile, the owners are simply trying to adapt to the changing business climate: “You said you’d have that done in 6 months, and here we are 9 months later. I can’t wait any longer. Time to move on.”
Most recently when I bumped into this, rather than just get on my high horse and shake my head at the fickle nature of CEOs, it occurred to me that maybe I should just embrace the fact that we can’t make up our minds. Instead, build an organization that can quickly adapt without feeling the pain. To do otherwise would actually just be denying reality. That’s nimble.
But it isn’t just about how we build software. It’s about how we organize. Say my software team is split up into business domain groups each with a manager, and the business decides to shift strategy so that those domains don’t make sense anymore. That means I have to do a reorganization. Painful. What if we had a system that could just reorganize itself in reaction to the new strategy? That’s nimble.
In Salim Ismail’s talk on Exponential Organizations, he makes the point that business is changing so rapidly due to automation and innovation, traditional organizational approaches simply won’t survive. Maybe nimble isn’t just a nice goal but rather necessary for survival.
Innovative
At the heart of any entrepreneurial endeavor is novelty and disruption — some product or service that people are willing to pay money for. That has to come from somewhere in the business. I’m going to claim that you’ll be better off if it comes from your software organization.
Too many organizations get this wrong. They have designated “geniuses” or inventors in the business, and the software team is a mysterious beast that has to be pushed to get the genius’s ideas implemented. That’s a recipe for slow, frustrating product delivery.
So let’s ad innovative to our list of goals.
Productive
The final goal is perhaps the most obvious. An organization can’t be characterized as successful unless it’s actually producing something — more specifically, the right thing. It might be tempting to add “efficiency” or something similar in here, but I actually don’t think that is a standalone characteristic of a successful organization. If we were to walk down that path, I would say “profitable” is probably a better stand-in.
It isn’t obvious with the information I have presented so far, but “efficiency” can actually lead to behaviors that undermine the other goals. Perhaps there is an implied minimum standard that to exist, an organization must be profitable, and if the market won’t bear the cost of production, you will be fighting an uphill battle by focusing on efficiency. If you discover that, it’s likely time for a strategic shift.
It’s an Agile concept, but the other problem with focusing on efficiency is we lose sight of the end goal. What have we accomplished if we have very efficiently built the wrong thing? Therefore an intended nuance of the word “productive” is that we’re focused on the end result.
There you go: scalable, antifragile, nimble, innovative, and productive. Pretty simple, but maybe not so simple to achieve? My claim is that the principles I enumerate over the coming weeks and months will each directly feed one or more of these organizational goals.
My immediate reaction as I saw these emerge from the effort to logically structure my thoughts was basically, “Of course. These are obvious.” What’s your reaction? Am I missing something? Oversimplifying? Share your comments, and let’s help each other get better at how we organize and manage.
Footnotes
[1] In watching this video, I realized that I might be abusing the terms economy/diseconomy of scale. It would see the term that is appropriate is “increasing return to scale”. I don’t think it changes my arguments, but I’m wondering if it might clarify some of my future research.
Curious to Learn More?
Check out Principles for Leading Software Teams: A Guide for related articles and reference materials.
