Two of the most contentious words in technology culture today are “Agile” and “Architect.” Perhaps, in your world, this is not true. If so, I envy you (and want to work with you). I experience an endless stream of divisive opinions about both. A well-known voice on social media posted “Architect is a bullshit role” and mayhem ensued. Another well-known voice quoted yet another well-known voice in an interview, “Agile has come to mean doing half of scrum badly and using Jira.” The host laughs and nods.

(I’ve rarely used JIRA. Confusing Agile with a tool is like confusing event-driven architecture with Kafka.)

I’m not interested in fighting to define “Agile” or “Architecture”. In my experience, those arguments are a waste of energy because we are usually comparing Apples to Oranges. The “bullshit role” person went on to describe a bullshit approach to architecture – a strawman that could be stabbed with the pitchfork of opinion. At conferences, I’ve been told that “architecture is Kubernetes implementation.” Cool, in some circumstances, that’s architecture. But what I do, designing conceptual integrity in modern systems, also matters.

Agile haters, when we dive into why, describe to me a process that breaks the spine of Actual Agility. So, of course, they hate it. I’d hate “it” too but I wouldn’t call it Agile. They describe warped processes that shield people from interacting skillfully with other people, especially “non-technical” people. In systems architecture, the more complex and novel the challenge, the more our success depends on sharing expertise and thinking well together. In my experience, you can’t skip that part.

Solving the Same Problems, Again

“Agile was invented because reality refuses to bow down to power.”

— Joe Eaton, PhD, systems engineer

Reality refuses to bow down to power. If that problem is solved, I didn’t get the memo. As an engineer, I’ve seen command and control approaches capsize innovation. Again and again. As a systems architect, I’ve seen  “modernization” initiatives topple organizational power structures (rather than succeed). Over my career, I’ve seen initiative after initiative hit the same iceberg – top-down, reductionistic approaches don’t scale as complexity increases. Complexity is increasing everywhere. My colleagues and I succeeded when we could avoid hitting that iceberg.

Knowledge work, which is what we do in Agile and in Architecture, requires excellent communication skills. The trickle-down approach where some people (leaders) do the thinking and others (implementors) do the work is insufficient and fails in ways that lack transparency. When nobody architects the system as a whole, the organization’s communication structure (or lack thereof) architects the system, according to Conway’s Law. When some people (product) outline the requirements and others (developers) deliver on deadline, well, we’ve been down that road already, haven’t we? Isn’t that waterfall why we started becoming more agile? 

Teams without fundamental mental and social agility can't effectively scale as the system increases in complexity. Software is increasingly interrelational and interdependent. We must evolve our ability to respond to change together with interrelational dexterity.

In systems, the smallest patterns of thinking and behavior scale into The Whole System. Fortunately for me, the core qualities and capabilities that support systems architecture were also the core qualities embodied in high-functioning and mature Agile teams. 

The Five Qualities

Five qualities have always been present in my high-functioning Agile teams. They are also imperative when designing systems of software, which we are all increasingly called on to do.

• Listening to each other’s contributions and partnering to share expertise.
• Respect each other’s leadership.
• Time, energy and attention as the three main ingredients needed to deliver anything. 
• Learning is the point of it all. This is what breathes life into our software systems.
• Playing by the rules and changing them as needed. Learn from your feedback loops.

Listening and Partnership

As an engineer on an Agile team, I figured out complex challenges with the Product Owner, Scrum Master, designer (or other roles) and fellow engineers. We listened respectfully to each other. We needed to integrate diverse expertise in order to succeed. None of us knew the “right” answers because there were no right answers, only the best possible solutions we could create together based on what we understood at the time. 

As mature adults, we managed our process through self-organizing. Yes, we had managers. I have been the manager of engineers on Agile teams. Which meant I supported their career aspirations, improved our sociotechnical systems and resolved organizational blockers. They rarely needed me to tell them what to do or how to do it. Work got done because we had a strong (Agile) process, team autonomy, support for continuous learning and a passion for our work. 

Also, we cared about each other. 

As a systems architect, I can’t know what’s going on everywhere in the system. I need other people’s viewpoints and expertise. I partner with people who speak accounting or understand business goals or coded the legacy software or can correct my models or help them communicate better visually. The sum of us together is greater than I could ever be individually. My job is to discover, communicate, share and structure thinking so other people can make use of it. I did that on Agile teams facing complex coding challenges too.

Support Each Other’s Leadership

As an Agile team, we make decisions together. But not all decisions, inevitably, belong to the same person. From the start, the product owner interconnects value and what we are building.  If we deliver something that has no value, the product owner’s job is on the line. If we push code to production that takes down production at 2 AM, an engineer who wrote it gets up and fixes it. If the front end isn’t accessible to blind users, the UX person will get called out. We are each, in our roles, developing our leadership skills. We are accountable, individually and as a team, when we are wrong. 

In this way, we practice leadership as L. David Marquet envisions it. “The leader-leader structure is fundamentally different from the leader-follower structure. At its core is the belief that we can all be leaders and, in fact, it’s best when we all are leaders”. We create an environment in which we can thrive and deliver high-impact work together. We do this by trusting each other with decisions that might be wrong. We will inevitably be wrong, in part because we discover information we didn’t have at the time, which is why we iterate and improve rather than blame.

That doesn’t mean tolerating bad behavior or staying stuck in unworkable patterns. A product owner assigned by a client was not listening (at all) to the reality of the team’s sizing. He explained why the work was trivial, despite not understanding the implementation. (Trivial is a trigger word.) He said he had no time for planning but had plenty of time to correct whatever he thought was wrong. Nope, nope nope. That product owner had to be replaced.  We have to trust each other to make relevant decisions, which means we each need to be trustworthy.

We support each other’s leadership and we own when we are wrong. On another team, with a kind and careful product owner:

PO: “Diana, do we need to do performance testing before pushing this change to production?”

Me: “Nope! It’s not comprehensive and we aren’t adding load.”

PO:  “Would you bet my career on it?” 

Me: “Yup! It’s fine.”

A few days later, the production instance slowed to a crawl. Turns out, there was a bug in the MySQL MAX function that got hung up in some circumstances. Over time, at scale, those little pending queries bogged down the entire system. If we had done performance testing, we might have caught it. 

I took unequivocal responsibility for the outcome. I decided and I was wrong. The PO didn’t agree but he respected that it was my decision. I fixed it. I owned it. And I did not make that mistake again. (I have, of course, made lots of different mistakes.)

Invest Time, Energy and Attention

I’ve seen, again and again, that measuring the energy required to build something is more accurate than estimating time. On Agile teams, that’s what story points did and our velocity was generally reliable (our time estimates rarely were). Time is static, a reliable 24-hour day, but energy varies depending on circumstances. A team that creates an energizing space to do hard things will have more energy to invest. A team that requires a lot of emotional labor will sap energy.

Endless bikeshedding is exhausting. “My way or the highway” leaders create unworkable situations. Engineers who disrespect other people’s ideas (especially “non technical” people)  are incorrigible. Patterns of thinking and behavior can deplete our energy and therefore our ability to generativity apply our time, energy and attention to the hard problems. Other patterns, like mutual support and enjoyable discussions, increase our capacity to do hard things.

Even when we have plenty of time and energy, we need to focus our attention, especially when something is new or complex. This means, for example, working when our attention span is longest (for me, that’s early morning). Trusting that impactful things are being tended and we are being told the truth. Getting help when blockers arise. All of these social accommodations were inherent in the Agile processes I experienced. The most successful teams I’ve been on were also the most enjoyable to work on. Science supports my personal experience, emotional positivity is better for thinking.

As a systems architect, the value of my work product depends on my ability to structure time, energy and attention (my own and other people’s) so we can see, understand and resolve complex systems challenges. A structured investment of time, energy and attention can unlock the mystery of “what the heck is going on between these two pieces of software that keeps causing weird outages? “ Thinking is a craft that thrives when we pay closer attention to it.

Too many times, I’ve gone into discussions where energy and attention are drained overcoming defensiveness – a hostile, unwilling to engage, this isn’t worth my time, “change my mind” attitude.  I understand, of course,  the damage done to productivity by unnecessary meetings. No unnecessary meetings! But in my experience, significantly more energy and attention is wasted when people refuse to think well together and say no constantly, like they are a castle to siege. 

Learning is Everything

I have a voracious need to learn and grow. So do many of my colleagues. At the heart of our Agile process was the desire to discover new ideas, invent solutions, try things and build things and test things and think differently about challenges and learn through feedback loops. I feel sad when I hear about Agile teams that aren’t driven by learning. Where’s the joy? 

Learning isn’t included here because it’s my favorite quality, though it is. The ability to learn well, together, is the single most important capability when architecting systems.

Follow and Improve the Rules of the Game

Through retrospecting, Agile invites doing the work and improving how we do the work. Meta thinking, thinking about our thinking, is part of the process. We follow some basic principles: self-organization, cross-functional collaboration, mutual respect and the protection of time, energy and attention. We also adapt those principles as circumstances change.

Systems architecture will always change the rules of the game. The previous rules and structures keep the legacy system in place. To change patterns, trends and feedback loops, we need to engage our previously-held mental models. We think about what we are doing, how we are doing it and why we believe it matters to do.

In other words, if I can’t do meta-thinking, I can’t do systems design. 

The trick, in Agile and in Architecture is “just enough”. Just enough of the right rules to enable collective action without weighing the process down. This is tricky business. As Agile practitioners and as Architects, our work includes the subtle understanding of sociotechnical patterns and a willingness to work with them, iterate and improve them, over time as we learn. 

If you’ll be at the Agile Meets Architecture conference, introduce yourself! I look forward to seeing you there.

‍