Software Never Mattered More

When the Agile Manifesto was created, its authors’ goal was to challenge the status quo, and document what they perceived as common sense for how to best build software. It’s fair to say it has served its purpose – across the world, software development teams have adopted, or at least tried to adopt, an agile mindset. And it’s hard to find anyone who disagrees with the core ideas of agile software development, such as cross-functional collaboration, early and frequent delivery, and its focus on creating value for customers and end users.

Software architecture, whether it’s defined as “the structure of a system, its components, their relationships and the principles that govern its development and evolution”, “the things that are costly to change your mind about” or simply “the discipline of focusing on the important aspects”, has always played an incredibly important role, at least in the mind of one particular kind of participant in software organizations – namely, the architects. 

But while some still perceive some tension between architecture and agile development, many teams have come to conclude that too little architectural focus is just as bad as too much of it. Nobody maintains that agile development means you’re not allowed to think things through at the beginning, and nobody believes that architecture means perfect up-front design and tons of documents that tell others what to do. (At least not anymore.)

Many agile practitioners have recognized the value of architecture work, often assigned as a responsibility to the team rather than being the dedicated task on someone with a specific role. Many architects have adopted an agile approach to system delivery, deriving feedback from real world usage of their systems, and hence getting information about the success or failure of their architectural decisions, allowing them to change and adapt.

Agile software development and software architecture have been around for a while, and their respective practitioners have had to work with each other for just as long – and they mostly succeeded. So why do I believe in the need for the two communities to meet, interact, and collaborate? 

While agile development and software architecture might mean roughly the same thing they did two decades ago, the world has changed drastically. We, the software development community, used mostly to build things that automated something that existed before and made it more efficient. How to do things, at least from a business perspective, was something known by the domain experts, and the main goal was to work with them closely to extract what they knew to be “right” from their minds. 

Technology was a tool used to enable this, and tech expertise was an aspect necessary for implementation only. The famous “-ilities” focus of software architecture (on what is sometimes still nonsensically called “non-functional requirements”, but better labelled “quality attributes”) is a sign of this orthogonal view: A system’s scalability, testability, maintainability, stability, etc., are all perceived as mostly orthogonal to the business functions it supports.

But today, the role software expertise plays is both much less, and much more, than it used to be. Software developers and practitioners are no longer experts in some niche technology – instead, what they know is required as part of most mainstream activities, processes, and businesses. They still automate existing things, but just as often they help to use technology to do completely new things, or create a demand to do things completely differently.

Automation via software is no longer the exception, but the rule. There is almost no business, no public service, no kind of entertainment, nothing that offers any sort of capability to humans, that does not rely on software to deliver its services. To a certain degree, software has become a commodity – everyone knows (roughly) what it is, what it can do, how it’s built, what value it brings, how necessary it is. “Software developer” used to be a somewhat exotic profession and has now become a very common, in one of the most frequent jobs in many countries.

Technology matters. Knowing how software and software ecosystems (in detail) is no longer a skill that is only relevant for implementation aspects. How to best use some new technology, some software as a service offering or a new infrastructure capability, can become an essential competitive advantage to a business, improve people’s life in a community, or even change how humanity works (for both good and bad, cf. social networks). Technical knowledge has never been more important to the business domain, to the effect that technical knowledge sometimes is domain knowledge.

Short-term projects turn into long-term services. Both software feature delivery and architecture work are no longer tasks that need to be done in the exceptional circumstances of moving from one stable state to another, commonly known as a “project”. While many organizations still rely on projects as a means of scheduling work, more and more have recognized that delivering a service to their users requires continuous creation of new features, introduction of new business processes, optimization of existing ones, new ways of interacting with partners, etc. Software like this never goes into a boring “maintenance” mode anymore; instead, it is significantly evolved throughout its lifetime, creating the need for long-term maintainability and changeability on the fly.

Software and organizations have merged. The role a software system’s structure plays, and its connections to the organization that creates it, as well as to the humans involved and to the processes they need to follow, has been known for a long time. (Mel Conway wrote about in the late 60s; he’s still active on Twitter, you should follow him at @conways_law.) The importance of software these days means that the structure of a system can have wide-ranging consequences not only for its developers and maintainers, but also for the whole organization, its customers and partners, potentially even a whole ecosystem. This is of course true for the quality aspects, such as security, availability, or performance. But a system’s architecture, by definition, defines how well it can be evolved. If the system and the organization it supports are as intertwined as it’s typically the case these days, software architecture determines how a business can change and adapt – or possibly even if it can do so at all.

(De-)Centralization requires trade-offs. The idea of cross-functional teams, where people with different skills interact to create a desired, measurable outcome, has led many organizations to view end-to-end responsibility as a key factor for success. But autonomous, decentralized teams often do not exist in isolation. Whatever they create needs to interact with systems, or parts of a system, built by other teams. Some coordination will be necessary, but must not become a bottleneck. Too much centralized control is bad, but so is too little of it: Some aspects, such as compliance with laws and regulations, security standards or organization-wide changes might apply to all teams. Rejecting all forms of “governance” (something that has become a bad word if there ever was one) might be needed, but should not get in the way. Similarly, while re-use has (rightly) become less of a focus in modern development, with parallel, independent flexibility often deemed more important than ultimate efficiency, sometimes doing or storing the same thing twice actually is a waste of time and effort. 

The contrasts of centralization and decentralization, of over-arching macro decisions and independent micro architecture, of team goals and organization goals allow architects to show off their favorite skill, handling trade-offs, and how well they actually are at applying it might decide the fate of an organization.

Humans Matter. Software architects, at least in some cases, still need to learn what agile practitioners have recognized a long time ago (again, at least is some, or maybe most, cases): The importance of humans, and human interaction. This is true for the interaction among the team members involved in system creation and evolution: Communication with respect, consideration of different viewpoints, communication skills, methods for decision making, are all crucial. 

But it’s not only relevant for our teams: Almost every software system is ultimately created to support humans, and even if it doesn’t have direct user interfaces, always affects them. We need to look at the effects and consider them not only from a logical, but also from a human perspective. We need to ensure there are no ill effects on some part of humanity that happened to be not involved in its creation, some misuse for some evil purpose, or some more or less catastrophic outcome due to negligence on the architects’ or developers’ part.

Ultimately, what we do has become more important than most of us ever expected (at least I can say so confidently for myself). We should be happy – after all, who doesn’t like to work in a profession that is sought after? –, but we should also be aware of the responsibility that comes with it.

Valuing technology, but considering humans more important; recognizing the importance of architectural decisions, but understanding we get feedback only from actual delivery; changing and adapting our processes to the increasingly dominant role of software, while standing on the shoulders of the people before us; refining our existing ways, and finding new ones, to deliver responsible software, and deliver it responsibly: Those are the kinds of discussions I look forward to in the Agile Meets Architecture community.