Agile and Architecture – evolving together

It seems easy to see agile methods and software architecture as two contradictory things. Software architecture is the older of the two and was already an essential part of all development methods that came before agile. Many see architecture as the core of Big Upfront Design and Waterfall processes. Agility, on the other hand, has often been understood as an approach that works without an architecture at all. 

The impression of a contradiction is reinforced by the fact that the disciplines often are performed by different people. They have different goals. The agile coach wants to deliver results fast and in a continuous manner and the Architect enforces consistent design and cares about long-term costs and maintainability.

But this impression misses an important fact. Both disciplines have influenced each other very strongly in recent years. One could say they have evolved together. Here are a few examples of this effect, based on real project events.

The first example is from the early days of agile development. The company at question developed a Software as a Service Solution. The development teams were recently split into sprint teams. The teams developed different parts of the Solution but all parts were still using a common Java framework. The fact that all teams now worked by their own cadences made problems caused by the close coupling with the framework painfully obvious. The further development of the framework required a lot of coordination. In some cases, several sprints had waited for certain features to become available, which were necessary for the teams to fulfill their actual goals. 

The consequence of this situation was a change in the architecture: The central framework was discarded and replaced by small, very thematically limited libraries. These were still shared between the teams but could be adapted by the respective teams within their sprints. Development became faster again. The software architecture is adapted to the requirements of the agile organization. A similar shift of the focus in software architecture from the DRY principle (Don't repeat yourself) to more decoupling happened in many projects not only at this company. 

A reverse effect took place at the same company some years later. Continuous Delivery proved to be a successful method to make software faster and releases less painful. Many business demands were better supported by architectures optimizing for Continuous Delivery. However, continuous delivery did not work well with the classical Scrum approach of fixed timeboxes. The features were rolled out before they were presented in the Sprint Review. New requirements based on the findings of the feature rollout had to be considered within a Sprint already, which led to frequent Replannings. 

Sprints proved to be unsuitable for developing software with Continuous Delivery. Another agile approach, in this case Kanban, was introduced. This worked much better. In the early days of agile methods delivering once a month was revolutionarily fast, and a Sprint was an appropriate method. Since then the possibilities of software architecture evolved so much that now the fixed timebox of a Sprint becomes a bottleneck.

In another example, the agile approach has had a large impact on the role of the architect. The initial situation was a classical architect who made all important decisions on software architecture within the team but also across the teams. It turned out that this organization was a real bottleneck. 

If all decisions about architecture had to be made by one person, there would be waiting times. After all, this person does not know everything that is necessary to make the right decision in each case. This knowledge lies with the developers in the team, who know their problem best. Architecture could therefore not be a task assigned to a single person. It did not work with cross-functional and self-organizing teams as propagated by agile methods. 

Architecture had to become a responsibility of the entire team. In the design of this transition, the agile coach was used as a role model. The architect also became a coach. Similar to the way the agile coach brings the agile methods closer to the team, the architect taught the tasks of architecture to the team over a longer period of time. Agility has thus completely redefined the role of this architect. The ideas and values of Agility change the way organisations work and this affects not only the architect.

You can easily find many more such examples. Agility has significantly influenced software architecture in recent years. Architecture trends such as Microservices, Continuous Delivery, or Evolutionary Architecture all try to bring the principles of agile software development into suitable forms of architecture. But agility is also clearly influenced by new developments in architecture. The consistent application of Conway’s Law and domain-driven design in the definition of teams is one example. Another one is the move away from fixed cadences towards a continuous management of tasks with methods like Kanban. 

This process of evolving together is no surprise. After all, agility and modern software architecture try to solve very similar problems. Both have to deal with a very complex environment, where the requirements for software are only completely known when the software is finished. Managing risks and adapting to changes is a central task of both the software architecture and the development process.

So even if it often seems that Agile methods and software architecture contradict each other, this is often only an indication that adaptation of one or the other is necessary. After more than 40 years of software architecture and more than 20 years of agility, these adaptations have significantly enhanced both disciplines. 

So is everything perfect now? Of course not. Many of the recent developments to agile and architecture are not commonplace yet. There is still a lot of communication to do. On the other hand, some of the concepts have already become unpopular. Mostly because they have been used as a solution no matter the problem. Agile has sometimes become more of a cult than a method. The same is true for Microservice architectures. They have seen a lot of backlash recently. People discover many situations where a classic monolithic approach is the better solution.

Looking forward we should focus more on why certain methods are needed in the first place. What is the benefit to the stakeholders of a software? We should expand our individual agile and architecture toolboxes to be able to choose the right tool for the situation. And we should look at agile methods and software architecture as two sides of a coin. Both are needed together to create the right software that fits the needs.