Agile meets Architecture: A Project and Program Perspective

I’m a big fan of evolutionary architecture—mostly because I’ve never seen a planned architecture fully work. Every time I’ve tried—as a developer, project manager, or program manager—Something Happened near the end of the project. That Something made the best-planned architecture crash and burn. I can’t count the hours, days, or weeks I spent trying to make that architecture work. 

However, the more I use agile approaches, the more I see the need to have a guide to shepherd the business value of the product. 

Who should be that guide? Is it one person? A committee? (No.) How do we create just enough architecture as we learn what the product should be? 

How can we support the idea of agile meeting architecture?

I’ve had a few successes. Not enough successes to know that these ideas work for everyone at all times. However, enough successes that I’ve created guidelines for my clients. These guidelines work for both projects and programs.

Sidebar: Here are my definitions of project and program:

An agile project is a container of some number of related problems we want a product to solve. We expect to need not more than about fourteen people, organized in small teams, to deliver those solutions. This is a project because it has risks. 

An agile program is a strategic collection of multiple projects that deliver a business outcome. That outcome often takes the form of a product. The program requires people and teams across the organization to deliver that business outcome.

The guidelines I’ve seen work for agile meeting architecture:

• Focus on business value.
• Ask the architects to focus on exposing risk.
• Encourage the architecture effort to be a social activity.

Let’s start with a focus on business value.

Focus on Business Value

A colleague once said to me, “Business value is like quality. I’ll know it when I see it.”

I disagree. We can describe business value in architecture in at least these ways:

• Easy to change, so we can change the product itself. Also, how easy is it to change the tests, or how the team assesses the changes? 
• Fast to change, so we can gain some feedback, internally or externally.
• Enable a customer to perceive the new product value.

Not all architecture changes can be easy and fast. However, if we think about those qualities as we start the work, we can ask these questions:

“Which future possibilities does this particular decision remove? Which future possibilities does this decision encourage?”

We might decide to remove future possibilities in the interest of releasing something useful sooner. We might not. We can decide, based on the project’s or program’s risks and needs.

Each architectural decision creates, reduces, or exposes some risk. As a project or program manager, I want to know about risks earlier, not later.

Expose Risk

Since I expect architects to be part of a collaborative cross-functional team, I often see architects regularly discuss risks with their team.

Sometimes, architects discuss local risks—risks to this feature or feature set. Sometimes, since we ask architects to take a big-picture perspective, the risks cross the product.

I like to use spikes of various kinds throughout the project or program to reduce the architectural risks. (Spikes are short timeboxes where the entire team works to understand more about a particular area of the product to inform the future work.)

I’ve used different kinds of spikes differently throughout the effort: in research or at the start of a project; and when we encounter some kind of architectural difficulty.

Research or Early Spikes

Some organizations create R&D “projects.” Those projects are rarely released to actual users. Instead, the project tends to be between one week and three months in duration. The output of that effort informs the decision-makers about the actual risks for the project or program.

Some organizations use the start of the actual project or program to create the spikes. 

The reason for the spikes is the same. We know the least about the effort. We expect the requirements to change as we learn. We want to have a little foresight into where the most architecturally challenging parts of the work will be.

In my experience, almost all projects start with a proposed architecture. We might not know all the details about that architecture, but we start somewhere. Then, we can use feature-based spikes. 

When we use features, we test the proposed architecture as we work, reducing risks. 

And, if the entire team works with the architect, we reduce the risk that the team doesn’t understand what the architect proposes.

What if some other team already spiked several features? As a project/program manager, I review the changes in requirements since that architectural effort. 

I often discover that someone added more features or changed the roadmap further out. When that occurs, I assume the previous architectural spikes were useful but not sufficient. I often ask the team(s) to define what they would like to investigate for one week. Then, I work with the product owner(s) to make sure we investigate those areas sooner, not later.

We now know a little more than we did before. Probably enough to start the project/program in earnest. 

Risk During the Project/Program

As I said at the start, Something Happened that required an architectural change on every project or program I’ve worked on.

Sometimes, that means I ask the product owners to maintain an architectural kanban board to determine the information we might need to inform our product work in the future.

An architectural kanban is a list of spikes we might need in the future. Each spike allows us to learn so we can reduce risk later.

And, sometimes, we don’t have an architectural kanban we can use to pull work to reduce risks. We might have thought we covered our risks, and then Something Happened.

That’s when I ask the people—all the architects and teams—to collaborate to discover these answers:

• How frequent is this scenario? Do we need to address this particular problem now, for this release? Is this something we could postpone when we’re not trying to complete this version of the product?
• How valuable is this scenario? Would adding this scenario make life easier for the people who use the product or test or install the product? Would we sell to more customers or retain more customers if we added this?
• What choices do we allow? What choices do we remove?

Once we have a proposed architecture, I also like to add these questions:

• What problems do we solve with this approach?
• What problems do we create?

Once we understand the relative value of the proposed architecture, we can discuss what to do and when. I prefer to learn with spikes, where possible.

We all learn more—and earlier—when we socialize the architecture.

Socialize the Architecture

When we use agile approaches, we have many opportunities to share what we know with others about everything in the project/program, including architecture. In fact, the named architect’s greatest value might be the person who can socialize the architectural issues. 

One of the most important socialization aspects is around performance and security. We’ve all attempted to use products that didn’t have sufficient performance. And, by now, all of us have experienced data breaches. We need other architectural characteristics for our products, but I think of performance and security as the biggest risks.

If we have one named architect, I like to ask that person to ask these questions:

• How does this scale from 3 to 300 to 3,000 to 30,000 to 300,000 to 3,000,000 and more users? (You might need to substitute transactions or threads for users.)
• How will bad actors use this feature or characteristic?
• How will well-meaning, but ignorant, users misuse this feature or characteristic?

When architects ask these questions during the entire project or program, they coach the rest of us to think of these architectural risks.

Architect as Shepherd

Some architects tell me they can envision the big picture of the system at the start. They claim they can create that architecture and it succeeds. 

Most of us realize the original big picture will change and want to create a more agile approach to creating that big picture. 

And, some of us—and I count myself among these people—can’t envision that big picture even halfway through the effort. We can’t see all the possibilities and the potential risks at the start.

That’s why an agile approach to architecture works for most of us. 

We're more likely to learn when we focus on business value, expose risks, and socialize what we do and don’t know. That learning helps us iterate and increment over a feature set, not to get it right the first time, but to get it more right over time.

And that’s the point of an agile approach—to use what we know now to inform our next set of work to create products and services our customers will love.