"The Irrelevance of Architecture" is the title of an IEEE Software paper by Grady Booch (May/June 2007). Is it a ploy to win mindshare among the anti-architecture crowd, or a Trojan horse? More the latter, I gather. I'm sorry to say I don't have the paper and I'm not about to pay $19 for a 2 pager, so I will conclude that this Trojan horse carries great insight (it is by Booch, after all). But I have to react to the synopsis, because it could reinforce a notion that architecture doesn't impact user experience, and that would be misleading. I realize that reacting to a synopsis is unfair, but I have a hammer and this looks like a nail.

In the synopsis, Booch says:

"The architecture of a software-intensive system is largely irrelevant to its end users. Far more important to these stakeholders is the system's behavior, exhibited by raw, working source code."

Even if the thing is a mess under the covers, users don't care so long as it behaves the way they need it to. (Booch said this, though more discretely.) Still, is that the same as saying that the architecture is irrelevant to users? That would be like saying car design, beyond external aesthetics, is irrelevant to car drivers. Now, go drive a car that accelerates like a Maserati but brakes like a Prius and see what you think of the architecture. If you aren't an architect, you may not call it that, but it's the jarring lack of consistency between qualities that you notice—if you live to. Users experience the architecture, they just don't know to call it that. Business managers experience the architecture indirectly too. They experience lack of agility, spending 80% of the budget to keep the lights on, whatever. So we have to recognize that what users and other stakeholders perceive, desire, care about may not be expressed in terms that they associate with architecture—and that's ok, as long as we do.

Let's take a scenario: The counter person at "ShipWithUs" has to go through and re-enter data to tell me the cost and delivery date for international priority shipping after he has just given me the rate and delivery date for international economy shipping. There can be a difference of one to three days and sometimes as little as $3 on a $75 transaction, so I care to know and make an informed choice. This not only costs me time, but because it is cumbersome for the "ShipWithUs" counter person to do, I get flack from them when I ask for the second price and delivery date before I commit to my shipping choice. How much better, if as a matter of course, the "ShipWithUs" person told me what the other faster or slower alternatives would cost, and what I would gain or lose in delivery time? Now, we have to ask, is that (missing) functionality architecturally significant? Does the business person stating the requirements need to ask for the option to cross-sell, up-sell, or whatever, or is it something the architect should bring to the attention of the business person, as a relatively high value-to-cost proposition? Alternately put, should we direct the fire-hose at the architect, or should we just shrug and say it fell through the cracks somewhere along the way, possibility between the business person expressing the requirements and the business analyst capturing them? If you are having your house architected and your architect puts the bathroom in the basement far from everything else, or even forgets a hall coat closet, is he at fault, or are you?

When we react to a building architecture with appreciation, it is not just the look but the feel that we are responding to. The feeling of light and space, the flow and ease of moving between spaces that need to be convenient to move between. The observable structures, yes, the function of the spaces, yes, and also the qualities like light, view, and energy efficiency. Building architecture is relevant to us in different ways, and as we think about it, we become conscious of more ways we just hadn't thought much about. Likewise with the architecture of software-intensive systems. And I'm not just talking appearance—what screens the "ShipWithUs" counter person sees is secondary to enabling the counter person to quickly and conveniently provide up-sell (from economy to priority international) or cross-sell (ground service versus express for domestic) information to the customer. I'm specifically not saying that screen design is architecturally significant. Unless it is, meaning unless it inhibits critical behaviors the user needs the system to support. System scope and countering scope-creep, yet staying flexible when business-critical functionality emerges, is a key part of the architect's balancing act.

Users and other stakeholders become acutely aware that the architecture is relevant as soon as the system becomes awkward, inconvenient, or worse, raises costs or loses revenue and opportunity for the business. So, yes, the point is that even if the thing is a mess under the covers, users don't care so long as it behaves the way they need it to. And yes, if the thing is a mess under the covers, it won't behave the way users want it to all the time. Perhaps not even much of the time. And fixing that, well, heroes will be the order of the day. Again. And again.

Yup, architecture is irrelevant to end users if they don't care what they get, when they get it, or how much it costs to get it. And it seems irrelevant to users when you truly get it right, when it is unobtrusive because the system just is right, delights, works. That's what Booch said in the paper, right? Next time I'm at the IU library, I'll have to look it up...

Yes, I acknowledge the synopsis contains these insights, but positions them on the flip-side to challenge us to think. And think I did. I expect the paper addresses these insights and more full-on, but I had fun writing my version.

As for my hammer? Well, you know I've been pounding the point that architecture is about design of the system, not just design of the structure that allows the design of the system to stand up to stresses and strains of the load placed on it. Architecture is about design of the user experience, not in the small like interior decorating, but in the large, like layout and flow, supporting function with qualities that delight, where it matters, and satisfice, where that is good enough. Architecture constrains and enables behavior, and architecture impacts properties of behavior.

"Architecture is important to both naturally occurring and overtly designed systems since architecture conveys behavior."

Edward Crawley, Olivier de Weck, Steven Eppinger, Christopher Magee, Joel Moses, Warren Seering, Joel Schindall, David Wallace and Daniel Whitney, "The Influence of Architecture in Engineering Systems," MIT esd, March 2004

3/5/08 Right System, Built Right

In the building architecture field, the architect designs the building—that is, the architect is responsible for the holistic view that balances functional needs and wants with properties or qualities within contextual constraints, and to do this, the architect works with the customer to understand goals, dreams, hopes, aspirations, frustrations, and the whole gamut of tangible and intangible "requirements." And the architect works iteratively with customers helping them to envision the building through various media to surface and clarify "requirements" and constraints, and create shared expectations. Although the architect does much of this before creating detailed structural design blueprints, the architect is taking her knowledge of structural design considerations into account, as she creates the customer-facing renderings that help the architect and customer communicate and iterate between intent and design.

One could design the user interface first and have that drive what the user and the system is able to do. One could design the structure, and fit the functionality to that form. A warehouse is a warehouse, a store is a store, and an app that serves dynamic web pages serves dynamic web pages, etc. Well, yes. And everyone wants the same black cars, right? There are lots of ways to unbalance a system, and if we carve up system design unnaturally, we will create imbalance.

"Form follows function - that has been misunderstood. Form and function should be one, joined in a spiritual union."  Frank Lloyd Wright

By designing function first, treating system envisioning/concept formulation/value definition as a separate discipline that precedes architecture, we neglect the all-important interplay between customer goals and concerns and what it will take to build the system. If we bring the architect in after system envisioning, to "reality check" system scope and feasibility, then we too often make the architect the nay-saying bad guy who is gated into bringing bad news about what it would take to build. Do this too often and we convey a message that IT or product development is not aligned with the business. And we have lost opportunities to innovate and add value along the way.

Structure specialists are needed in the building space, and likewise in the space of complex, demanding software-intensive systems. And system specialists are needed to ensure good, right and successful architecture—that is good: technically sound; right: meets stakeholders goals, needs, concerns; and successful: the architecture is built and delivering value. Alternatively put: architects are needed to ensure not only that our systems are built right, but that the right system is built.

[I have heard architects referred to as "generalists" who have areas of specialized skill that lends them insight and credibility. But I think of architects as specialists—in system design.]

3/7/08 Design Yourself, Then Design Your Own Curriculum!

I am looking (again) at curriculum design for architect development, and would sincerely value your input.

First, I would like to invite you to envision what you would like to accomplish, and then think about what would help you get there.

“The indispensable first step to getting the things you want out of life: decide what you want.”  — Ben Stein

Your personal vision: Where do you want to be in 2 to 5 years? What role would you like to be in, with what responsibilities and rewards (extrinsic and intrinsic)? What activities will you be doing? What would you like to have accomplished, in that time? How would you like to be viewed, by your peers, managers, and others you work and associate with? To reach these goals, what would you need to know, be able to do, and what personal characteristics would you need to build (on)? (That is, do a desired state interview with yourself!)

Your personal circle of excellence: Draw a circle. Think of a situation where you were really “on,” where you were at your best. What personal characteristics made you shine in that situation? Write these in your circle. Repeat, with another situation. Find your own stories of excellence, and your personal qualities that made you proud of yourself in that situation. These are your qualities, yours to use in other situations. When you are in a situation that is challenging, draw on your circle of excellence to remind yourself you have what it takes to shine in this situation. And these are the qualities you have to build on, as you expand your circle of excellence.

Your benchmark: Think of stellar architects, leaders, strategists, innovators, politicians, technologists you have worked with, and think of what capabilities and qualities made them effective.

You might find it helpful to use the chart at right, to think about where you are, and where you would like to be. The axes refer to the architect competency bands in the Architect Competency Framework below.

Alternatively, you might like to use Grove's Personal Compass.

Formulate your own personal development plan, and share it with me please (email: ruth at traceinthesand.com). We will keep it confidential, and only use it to review and shape our online materials and training offering. In short, it will enable us to better serve you.

Please ask other architects in your network of influence to do this. It will be valuable to them to reflect on their aspirations and path to achieving their career objectives, and I'd value their input. So, please point them to this journal entry: Design Your Own Curriculum (or http://www.ruthmalan.com/Journal/JournalCurrent.htm#Design_Your_Own_Curriculum). 

I would strongly prefer to hear what you believe will help you develop along your architect career path, rather than get your reactions to what I have proposed, though the latter is of value too. Anyway, I have shared my thinking (so far) below so that you get something in return for letting me know what you want for yourself.

(Draft) Proposed Curriculum

Understanding the "foundation" layer assumes you are aware of our Visual Architecting (VAP) workshops, and in particular that you understand that these were designed to weave a “full-encounter” experience that mimics the multi-dimensional nature of architecting in the real world, helping to stimulate a focus shift as well as providing useful techniques that help the architect along the path to good, right and successful architecture.

For the other development layers, we will figure out which pieces get delivered in which formats (reading, online training, workshops, community of practice formats like share/learn/do/reflect/debrief, team interventions, etc.) at a later point, once the topics have been firmed up.

 

	Technical Lead, Project Architect	Product Line, Domain or Portfolio Architect	Chief Architect, Enterprise Architect
Ground work	Project experience UML, RUP Design patterns	Architect experience VAP
Foundation	Architecture concepts and practices (VAP workshop, 4 days) Architect role and competencies (3 days)	Architecting across products: concepts and practices for product line and portfolio architects (4 days)	Case studies in excellence: leaders, strategists, innovators, and politicians
Good: technically sound	Technical specialist topics/ Patterns and practices: -   services -   security -   integration -   resource management -   scalability and availability -   decomposition and (re)factoring -   distribution and remoting - system health monitoring - business intelligence Conceptualization and visual modelling Architecture case studies	System thinking and system modelling Patterns and practices addressing multi-product/cross-system concerns: -   integration -   consistency and integrity -   leverage Product line (platform/family) case studies	Technical strategy Technology briefings  Architecting business capabilities
Right: meets stakeholder goals and concerns	System concept formulation and architectural requirements Creativity and innovation Technical Risk Management Trade-off analysis Design reviews	System envisioning and innovation Product and portfolio strategy Roadmaps, radars and dashboards Architecture assessments	Business strategy Strategy and innovation briefings
Successful: delivering strategic value	Giving and receiving feedback Technical communication Architect case studies	Group facilitation Leading teams Technical consulting Persuasion and influence, communication and effectiveness Dealing with politics, parochialism and anarchy (negotiation and conflict management)	Leadership: leading change, leading across organizations Leadership briefings Consulting with business leaders Organizational politics

This curriculum leverages our Architecture Competency Framework and Competency Elaborations, by Ruth Malan and Dana Bredemeyer:

• Architect Competency Framework (6 kb). 1 page summary of the architect's domains of competency (what you know, what you do, and what you are).  

• Leadership Competency Elaboration (21 kb). This module elaborates on the leadership competency domain in our Architect Competency Framework. 

• Strategy Competency Elaboration (19 kb). This module elaborates on the strategy competency domain in our Architect Competency Framework. You may be interested in our Strategy Primer for Architects. 

• Organizational Politics Competency Elaboration (25 kb). This module elaborates on the politics (or organizational effectiveness) competency domain in our Architect Competency Framework. 

• Technology Competency Elaboration (26 kb). This module elaborates on the technology competency domain in our Architect Competency Framework. 

To get a head-start with self-study, you may want to look at various resources related to these architect competencies.  And, of course, you could read back through the archives for this journal.

"They that won't be counseled, can't be helped."  Benjamin Franklin, 1758

Here are some blog posts on architect skills:

• Important Professional Skills for Software Architects, by Dan Vanderboom, February 2, 2008

• What is a Software Architect?, Ray King, July 4, 2007 (Technical practice, strategic practice and organisational awareness practice)

• Most Important Competencies of the Software Architect, Brian Sondergaard, June 1, 2007

• What is a Software Architect?, Ian McDowall, January 28, 2007

• Software Architects Skills, Angie Tan, July 4, 2006

• What Distinguishes the Software Architect?, Ruth Malan, May 31, 2006

• Architect Soft Skills, Arnon Rotem-Gal-Oz, August 10, 2005 and
  Architect Soft Skills: Part I, Introduction, July 26, 2006,
  Architect Soft Skills: Part II, Leadership, August 7, 2006
  Architect Soft Skills: Part III, System Thinking, August 17, 2006

See also: The Duties, Skills and Knowledge of Software Architects, Rick Kazman, Paul Clements, and Mark Klein

3/9/08 Capabilities Based Architecture

I read Denise Cook's "Business Capabilities Mapping: Staying ahead of the Joneses" (2007) in the IASA Architect Skills Library, and it reminded me not just of our "Enterprise Architecture as Strategic Differentiator" executive report, but also Stalk and Evans-Clark's seminal work in this area: "Competing on Capabilities: The new rules of corporate strategy," (Harvard Business Review, Mar/Apr 1992). This paper powerfully makes the case (without saying as much) that IT Does Matter, or at least IT can.

"Those who don't make dust, eat dust."

quoted in Peter Fuchs, Kenneth Mifflin, Danny Miller, and John Whitney, in "Strategic Integration in the Age of Capabilities," California Management Review, Sprint 2000.

3/9/08 Taking a Long View

I strongly encourage architects to view technology or architecture roadmaps as one of the defining deliverables of their role. That said, 

" Prediction is very difficult, especially about the future." Niels Bohr (physicist)

Here are some ideas to help build anticipatory awareness and practices, paying attention to trends and events in our market and technology space.

From Michael Watkins and Max Bazerman, "Predictable Surprises: The Disasters You Should have Seen Coming" (Harvard Business Review, March 2003):

Organizations are vulnerable to surprises in part because of our human predilection for harboring illusions that things are better than they are, giving weight to evidence that supports our preconceptions, giving more weight to maintaining the status quo and preferring to avoid some pain today than prevent a lot of pain tomorrow. Information and responsibility in organizations is fragmented across organizational silos, and this too makes organizations vulnerable to predictable surprises. To counter this propensity to be blind-sided by disasters the organization should have seen coming, Wakins and Bazerman encourage scenario planning and risk assessment:

Scenario Planning: Assemble a group of creative knowledgeable people from inside and outside the organization. Review company strategies, and identify external trends, critical business drivers and potential flash points. Based on this analysis, construct a plausible set of scenarios for potential surprises that could emerge over the next few years. Include scenarios that, though unlikely, could have a big impact.

Risk assessment: Identify risks, assess probabilities of these events, and estimate the costs and benefits of particular outcomes.

Is this relevant to architects? Well, we think it is, and utilize something along these lines in formulating risk strategies (part of Architecture Strategy a.k.a. Meta-Architecture in VAP).

"I often describe the life of a software architect as a long and rapid succession of suboptimal design decisions taken partly in the dark."

Philippe Kruchten, Common Misconceptions about Software Architecture, Rational Edge, 2001

3/9/08 Agile Architecting and the Wrecking Ball

Agile values drive iteration and validation activities in the Visual Architecting Process:

• Stakeholder participation:
  - to surface and validate requirements (stakeholder goals, concerns, hopes, frustrations, etc.)
  - to improve the architecture by bringing fresh perspective and experience to bear

• Iteration and incremental refinement:
  -- focus on “big rocks” first: the dominant uncertainties, the hard challenges or dominant complexities, the make-or-break areas
  - learn, improve and prove out critical aspects of the architecture  

• Refactor and continuously improve
  - "An architect's most useful tools are an eraser at the drafting board — and a wrecking bar at the site. " – Frank Lloyd Wright
  -
  -

For some counterpoint, see Tad Anderson's post on Agile. At least someone isn't kowtowing to the agile movement—even if I am. :-) 

3/10/08 Blogging Just Enough

I like "LouisD"s just enough cloud from his Playing the Just Enough Game blog post (August 2007). It's a great idea for organizing a blog around! Of course, Louis also got my attention with this quote from his first post:

"Architects should not be control freaks but innovation freaks," Louis D, Archi-Works blog, July 23, 2007

3/10/08 System Properties

• Modifiability Tactics, Felix Bachmann Len Bass Robert Nord, September 2007
• Architecture and the Quality Attributes Modifiability and Extensibility, Tad Anderson, September 27, 2007
• Utility Trees - Hatching quality attributes, Arnon Rotem-Gal-Oz, September 05, 2005

3/11/08 Market and Tech Watch

ITConversations series:

• Jon Udell's Interviews with Innovators series

• IEEE Spectrum Radio

Other communities:

• EnterpriseLeadership.org

• Social Innovation Conversations

Bits 'n pieces:

• How to Tap IT's Hidden Potential, Wall Street Journal, March 10th, 2008

• IT Alignment Trap: IT does matter, if you get IT right (video)

3/12/08 Architecture Workshops

We have two open enrollment classes coming up next month:

• Software Architecture Workshop: Chicago, IL, April 15-18, 2008

• Enterprise Architecture Workshop: Chicago, IL, April 21-24, 2008

There are still some places open in each, and it'd be great if you could help spread the word about the workshops. The next Architectural Leadership class is also tentatively set:

• Role of the Architect Workshop: Indianapolis, September 22-24, 2008

3/13/08 Software Architecture Action Guide

We're in the process of redrawing the Visual Architecting Process poster to more closely map what we've been doing. Here's the current draft:

As you can see from the poster, visualization and visual representations are integral to system and software architecture, not just building architecture. We still have some work to do on this, but it will give you an idea of where we're headed. I've decided that the Software Architecture Action Guide book stands between me and too many other projects, so we have to get that completed in the next few months. Please hold me to it! In the meantime, there's always the Software and Enterprise Architecture Workshops... the workshop materials have become books themselves!

3/14/08 Principled Leadership

Principles have been broadly adopted as a way to create alignment, allowing creativity and empowerment within the strategic framework created by these values-laden principles. We are seeing more examples among business leadership, and two great examples are the Patagonia Principles described by Yvon Chouinard, and the principles that Howard Behar used during his tenure as CEO of Starbucks.

In our technical communities we have been using principles broadly in enterprise architecture and software architecture, and it is proving to be a great way to communicate values within the technical community. The question then becomes, if our principles express our (technical) values, shouldn't we expect them to be stable for years? We want new hires to embrace these principles, and we want to hold our teams to them, even hold up a project if a team has ignored a principle (without any appeal to the governance board for a well-warranted escape clause)! Yet principles are a useful tool to express strategic choices, and we don't want to keep adding principles with each change in strategic direction. If we did that, we'd amass principles and fall foul of the minimalist architecture principle (see also guiding principles for enterprise architects)!

Ronald Fabel uses "strategy" in this way, when he says "One well-formulated strategy is worth 1000 quantifiable requirements." Along these lines, a shared understanding of (business, use, and system) context goes a long way to helping a team make choices that align with the business and architectural intent. Vision, context, strategy. Pound, pound. 

In Running an Architecture Principles Workshop, Craig Borysowich gives some good advice. I would add, though, that architecture principles are an expression of technical strategy, so it is perfectly justifiable to have principles that are directed at addressing key architectural challenges. The architect is a strategic leader; that is, she gets to set technical strategy. Yes, this must serve the business strategy and the business intent. If the architect decides a principle is needed to address a technical objective, challenge or risk, she gets to create that principle. Well, in my book she does. [pun intended]

Principles in architecture:

• SOA Principles website, Thomas Erl and SOA Principles Poster

• Architecture Principles, The Open Group

• Federal government architecture principles (2007)

• Principles template, Bredemeyer

Principles in software

• Interesting principles, Bala Dutt's blog

3/14/08 Monkey Business

A few days ago I was quite taken aback by the furor around the Lacy/Zuckerberg interview at SXSW. That "mind blind" state (Goleman) we've seen evidenced in flaming cow pies left in blog comments seems to have moved into twittering too. Either that, or it is those nucleus accumbens getting their punishment punch-up. At such times, I'm so, so glad that this journal is a quiet little backwaters place that attracts scant attention!

It'd help to be more careful with the stories we tell ourselves about other people and their motives. We need to recognize that we only know our own motives, and our interpretation of external "evidence" is shaped by our internal state. Our attitude shapes our outcomes; others may be involved too, but we can affect our own attitude. (John Maxwell's books aren't really my cup of soda pop, so I haven't read The Difference Maker, but I gather it makes that attitude point.) 

It does serve to remind me of some advice Rob Seliger (a leading architect of CCOW, part of the HL7 standard) gave us more than a decade ago: when you're addressing developers, start where they live, in the technical space; don't start with business concerns. Too bad Rob wasn't giving Sarah Lacy advice.

And while we're monkeying about, did you see this Dilbert and this blog post?

[If you look at some of the furor over Lacy here (and in the comments here), you'll see why my photo is just perfect! From time to time I'm tempted to tell some of the stories of sexism in our industry, but I usually regain my senses reasonably soon and send the piece to the outtakes bin; yup, such stories are generally stereotyped as whining, and that is so unpleasant. Now, I suppose, I could be damned for falling foul of a sexist ploy myself, except that not all of Lacy's detractors were men. That baboon could be a girl baboon... waugh, waugh... Grin. Ryan tells me there are people in his class (of 9 to 12 year olds) who think he has a "dry sense of humor." Gosh, I wonder where he gets that?]

3/16/08 Refactoring and Lean

Craig Cody's blog pointed me to a refactoring discussion on InfoQ (I do hang out on InfoQ a fair amount, but had overlooked this one, so thanks Craig). 

"Refactoring is one of the key technical practices in the Agile developer's toolkit.  Refactoring also has no measurable customer value by its very definition - it involves changing the structure (design) while maintaining the behavior.  In the Lean world - anything that does not have customer value is waste, and a customer only perceives behavior/functionality and not structure." 

Amr Elssamadisy, Opinion: Refactoring is a Necessary Waste, Dec 18, 2007

Perhaps you see why I reacted to Booch's irrelevance of architecture synopsis; it could (unintentionally) reinforce a misconception that is fairly broadly held. It has to do with an illusion that behavior and architecture are somehow separate concerns, and I sense that this is tied to an artificial separation between "functional" and "non-functional" requirements. But these are just different facets—any service has a "what it does" and a "how well it does it" (service level) aspect. A poorly designed system quickly erodes the "how well." Even when the "what" is being measured as the check box on user value delivery, the user notices, with less than delight, when system properties degrade.

Ok, so at what point is the system degraded enough to warrant "cleaning up"? By degraded, I mean in terms of system qualities that affect user experience directly (various dimensions of performance and reliability) and indirectly (system price tag and speed of getting access to new features), as well as the business ability to respond (development cost, time and predictability). Deferring quality (bugs, yes, but I'm really talking about kludgy structure here) at any point is just going to impact user experience and business agility down the road. We know that. In lean manufacturing, the factory floor is kept so clean you could eat off it. Why? Is it waste to keep the floors clean? Is it waste to ensure quality from the get-go, and at every point where quality can be improved, rather than inspecting for quality at the end? Lean manufacturing leans heavily on the total quality movement. The biggest waste of all are quality issues that cause delays at the end of the process, or worse, show up in the hands of the user.

So are we refactoring to simplify? to remove duplication? to remove complex or unnecessary dependencies? to improve performance (e.g., cleaning up a complex collaboration path)? to apply a pattern that is well-understood and easy to communicate? to make the system easier to extend? to make the system easier to debug? to make pieces of the system more useful in other contexts?   

"Yes" to some of these will show up in direct user experience, because user experience is affected by system properties (the "how well"). And some will show up in development efficiency (how much we can do with given resources) or agility (how quickly we can respond to the market). Sustaining innovations come from being able to incrementally improve the system, and a clean architecture allows for chunks of the system to be innovated on and improved at a different velocity from other chunks of the system. [You no doubt remember my "Trek" piece. The architecture allows, for example, Shimano to innovate on gears independently from Bontrager on frames.] Development efficiency shows up in future product or service cost, agility in new features, which all affects business competitiveness and future customer experience.

So, Craig, yes, you're right to be skeptical. It's a clever-sounding argument; beguiling even. But, I'd say, be wary. If your system is a one-off short-term special project that's one thing. But if it is to see your business through years of evolution, don't let the need to clean up get hidden under the covers too long. (I was just helping my son with his room, and you don't want to know what we found under his bed! No, really, you don't!)  

Remember, as architects and software developers we have TWO (classes of) voices to listen to: the voice of the customer and the voice of the business. The business cares about our attention to the voice of the customer, because that is how we compete in the near-term. But the voice of the business is also about the long-term viability of our business and the systems and products that support it.

And, if none of this persuades, focus on just one thing: in business after business, industry after industry, we hear "it costs 80% of our budget just to keep the lights on" (where "the lights" are the systems that undergird the operations of the business). Maintenance, or system evolution, matters. You could justify all the refactoring it takes, just to get that number down for the life of your system! And that is only one piece of the systemic economic puzzle. Structure matters!

Of course, I believe that this refactoring should be happening even before we get to code, by playing out system behavior (addressing the "what" and the "how well") through our models. Remember, architecture decisions are those that have a high cost of change (a key Booch insight), so we want to learn as much as we can, as cheaply as we can, as early as we can. That is, we want to do iterative modeling and improvement through stakeholder participation, and refactor responsibilities to achieve crisper, more balanced designs, or to address aspects or cross-cutting concerns more effectively. For larger-scale systems (where I mean code base, in this case, not transaction load), it becomes harder to refactor at the code level without impacting diffuse parts of the system, and that is typically error-prone. So, model, and test our models, create a skeletal architecture, test that, and so it goes. Start agile early, with architecture.

System behavior has to do with function, the properties (a.k.a. qualities) of that function, and the interaction of qualities across the system. In the Maserati-Prius example, there is an interaction between properties associated with the acceleration function, and properties associated with the braking function. If the car accelerates like a Maserati, it better brake like a Maserati, or there'll be times when the driver will face a life-threatening situation. The creators of the braking system may have nothing to do with the creators of the acceleration system (and may even be different external suppliers). But the architect has to ensure that the properties are consistent; that the architecture delivers behaviors with the intended properties and the system has integrity, so that properties of one subsystem are not inconsistent with those of another subsystem. System properties, or cross-cutting concerns, are among the factors that make architecture so crucial in complex systems.

If you don't like my Maserati example, try this: search delight site:www.ruthmalan.com on Google and on Yahoo! —yep, on Google the result is downright embarrassing! It is same function from a user perspective, right? With quite different results. So why do these results differ (in magnitude: 15 versus 3)? Is it a difference in the search algorithm, or a difference in trolling strategy, or something else? Is the difference architecturally significant?

Ok, now search deligt on Amazon, and on Google or Yahoo! or MSN. The search engines know you spelled it wrong, and offer a correction. Not Amazon. Is the omission of that piece of functionality architecturally significant? Is it strategically significant, in that Amazon loses revenue opportunity? I imagine it is! Any missed chance to give the customer what they are looking for (even if they don't know how to spell it) is significant to the business and hence, I claim anyway, it is architecturally significant! Architects need to be paying attention to what is happening in neighboring spaces, and if I'm in charge of search at Amazon, I want to be tracking what is happening in search at Google and Yahoo! But it doesn't end there; A9 (Amazon's search engine) gets that deligt is spelled wrong! And when one hand of the business doesn't know what the other is doing, isn't leveraging what the other is doing, that is architecturally significant! Ok, so maybe Amazon knows it is missing this piece of functionality and working on it.* Well, what does that tell you? Something about ease of extending the elephant? Incremental releases? I don't know. I've never worked with Amazon. And now, I probably never will. Oh no! Eh, don't worry, you're probably the only person reading this. Grin. A quiet, backwaters place...that will stay that way—I just pretty well ensured no "recommended essayist" accolades will come from Werner Vogels.

*Yeah, yeah, I do know about Conway's Law and if the architecture of the business creates silos these will be reflected in the architecture. But...

[3/17/8: A response to "The Three M's—The Lean Triad" (2/27/08 on InfoQ) directed me to Jim Womack's "Mura, Muri, Muda" e-letter. It makes an important point about the problem of focusing on "muda" (waste) exclusively. Roman Pichler argues "Value-creating activities enhance the product from a customer perspective. A good question to ask is: "Would I be happy to pay for this activity as the customer?" Another good question is, would I be willing to pay for this as the business manager, or architect. Because "the customer" is usually a euphemism for scores of customers, now, and into the future, for this product and its enhancements, and a set of variants in a portfolio or family of products (or services).

3/24/08: One might argue that the adaptability of software is both its greatest strength and its greatest curse. Adaptation and piecemeal growth means we don't have to know everything upfront—we can adapt the code to accommodate change. But typically the structure erodes as the system grows under schedule pressure to deliver value. This calls to mind comments about Netscape as well as Amy Tan's description of piecemeal growth in a traditional family home in China:

"The house had been in the family for many generations. It was not really so old or remarkable, but I could see it had grown up along with the family. There were four stories, one for each generation: great-grandparents, grandparents, parents, and children. The house had a confused look. It had been hastily built and then rooms and floors and wings and decorations had been added on in every which manner, reflecting too many opinions."

from The Joy Luck Club by Amy Tan p. 48.

The following piece from The Oregon Experiment (1988), by Christopher Alexander, has been influential in software:

"Large-lump development is based on the idea of replacement. Piecemeal Growth is based on the idea of repair. … Large-lump development is based on the fallacy that it is possible to build perfect buildings. Piecemeal growth is based on the healthier and more realistic view that mistakes are inevitable. … Unless money is available for repairing these mistakes, every building, once built, is condemned to be, to some extent unworkable. … Piecemeal growth is based on the assumption that adaptation between buildings and their users is necessarily a slow and continuous business which cannot, under any circumstances, be achieved in a single leap."

Piecemeal growth, or incremental development, is not just desirable but a fact of life in software (even big-bang first releases are evolved thereafter). Even so, we need to build more learning into our process. More learning when it is cheaper to find and fix problems with the vision (doing course corrections toward "right system") and structure (built right). Then, accepting that we will continue to learn and evolve our system, we need to invest in fixing the mistakes—incrementally adding functionality yes, but repairing structural defects too. This investment is the crucial dual to piecemeal growth that we too often forget in software. When we keep marching to a frenetic "add value" drumbeat, we get into a situation where the system threatens to crumble under the mass of deferred structural issues (like the Minneapolis I-35W Mississippi River Bridge that collapsed last year). This is what Grady Booch reports is done at eBay, though I can't point to any organization that does this systematically (even if I wasn't bound by NDAs).

Principle of simplicity: eBay sets aside some of development budget to simplify. This makes the architecture more maintainable, less brittle, more resilient to change.

from my notes from Grady Booch's Yahoo! talk (May 2007)] 

3/16/08 Strategy Dynamics

Rajiv Mistry pointed us to Strategy Dynamics. It certainly looks like something worth spending time with—thanks Rajiv.

3/17/08 Roadmaps or Projections

Dana Bredemeyer has been using the word "projections" instead of "roadmaps" for our exploration of trends and scenarios. I can see his reasoning, but I'm okay going with the flow. Whatever you want to call it, as long as you put some cycles there from time to time, I'm happy my job has been done.

What we want to do is map out our product release plan, our own internal development plans for infrastructure (platform) development, our plans for capability development, what we know of our vendors' release plans for technologies we depend on and so forth. We might call this our roadmap, though strictly speaking even this is projection. Still, these are intentional, and though obviously not deterministic, we will invest in making this future unfold, so it is a more likely future than most other futures we could speculate about.

But then we also want to overlay these plans with projections about how current trends might unfold. So we have moved beyond "product roadmap" turf into more reasoned projections and educated guesses about the future of the markets we compete in and the technologies that might well (re)shape these markets.

Why? Because our architecture has to see us through that world of tomorrow—when the system will be fielded, and evolved. Today's world may well be a better predictor of that world than any other prediction we might come up with. But we still need to think about what the future is likely to bring. Even if we had a perfect crystal ball, we couldn't build all of the functionality we will come to need into our first release—if we want to release in competitive timeframes, anyway. And we don't have crystal ball, so we can't build to meet every contingency. But it also doesn't make any sense to act like an ostrich, bury our head in the sand, and ignore likely threats and opportunities. Not if we want to be great. Remember—foresight!

"Programming is the act of deciding now what will happen in the future."

James Coplien and Neil Harrison, Organizational Patterns of Agile Software Development, 2004.

When my daughter runs full-speed down long, steep stairs, I am reminded that some people are simply more prescient than others. In part, it is the school of hard knocks, and in part it is a tolerance for the ambiguity we face dealing with fuzzy futures. But the architect, as leader into the future, has to do some scouting out of that future terrain she's leading her team into, and we've found that roadmaps, projections and scenarios help the architect take a long view. Not to attempt take care of every detail of tomorrow today, for that is so futile as to be a sign of insanity! We carry a spare tire in the car, but not a spare battery. It is a matter of finding the balance, for our project.

"Think like a person of action, and act like a person of thought."

Howard Behar, It's Not About the Coffee, 2007

We have treated group (graphical) history and roadmaps/projections as two separate activities and artifacts. But working with groups across time, I've started to realize that having a rolling horizon, where today's present is updated with events as they unfold, and become part of the tracked history, is useful. Of course, I'm talking only about keeping architecturally significant events on our rolling radar. So, Vista was announced. The Vista delay was announced. (This was earth shattering enough that I remember when and where I was! Indeed, I was facilitating an architecture workshop in the Indianapolis Marriott—the one where Greg Russell impressed me as an all-round outstanding person and resonant architectural thinker.) Vista was released. ... If your products or applications were impacted, how did you track and make visible the values, dependencies, and risks associated with these unfolding plans, resets, and events? 

With regard to projections, let me just say Dana was an unusual (Indiana University) computer science student in that he scouted out and spent a summer in Philadelphia studying under Russell Ackoff, the systems-thinking (anti-)guru. He has since spent 3 decades in the software industry, but that early influence was formative and Dana had already earned the title of architect at HP in 1987. Anyway, "projection" has Ackoffian residue.

[5/8/08: There's a nice example of a Graphical History done by Dave Sibbet of Grove here (Evolution of the Internet) and here (HBR Management Ideas and Practice). See also National Semiconductor's vision here and VizThink's Context Map here.]

3/18/08 Structure and Behavior: Take 3 (or is it 4?)

Since it is "Brain Awareness Month", it is interesting to note that much the same structure (the human brain) can give rise to very different behaviors (yours versus mine, for example).

I have also been thinking, prompted by Booch's "architecture is irrelevant" and Amr Elssamadisy's "refactoring is waste" pieces, about how quite different structures can support largely similar behaviors. This is most striking at the poles: kludgy code versus well-structured code. So we can make statements like "it is largely irrelevant to the user whether our architecture is service oriented or not." But such statements are generally true only within a range of assumptions about required behaviors. This became clear to me early on, when we wanted to interrupt a long print job to insert a high priority rush job, and that couldn't be done with the pipe-and-filter architecture that was the dominant design for printer firmware. This new "use case" (or variation on a use case) couldn't be accommodated by the existing architecture, and required a fundamental redesign if we were to elevate this piece of behavior to "must" status in our feature scope. For a narrow range of assumptions about required behavior, we can accommodate that behavior with a variety of structures—from structures that are highly tuned up and specific to the behavior, to structures that are more flexible and could support more variance or spread in behaviors. Indeed, it is when we throw variance at the architecture, due to evolving needs over time, or different product variations at a point in time, that the structure becomes very important—even for relatively simple systems. Add to that demands like (operational) scalability and reliability, and structure really factors.

"Not only are a system’s desired operating modes influenced by its architecture, but so are some of its failure modes. Thus an architecture that permits only one path between elements may fail if a leg of any path breaks. All of a tree below a broken node is isolated from the rest of the tree."

Edward Crawley, Olivier de Weck, Steven Eppinger, Christopher Magee, Joel Moses, Warren Seering, Joel Schindall, David Wallace and Daniel Whitney, "The Influence of Architecture in Engineering Systems,"
MIT esd, March 2004

A system could be coded in such a way that it is very tightly bound to a particular workflow, forcing diffuse changes to the code to accommodate variations in the workflow. Duh! But it makes the point that structure can bound the range of possible system behavior in ways that become quite obtrusive and frustrating to the business as it seeks to adapt to changes in the competitive landscape. If we don't throw some variation at the architecture, assessing the impact using interaction models, thought experiments and reasoned arguments, we can be blinded to coupling to the particular workflow "the" customer has expressed in the requirements. But I already reacted to invoking YAGNI at every turn. The architect has to use judgment to balance exigency and structural robustness. Use lightweight, "Pareto" tools (even models on flip charts or whiteboards) and keep track of assumptions and rationale along with your decisions. Apply that uncommon common sense. Time to move on.

"Common-sense in an uncommon degree is what the world calls wisdom." Samuel Taylor Coleridge

3/18/08 Organizing for Architectural Excellence

References:

• Melvin Conway, How Do Committees Invent?, Datamation, 1968

• Manuel E. Sosa, Steven D. Eppinger, and Craig M. Rowles, "The Misalignment of Product Architecture and Organizational Structure in Complex Product Development," Management Science, Volume 50 , Issue 12,  December 2004

• Rosaline Gulati, The Coupling of Product Architecture and Organizational Structure Decisions, MIT masters thesis, 1996.

• Todd Veldhuizen, SE362 Lecture 7: Work Breakdown Structures, May 16, 2007 [Architecture implications for project manager and Conway's Law.]

• The Organization and Architecture of Innovation: Managing the Flow of Technology, by Thomas J. Allen, Gunter Henn, 2006

• Architecture Governance, The Open Group

• "Impact of service-oriented architecture on enterprise systems, organizational structures, and individuals," by N. Bieberstein, S. Bose, L. Walker, and A. Lynch, IBM Systems Journal, Vol44#4, 2005.

3/18/08 Architecture and Agile

My commentary/essays:

• Agile architecting and business agility

• Architecture and the Agile Quest

Scott Ambler's commentary/essays: Agile Architecture: Strategies for Scaling Agile Development

If you read Scott Ambler's Agile Architecture: Strategies for Scaling Agile Development, you'll see that the piece has progressed a fair amount—at least it has since I read it a couple of years ago. He still has that common practice/agile practice table that I find offensive—Scott says it is common practice for architects to place themselves on pedestals. Common practice? Hmmpf! And he's updated his references and suggested reading and doesn't recommend this journal. Hmmpf again. Oh, yeah. I don't like that he sets up architects for target practice. OK, Scott Ambler is doing good, important work that will make a big difference in software engineering because he appeals to a broad base. But context matters, and some projects are vastly more complex than others, and some architecture efforts are strategic and broad scoped, so not everything should be tarred with the same brush.

One point worth pondering: the deeper the team goes on detailed concepts, the higher the cost of changing the overall concept. The more quickly we can expose the overall concept to tests of desirability and fit (determining whether this is the "right" system), the more confidently we can launch teams to work concurrently on different components or services. Remember, development concurrency (more teams working in parallel) is the path to "agility," meaning speed to market with a new or enhanced product or service, for larger, more complex systems.

We can cover good distance by simply asking ourselves what kinds of requirements changes we are concerned about:

• are the "requirements" well understood? does "the" customer know what they want? are we inventing function?

• is the environment stable or changing?

We want to get a handle on our dominant uncertainties, just as we want to get a handle on our dominant complexities. The first helps us with "right system," the second with "built right."

Once we have a better sense of where our uncertainties lie, we can formulate strategies for resolving them. It is generally a good idea to figure out how to deliver increments of value that build out the vision, give us opportunities to refine and elaborate the vision, and even to ditch the vision and start afresh, if that is what it will take to ultimately be successful. But we can learn so much from rapid modeling cycles with active stakeholder involvement, that we can get rapid and cheap course correction even before we launch teams on sprints, or other incremental/evolutionary development process styles.

This paper (from systems engineering) is interesting: "Agile SYSTEMS ENGINEERING versus AGILE Systems engineering," by Reinhard Haberfellner and Olivier de Week, INCOSE 2005.

3/19/08 Animals in Translation and Bad Smells

Dana picked Animals in Translation back up and was telling me another story from it; this time about the way that Temple Grandin audits slaughter houses. She says that while most auditors audit documents, she audits the animals. You can change the documents to mislead, but animals don't mislead. Why inspect for dirty floors, because problems with filth will show up in the animals? And so forth. She has a small set of questions about the animals, that are key questions that reveal answers to many questions. And she looks at direct evidence.

This reminded me of Booch's questions that he uses to assess "how a project smells" when he goes in to do an architecture archeological dig:

• tell me about your rhythm of releases, do you have a regular release strategy? (If not, the project is chaotic.)

• what is the frequency of releases? (If it's every 6 months, it's not fast enough; if it's every day, that is stupid because it never gives an opportunity to step back and make it simple—the pace to introduce features is just too frantic)

• do you have an architect? (If there is an architecture board with representatives from each of groups that is a good sign; the wrong way to set things up is to have an architecture group that is not associated with any running projects, that is disjointed from real problems)

• do you have a culture of patterns?

from my notes from Grady Booch's Yahoo! talk (May 2007); the wisdom is Booch's, any mistakes in transcription are mine

The "smells" language reminds me of James Navarro's advice to me in my early days at HP. Of course, it also reflects the "bad code smells" language that Kent Beck apparently introduced into our parlance. Some of the smells in Mika Mäntylä's Taxonomy of "Bad Code Smells" apply just as well to architecture—we apply heuristics along these lines looking at responsibilities of components in conceptual architecture, and again as we delve further in logical architecture, designing architectural mechanisms and key interfaces. Of course, these are along the lines of Booch's guidelines: