A Trace in the Sand
Online Architecture Journal by Ruth Malan
I also write at:
Trace in the
3/1/08 Minimalist—in this Moment
It is so nice to start with my traces in the sand wiped clean each month!
If you're just tuning in, these notes are first for me, to keep track of my thinking (no-one else will write the Cliff notes on my thoughts), things I read, and so forth. Second, these notes are for architect-explorers who navigate outside the box, reaching to be great. Architects who recognize that innovation and strategic contribution, and leadership and personal effectiveness, are as much keys to excellence in architecture as structural design elements. Not one without the other, but one in service of the other.
3/1/08 Ambition to excel
"I would to God there were more ambition in the country," Adams said. Then he paused and added, "Ambition of that laudable kind, to excel."
John Adams, as quoted by David McCullough in "Timeless Leadership," Harvard Business Review, March 2008
3/2/08 Ambition to Delight
Wouldn't it be nice if the ambition of a company was to strive to delight especially the customer, but also employees, in all its endeavor? A rallying call to every person at every point in the organization to figure out what delight means and do that, and what quells delight, and not do that. I've been looking at what Branson and Virgin are doing, and I'm delighted that they are taking corporate ethics and environmental responsibility seriously! And Virgin's attitude to customers ("We view our customers’ happiness as the lifeblood of Virgin’s success") shapes the customer's experience. Yes, delight is a many-faceted thing, and hard to achieve. Yet the striving is engaging. Engaging to the workforce, and engaging to the consumer.
"The greatest use of life is to spend it for something that will outlast it." William James
Don't we all want, in some small or big way, to have made an impact on the world? But how shameful if the impact is to destroy it!
Yvon Chouinard expresses dire depression at the world's outlook, but I'm so encouraged by leaders who are making personal sacrifices, shifting their lives, devoting time, not just excess wealth, and taking a personal stand to do good in the world. Bill and Melinda Gates, Richard Branson, Bill Clinton, Al Gore and many more. These people, and all the millions and millions of people working at grassroots levels, who led the leaders, who lead their communities, they give me hope. Wouldn't it be great if we all, every one of us, took it upon ourselves to leave this planet better off than it was when we entered it?
Google's "don't be evil" Code of Conduct brings them some flack because, even for those who really work at it, it is just neigh impossible to do no evil and keep the lights on. Yet, Google is trying, using solar energy to cover about 30% of their energy needs. Following their lead, HP and Wal-Mart are also investing in renewable energy, and specifically solar. As far as environmental impact goes, yes, even our small business has a high carbon footprint because of all the travel we do. Patagonia's mission to "do no unnecessary evil" is laudable in its honesty. Patagonia strives in all its endeavors to reduce environmental impact, even pioneering fleece from recycled soda bottles. Still, they're in the business of selling manufactured goods and that has an impact up and down the supply chain, so they also "tax" themselves 1% of sales which they give mainly to environmental causes. Individuals and corporations are taking a hard look at themselves, and asking how to shift the rising tide of environmental destruction.
3/5/08 High Ambitions
High Ambitions in the Himalaya is a movie by Curt Dowdy. It is showing at the Cinequest film festival in San Jose this week (today, Wednesday, March 5 at 4:45pm, and Saturday March 8 at 2:15pm). If you're in the Bay Area this week, I highly recommend it. It is a movie that speaks to anyone who strives for something hard to reach, and finds that there is real value is in the striving. It is a beautiful movie, in terms of the scenery and music but also in terms of the human lessons, honestly told. And you don't have to take my word for it—this movie won the Cinequest Vuze audience award.
3/5/08 The Irrelevance of Architecture
"The Irrelevance of Architecture" is the title of an IEEE Software paper by Grady Booch (May/June 2007). Is it a ploy to win mindshare among the anti-architecture crowd, or a Trojan horse? More the latter, I gather. I'm sorry to say I don't have the paper and I'm not about to pay $19 for a 2 pager, so I will conclude that this Trojan horse carries great insight (it is by Booch, after all). But I have to react to the synopsis, because it could reinforce a notion that architecture doesn't impact user experience, and that would be misleading. I realize that reacting to a synopsis is unfair, but I have a hammer and this looks like a nail.
In the synopsis, Booch says:
"The architecture of a software-intensive system is largely irrelevant to its end users. Far more important to these stakeholders is the system's behavior, exhibited by raw, working source code."
Even if the thing is a mess under the covers, users don't care so long as it behaves the way they need it to. (Booch said this, though more discretely.) Still, is that the same as saying that the architecture is irrelevant to users? That would be like saying car design, beyond external aesthetics, is irrelevant to car drivers. Now, go drive a car that accelerates like a Maserati but brakes like a Prius and see what you think of the architecture. If you aren't an architect, you may not call it that, but it's the jarring lack of consistency between qualities that you notice—if you live to. Users experience the architecture, they just don't know to call it that. Business managers experience the architecture indirectly too. They experience lack of agility, spending 80% of the budget to keep the lights on, whatever. So we have to recognize that what users and other stakeholders perceive, desire, care about may not be expressed in terms that they associate with architecture—and that's ok, as long as we do.
Let's take a scenario: The counter person at "ShipWithUs" has to go through and re-enter data to tell me the cost and delivery date for internationalpriority shipping after he has just given me the rate and delivery date for international economy shipping. There can be a difference of one to three days and sometimes as little as $3 on a $75 transaction, so I care to know and make an informed choice. This not only costs me time, but because it is cumbersome for the "ShipWithUs" counter person to do, I get flack from them when I ask for the second price and delivery date before I commit to my shipping choice. How much better, if as a matter of course, the "ShipWithUs" person told me what the other faster or slower alternatives would cost, and what I would gain or lose in delivery time? Now, we have to ask, is that (missing) functionality architecturally significant? Does the business person stating the requirements need to ask for the option to cross-sell, up-sell, or whatever, or is it something the architect should bring to the attention of the business person, as a relatively high value-to-cost proposition? Alternately put, should we direct the fire-hose at the architect, or should we just shrug and say it fell through the cracks somewhere along the way, possibility between the business person expressing the requirements and the business analyst capturing them? If you are having your house architected and your architect puts the bathroom in the basement far from everything else, or even forgets a hall coat closet, is he at fault, or are you?
When we react to a building architecture with appreciation, it is not just the look but the feel that we are responding to. The feeling of light and space, the flow and ease of moving between spaces that need to be convenient to move between. The observable structures, yes, the function of the spaces, yes, and also the qualities like light, view, and energy efficiency. Building architecture is relevant to us in different ways, and as we think about it, we become conscious of more ways we just hadn't thought much about. Likewise with the architecture of software-intensive systems. And I'm not just talking appearance—what screens the "ShipWithUs" counter person sees is secondary to enabling the counter person to quickly and conveniently provide up-sell (from economy to priority international) or cross-sell (ground service versus express for domestic) information to the customer. I'm specifically not saying that screen design is architecturally significant. Unless it is, meaning unless it inhibits critical behaviors the user needs the system to support. System scope and countering scope-creep, yet staying flexible when business-critical functionality emerges, is a key part of the architect's balancing act.
Users and other stakeholders become acutely aware that the architecture is relevant as soon as the system becomes awkward, inconvenient, or worse, raises costs or loses revenue and opportunity for the business. So, yes, the point is that even if the thing is a mess under the covers, users don't care so long as it behaves the way they need it to. And yes, if the thing is a mess under the covers, it won't behave the way users want it to all the time. Perhaps not even much of the time. And fixing that, well, heroes will be the order of the day. Again. And again.
Yup, architecture is irrelevant to end users if they don't care what they get, when they get it, or how much it costs to get it. And it seems irrelevant to users when you truly get it right, when it is unobtrusive because the system just is right, delights, works. That's what Booch said in the paper, right? Next time I'm at the IU library, I'll have to look it up...
Yes, I acknowledge the synopsis contains these insights, but positions them on the flip-side to challenge us to think. And think I did. I expect the paper addresses these insights and more full-on, but I had fun writing my version.
As for my hammer? Well, you know I've been pounding the point that architecture is about design of the system, not just design of the structure that allows the design of the system to stand up to stresses and strains of the load placed on it. Architecture is about design of the user experience, not in the small like interior decorating, but in the large, like layout and flow, supporting function with qualities that delight, where it matters, and satisfice, where that is good enough.Architecture constrains and enables behavior, and architecture impacts properties of behavior.
"Architecture is important to both naturally occurring and overtly designed systems since architecture conveys behavior."
Edward Crawley, Olivier de Weck, Steven Eppinger, Christopher Magee, Joel Moses, Warren Seering, Joel Schindall, David Wallace and Daniel Whitney, "The Influence of Architecture in Engineering Systems," MIT esd, March 2004
3/5/08 Right System, Built Right
In the building architecture field, the architect designs the building—that is, the architect is responsible for the holistic view that balances functional needs and wants with properties or qualities within contextual constraints, and to do this, the architect works with the customer to understand goals, dreams, hopes, aspirations, frustrations, and the whole gamut of tangible and intangible "requirements." And the architect works iteratively with customers helping them to envision the building through various media to surface and clarify "requirements" and constraints, and create shared expectations. Although the architect does much of this before creating detailed structural design blueprints, the architect is taking her knowledge of structural design considerations into account, as she creates the customer-facing renderings that help the architect and customer communicate and iterate between intent and design.
One could design the user interface first and have that drive what the user and the system is able to do. One could design the structure, and fit the functionality to that form. A warehouse is a warehouse, a store is a store, and an app that serves dynamic web pages serves dynamic web pages, etc. Well, yes. And everyone wants the same black cars, right? There are lots of ways to unbalance a system, and if we carve up system design unnaturally, we will create imbalance.
"Form follows function - that has been misunderstood. Form and function should be one, joined in a spiritual union." Frank Lloyd Wright
By designing function first, treating system envisioning/concept formulation/value definition as a separate discipline that precedes architecture, we neglect the all-important interplay between customer goals and concerns and what it will take to build the system. If we bring the architect in after system envisioning, to "reality check" system scope and feasibility, then we too often make the architect the nay-saying bad guy who is gated into bringing bad news about what it would take to build. Do this too often and we convey a message that IT or product development is not aligned with the business. And we have lost opportunities to innovate and add value along the way.
Structure specialists are needed in the building space, and likewise in the space of complex, demanding software-intensive systems. And system specialists are needed to ensure good, right and successful architecture—that is good: technically sound; right: meets stakeholders goals, needs, concerns; and successful: the architecture is built and delivering value. Alternatively put: architects are needed to ensure not only that our systems are built right, but that the right system is built.
[I have heard architects referred to as "generalists" who have areas of specialized skill that lends them insight and credibility. But I think of architects as specialists—in system design.]
I am looking (again) at curriculum design for architect development, and would sincerely value your input.
First, I would like to invite you to envision what you would like to accomplish, and then think about what would help you get there.
“The indispensable first step to getting the things you want out of life: decide what you want.” — Ben Stein
Your personal vision: Where do you want to be in 2 to 5 years? What role would you like to be in, with what responsibilities and rewards (extrinsic and intrinsic)? What activities will you be doing? What would you like to have accomplished, in that time? How would you like to be viewed, by your peers, managers, and others you work and associate with? To reach these goals, what would you need to know, be able to do, and what personal characteristics would you need to build (on)? (That is, do a desired state interview with yourself!)
Your personal circle of excellence: Draw a circle. Think of a situation where you were really “on,” where you were at your best. What personal characteristics made you shine in that situation? Write these in your circle. Repeat, with another situation. Find your own stories of excellence, and your personal qualities that made you proud of yourself in that situation. These are your qualities, yours to use in other situations. When you are in a situation that is challenging, draw on your circle of excellence to remind yourself you have what it takes to shine in this situation. And these are the qualities you have to build on, as you expand your circle of excellence.
Your benchmark: Think of stellar architects, leaders, strategists, innovators, politicians, technologists you have worked with, and think of what capabilities and qualities made them effective.
You might find it helpful to use the chart at right, to think about where you are, and where you would like to be. The axes refer to the architect competency bands in the Architect Competency Framework below.
Alternatively, you might like to use Grove's Personal Compass.
Formulate your own personal development plan, and share it with me please (email: ruth at traceinthesand.com). We will keep it confidential, and only use it to review and shape our online materials and training offering. In short, it will enable us to better serve you.
Please ask other architects in your network of influence to do this. It will be valuable to them to reflect on their aspirations and path to achieving their career objectives, and I'd value their input. So, please point them to this journal entry: Design Your Own Curriculum (or http://www.ruthmalan.com/Journal/JournalCurrent.htm#Design_Your_Own_Curriculum).
I would strongly prefer to hear what you believe will help you develop along your architect career path, rather than get your reactions to what I have proposed, though the latter is of value too. Anyway, I have shared my thinking (so far) below so that you get something in return for letting me know what you want for yourself.
Understanding the "foundation" layer assumes you are aware of our Visual Architecting (VAP) workshops, and in particular that you understand that these were designed to weave a “full-encounter” experience that mimics the multi-dimensional nature of architecting in the real world, helping to stimulate a focus shift as well as providing useful techniques that help the architect along the path to good, right and successful architecture.
For the other development layers, we will figure out which pieces get delivered in which formats (reading, online training, workshops, community of practice formats like share/learn/do/reflect/debrief, team interventions, etc.) at a later point, once the topics have been firmed up.
This curriculum leverages our Architecture Competency Framework and Competency Elaborations, by Ruth Malan and Dana Bredemeyer:
To get a head-start with self-study, you may
want to look at
various resources related to these architect competencies.
And, of course, you could read back through the archives for this journal.
"They that won't be counseled, can't be helped."
"They that won't be counseled, can't be helped." Benjamin Franklin, 1758
Here are some blog posts on architect skills:
See also: The Duties, Skills and Knowledge of Software Architects, Rick Kazman, Paul Clements, and Mark Klein
I read Denise Cook's "Business Capabilities Mapping: Staying ahead of the Joneses" (2007) in the IASA Architect Skills Library, and it reminded me not just of our "Enterprise Architecture as Strategic Differentiator" executive report, but also Stalk and Evans-Clark's seminal work in this area: "Competing on Capabilities: The new rules of corporate strategy," (Harvard Business Review, Mar/Apr 1992). This paper powerfully makes the case (without saying as much) that IT Does Matter, or at least IT can.
"Those who don't make dust, eat dust."
quoted in Peter Fuchs, Kenneth Mifflin, Danny Miller, and John Whitney, in "Strategic Integration in the Age of Capabilities," California Management Review, Sprint 2000.
3/9/08 Taking a Long View
I strongly encourage architects to view technology or architecture roadmaps as one of the defining deliverables of their role. That said,
" Prediction is very difficult, especially about the future." Niels Bohr (physicist)
Here are some ideas to help build anticipatory awareness and practices, paying attention to trends and events in our market and technology space.
From Michael Watkins and Max Bazerman, "Predictable Surprises: The Disasters You Should have Seen Coming" (Harvard Business Review, March 2003):
Organizations are vulnerable to surprises in part because of our
human predilection for harboring illusions that things are better
than they are, giving weight to evidence that supports our
preconceptions, giving more weight to maintaining the status quo and
preferring to avoid some pain today than prevent a lot of pain
tomorrow. Information and responsibility in organizations is
fragmented across organizational silos, and this too makes
organizations vulnerable to predictable surprises. To counter this
propensity to be blind-sided by disasters the organization should
have seen coming, Wakins and Bazerman encourage scenario planning and
Is this relevant to architects? Well, we think it is, and utilize something along these lines in formulating risk strategies (part of Architecture Strategy a.k.a. Meta-Architecture in VAP).
"I often describe the life of a software architect as a long and rapid succession of suboptimal design decisions taken partly in the dark."
Philippe Kruchten, Common Misconceptions about Software Architecture, Rational Edge, 2001
3/9/08 Agile Architecting and the Wrecking Ball
Agile values drive iteration and validation activities in the Visual Architecting Process:
For some counterpoint, see Tad Anderson's post on Agile. At least someone isn't kowtowing to the agile movement—even if I am. :-)
3/10/08 Blogging Just Enough
I like "LouisD"s just enough cloud from his Playing the Just Enough Game blog post (August 2007). It's a great idea for organizing a blog around! Of course, Louis also got my attention with this quote from his first post:
"Architects should not be control freaks but innovation freaks," Louis D, Archi-Works blog, July 23, 2007
3/10/08 System Properties
3/11/08 Market and Tech Watch
Bits 'n pieces:
3/12/08 Architecture Workshops
We have two open enrollment classes coming up next month:
There are still some places open in each, and it'd be great if you could help spread the word about the workshops. The next Architectural Leadership class is also tentatively set:
We're in the process of redrawing the Visual Architecting Process poster to more closely map what we've been doing. Here's the current draft:
As you can see from the poster, visualization and visual representations are integral to system and software architecture, not just building architecture. We still have some work to do on this, but it will give you an idea of where we're headed. I've decided that the Software Architecture Action Guide book stands between me and too many other projects, so we have to get that completed in the next few months. Please hold me to it! In the meantime, there's always the Software and Enterprise Architecture Workshops... the workshop materials have become books themselves!
3/14/08 Principled Leadership
Principles have been broadly adopted as a way to create alignment, allowing creativity and empowerment within the strategic framework created by these values-laden principles. We are seeing more examples among business leadership, and two great examples are the Patagonia Principles described by Yvon Chouinard, and the principles that Howard Behar used during his tenure as CEO of Starbucks.
In our technical communities we have been using principles broadly in enterprise architecture and software architecture, and it is proving to be a great way to communicate values within the technical community. The question then becomes, if our principles express our (technical) values, shouldn't we expect them to be stable for years? We want new hires to embrace these principles, and we want to hold our teams to them, even hold up a project if a team has ignored a principle (without any appeal to the governance board for a well-warranted escape clause)! Yet principles are a useful tool to express strategic choices, and we don't want to keep adding principles with each change in strategic direction. If we did that, we'd amass principles and fall foul of the minimalist architecture principle (see also guiding principles for enterprise architects)!
Ronald Fabel uses "strategy" in this way, when he says "One well-formulated strategy is worth 1000 quantifiable requirements." Along these lines, a shared understanding of (business, use, and system) context goes a long way to helping a team make choices that align with the business and architectural intent. Vision, context, strategy. Pound, pound.
In Running an Architecture Principles Workshop, Craig Borysowich gives some good advice. I would add, though, that architecture principles are an expression of technical strategy, so it is perfectly justifiable to have principles that are directed at addressing key architectural challenges. The architect is a strategic leader; that is, she gets to set technical strategy. Yes, this must serve the business strategy and the business intent. If the architect decides a principle is needed to address a technical objective, challenge or risk, she gets to create that principle. Well, in my book she does. [pun intended]
Principles in architecture:
Principles in software
3/14/08 Monkey Business
A few days ago I was quite taken aback by the furor around the Lacy/Zuckerberg interview at SXSW. That "mind blind" state (Goleman) we've seen evidenced in flaming cow pies left in blog comments seems to have moved into twittering too. Either that, or it is those nucleus accumbens getting their punishment punch-up. At such times, I'm so, so glad that this journal is a quiet little backwaters place that attracts scant attention!
It'd help to be more careful with the stories we tell ourselves about other people and their motives. We need to recognize that we only know our own motives, and our interpretation of external "evidence" is shaped by our internal state. Our attitude shapes our outcomes; others may be involved too, but we can affect our own attitude. (John Maxwell's books aren't really my cup of pop, so I haven't read The Difference Maker, but I gather it makes that attitude point.)
It does serve to remind me of some advice Rob Seliger (a leading architect of CCOW, part of the HL7 standard) gave us more than a decade ago: when you're addressing developers, start where they live, in the technical space; don't start with business concerns. Too bad Rob wasn't giving Sarah Lacy advice.
[If you look at some of the furor over Lacy here (and in the comments here), you'll see why my photo is just perfect! From time to time I'm tempted to tell some of the stories of sexism in our industry, but I usually regain my senses reasonably soon and send the piece to the outtakes bin; yup, such stories are generally stereotyped as whining, and that is so unpleasant. Now, I suppose, I could be damned for falling foul of a sexist ploy myself, except that not all of Lacy's detractors were men. That baboon could be a girl baboon... waugh, waugh... Grin. Ryan tells me there are people in his class (of 9 to 12 year olds) who think he has a "dry sense of humor." Gosh, I wonder where he gets that?]
3/16/08 Refactoring and Lean
Craig Cody's blog pointed me to a
refactoring discussion on InfoQ (I do hang out on InfoQ a fair
amount, but had overlooked this one, so thanks Craig).
is one of the key technical practices in the Agile developer's
toolkit. Refactoring also has no measurable customer value by its
very definition - it involves changing the structure (design) while
maintaining the behavior. In the
Lean world - anything that does not have
customer value is waste, and a customer only perceives
behavior/functionality and not structure."
"Refactoring is one of the key technical practices in the Agile developer's toolkit. Refactoring also has no measurable customer value by its very definition - it involves changing the structure (design) while maintaining the behavior. In the Lean world - anything that does not have customer value is waste, and a customer only perceives behavior/functionality and not structure."
Refactoring is a Necessary Waste, Dec 18, 2007
Perhaps you see why I reacted to
Booch's irrelevance of
architecture synopsis; it could (unintentionally) reinforce a misconception that is fairly broadly
held. It has to do with an illusion that behavior and architecture
are unrelated, and I sense that this is tied to an
artificial separation between "functional" and "non-functional"
requirements. But these are just different facets—any service has a
"what it does" and a "how well it does it" (service
level) aspect. A poorly designed system quickly erodes the "how
well." Even when the "what" is being measured as the check box on
user value delivery, the user notices, with less than delight, when
system properties degrade.
Ok, so at what point is the system degraded
enough to warrant "cleaning up"? By degraded, I mean in terms of
system qualities that affect user experience directly (various
dimensions of performance and reliability) and indirectly (system
price tag and speed of getting access to new features),
as well as the business ability to respond (development cost, time
and predictability). Deferring quality (bugs, yes, but I'm really
talking about kludgy structure here) at any point is just going to
impact user experience and business agility down the road. We know
that. In lean manufacturing, the factory floor is kept so clean you
could eat off it. Why? Is it waste to keep the floors clean?
Is it waste to ensure quality from the get-go, and at every point
where quality can be improved, rather than inspecting for quality at
the end? Lean manufacturing leans heavily on the total quality
movement. The biggest waste of all are quality issues that cause
delays at the end of the process, or worse, show up in the hands of
So are we refactoring to simplify? to remove
duplication? to remove complex or unnecessary dependencies? to
improve performance (e.g., cleaning up a complex collaboration
path)? to apply a pattern that is well-understood and easy to
communicate? to make the system easier to extend? to make the system
easier to debug? to make pieces of the system more useful in other
"Yes" to some of these will show up in direct
user experience, because user experience is affected by system
properties (the "how well"). And some will show up in development
efficiency (how much we can do with given resources) or agility (how
quickly we can respond to the market). Sustaining innovations come
from being able to incrementally improve the system, and a clean
architecture allows for chunks of the system to be innovated on and
improved at a different velocity from other chunks of the system.
[You no doubt remember my
"Trek" piece. The architecture allows, for example, Shimano to
innovate on gears independently from Bontrager on frames.]
Development efficiency shows up in future product or service cost,
agility in new features, which all affects business competitiveness
and future customer experience. So, Craig, yes,
you're right to be skeptical. It's a clever-sounding argument;
beguiling even. But, I'd say, be wary. If your system is a one-off
short-term special project that's one thing. But if it is to see
your business through years of evolution, don't let the need to
clean up get hidden under the covers too long. (I was just helping
my son with his room, and you don't want to know what we found under
his bed! No, really, you don't!)
Remember, as architects and software developers
we have TWO (classes of) voices to listen to: the voice of the
customer and the
voice of the business. The business cares about our attention to
the voice of the customer, because that is how
we compete in the
near-term. But the voice of the business is also about the long-term
viability of our business and the systems and products that support
And, if none of this persuades, focus on just
one thing: in business after business, industry after industry, we
hear "it costs 80% of our budget just to keep the lights on" (where
"the lights" are the systems that undergird the operations of the
business). Maintenance, or system evolution, matters. You could
justify all the refactoring it takes, just to get that number down
for the life of your system! And that is only one piece of the
systemic economic puzzle. Structure matters!
Of course, I believe that this refactoring
should be happening even before we get to code, by playing out
system behavior (addressing the "what" and the "how well") through
our models. Remember, architecture decisions are those that have a
high cost of change (a key Booch insight), so we want to learn as
much as we can, as cheaply as we can, as early as we can.
we want to do iterative modeling and improvement through stakeholder
participation, and refactor responsibilities to achieve crisper,
more balanced designs, or to address aspects or cross-cutting
concerns more effectively. For larger-scale systems (where I mean
code base, in this case, not transaction load), it becomes harder to refactor at the code level without impacting diffuse parts of the
system, and that is typically error-prone. So, model, and test our
models, create a skeletal architecture, test that, and so it goes.
Start agile early, with architecture. System
behavior has to do with function, the properties (a.k.a. qualities)
of that function, and the interaction of qualities across the
system. In the Maserati-Prius
example, there is an interaction between properties associated
with the acceleration function, and properties associated with the
braking function. If the car accelerates like a Maserati, it better
brake like a Maserati, or there'll be times when the driver will
face a life-threatening situation. The creators of the braking
system may have nothing to do with the creators of the acceleration
system (and may even be different external suppliers). But the
architect has to ensure that the properties are consistent; that the
architecture delivers behaviors with the intended properties and the
system has integrity, so that properties of one subsystem are not
inconsistent with those of another subsystem. System properties, or
cross-cutting concerns, are among the factors that make architecture
so crucial in complex systems. If you don't like
my Maserati example, try this: search
Perhaps you see why I reacted to Booch's irrelevance of architecture synopsis; it could (unintentionally) reinforce a misconception that is fairly broadly held. It has to do with an illusion that behavior and architecture are unrelated, and I sense that this is tied to an artificial separation between "functional" and "non-functional" requirements. But these are just different facets—any service has a "what it does" and a "how well it does it" (service level) aspect. A poorly designed system quickly erodes the "how well." Even when the "what" is being measured as the check box on user value delivery, the user notices, with less than delight, when system properties degrade.
Ok, so at what point is the system degraded enough to warrant "cleaning up"? By degraded, I mean in terms of system qualities that affect user experience directly (various dimensions of performance and reliability) and indirectly (system price tag and speed of getting access to new features), as well as the business ability to respond (development cost, time and predictability). Deferring quality (bugs, yes, but I'm really talking about kludgy structure here) at any point is just going to impact user experience and business agility down the road. We know that. In lean manufacturing, the factory floor is kept so clean you could eat off it. Why? Is it waste to keep the floors clean? Is it waste to ensure quality from the get-go, and at every point where quality can be improved, rather than inspecting for quality at the end? Lean manufacturing leans heavily on the total quality movement. The biggest waste of all are quality issues that cause delays at the end of the process, or worse, show up in the hands of the user.
So are we refactoring to simplify? to remove duplication? to remove complex or unnecessary dependencies? to improve performance (e.g., cleaning up a complex collaboration path)? to apply a pattern that is well-understood and easy to communicate? to make the system easier to extend? to make the system easier to debug? to make pieces of the system more useful in other contexts?
"Yes" to some of these will show up in direct user experience, because user experience is affected by system properties (the "how well"). And some will show up in development efficiency (how much we can do with given resources) or agility (how quickly we can respond to the market). Sustaining innovations come from being able to incrementally improve the system, and a clean architecture allows for chunks of the system to be innovated on and improved at a different velocity from other chunks of the system. [You no doubt remember my "Trek" piece. The architecture allows, for example, Shimano to innovate on gears independently from Bontrager on frames.] Development efficiency shows up in future product or service cost, agility in new features, which all affects business competitiveness and future customer experience.
So, Craig, yes, you're right to be skeptical. It's a clever-sounding argument; beguiling even. But, I'd say, be wary. If your system is a one-off short-term special project that's one thing. But if it is to see your business through years of evolution, don't let the need to clean up get hidden under the covers too long. (I was just helping my son with his room, and you don't want to know what we found under his bed! No, really, you don't!)
Remember, as architects and software developers we have TWO (classes of) voices to listen to: the voice of the customer and the voice of the business. The business cares about our attention to the voice of the customer, because that is how we compete in the near-term. But the voice of the business is also about the long-term viability of our business and the systems and products that support it.
And, if none of this persuades, focus on just one thing: in business after business, industry after industry, we hear "it costs 80% of our budget just to keep the lights on" (where "the lights" are the systems that undergird the operations of the business). Maintenance, or system evolution, matters. You could justify all the refactoring it takes, just to get that number down for the life of your system! And that is only one piece of the systemic economic puzzle. Structure matters!
Of course, I believe that this refactoring should be happening even before we get to code, by playing out system behavior (addressing the "what" and the "how well") through our models. Remember, architecture decisions are those that have a high cost of change (a key Booch insight), so we want to learn as much as we can, as cheaply as we can, as early as we can. That is, we want to do iterative modeling and improvement through stakeholder participation, and refactor responsibilities to achieve crisper, more balanced designs, or to address aspects or cross-cutting concerns more effectively. For larger-scale systems (where I mean code base, in this case, not transaction load), it becomes harder to refactor at the code level without impacting diffuse parts of the system, and that is typically error-prone. So, model, and test our models, create a skeletal architecture, test that, and so it goes. Start agile early, with architecture.
System behavior has to do with function, the properties (a.k.a. qualities) of that function, and the interaction of qualities across the system. In the Maserati-Prius example, there is an interaction between properties associated with the acceleration function, and properties associated with the braking function. If the car accelerates like a Maserati, it better brake like a Maserati, or there'll be times when the driver will face a life-threatening situation. The creators of the braking system may have nothing to do with the creators of the acceleration system (and may even be different external suppliers). But the architect has to ensure that the properties are consistent; that the architecture delivers behaviors with the intended properties and the system has integrity, so that properties of one subsystem are not inconsistent with those of another subsystem. System properties, or cross-cutting concerns, are among the factors that make architecture so crucial in complex systems.
If you don't like my Maserati example, try this: searchdelight site:www.ruthmalan.com on Google and on Yahoo! —yep, on Google the result is downright embarrassing! It is same function from a user perspective, right? With quite different results. So why do these results differ (in magnitude: 15 versus 3)? Is it a difference in the search algorithm, or a difference in trolling strategy, or something else? Is the difference architecturally significant?
Ok, now search deligt on Amazon, and on Google or Yahoo! or MSN. The search engines know you spelled it wrong, and offer a correction. Not Amazon. Is the omission of that piece of functionality architecturally significant? Is it strategically significant, in that Amazon loses revenue opportunity? I imagine it is! Any missed chance to give the customer what they are looking for (even if they don't know how to spell it) is significant to the business and hence, I claim anyway, it is architecturally significant! Architects need to be paying attention to what is happening in neighboring spaces, and if I'm in charge of search at Amazon, I want to be tracking what is happening in search at Google and Yahoo! But it doesn't end there; A9 (Amazon's search engine) gets that deligt is spelled wrong! And when one hand of the business doesn't know what the other is doing, isn't leveraging what the other is doing, that is architecturally significant! Ok, so maybe Amazon knows it is missing this piece of functionality and working on it.* Well, what does that tell you? Something about ease of extending the elephant? Incremental releases? I don't know. I've never worked with Amazon. And now, I probably never will. Oh no! Eh, don't worry, you're probably the only person reading this. Grin. A quiet, backwaters place...that will stay that way—I just pretty well ensured no "recommended essayist" accolades will come from Werner Vogels.
[3/17/8: A response to "The Three M's—The Lean Triad" (2/27/08 on InfoQ) directed me to Jim Womack's "Mura, Muri, Muda" e-letter. It makes an important point about the problem of focusing on "muda" (waste) exclusively. Roman Pichler argues "Value-creating activities enhance the product from a customer perspective. A good question to ask is: "Would I be happy to pay for this activity as the customer?" Another good question is, would I be willing to pay for this as the business manager, or architect. Because "the customer" is usually a euphemism for scores of customers, now, and into the future, for this product and its enhancements, and a set of variants in a portfolio or family of products (or services).
3/24/08: One might argue that the adaptability of software is both its greatest strength and its greatest curse. Adaptation and piecemeal growth means we don't have to know everything upfront—we can adapt the code to accommodate change. But typically the structure erodes as the system grows under schedule pressure to deliver value. This calls to mind comments about Netscape as well as Amy Tan's description of piecemeal growth in a traditional family home in China:
"The house had been in the family for many generations. It was not really so old or remarkable, but I could see it had grown up along with the family. There were four stories, one for each generation: great-grandparents, grandparents, parents, and children. The house had a confused look. It had been hastily built and then rooms and floors and wings and decorations had been added on in every which manner, reflecting too many opinions."
from The Joy Luck Club by Amy Tan p. 48.
"Large-lump development is based on the idea of replacement. Piecemeal Growth is based on the idea of repair. … Large-lump development is based on the fallacy that it is possible to build perfect buildings. Piecemeal growth is based on the healthier and more realistic view that
Piecemeal growth, or incremental development, is not just desirable but a fact of life in software (even big-bang first releases are evolved thereafter). Even so, we need to build more learning into our process. More learning when it is cheaper to find and fix problems with the vision (doing course corrections toward "right system") and structure (built right). Then, accepting that we will continue to learn and evolve our system, we need to invest in fixing the mistakes—incrementally adding functionality yes, but repairing structural defects too. This investment is the crucial dual to piecemeal growth that we too often forget in software. When we keep marching to a frenetic "add value" drumbeat, we get into a situation where the system threatens to crumble under the mass of deferred structural issues (like the Minneapolis I-35W Mississippi River Bridge that collapsed last year). This is what Grady Booch reports is done at eBay, though I can't point to any organization that does this systematically (even if I wasn't bound by NDAs).
Principle of simplicity: eBay sets aside some of development budget to simplify. This makes the architecture more maintainable, less brittle, more resilient to change.
from my notes from Grady Booch's Yahoo! talk (May 2007)]
[11/10/08: Well, well, Amazon now has type-ahead-like search suggestions and also catches some typos and comes back with "did you mean?" -- though the application is not consistent across the site...]
3/16/08 Strategy Dynamics
Rajiv Mistry pointed us to Strategy Dynamics. It certainly looks like something worth spending time with—thanks Rajiv.
3/17/08 Roadmaps or Projections
Dana Bredemeyer has been using the word "projections" instead of "roadmaps" for our exploration of trends and scenarios. I can see his reasoning, but I'm okay going with the flow. Whatever you want to call it, as long as you put some cycles there from time to time, I'm happy my job has been done.
What we want to do is map out our product release plan, our own internal development plans for infrastructure (platform) development, our plans for capability development, what we know of our vendors' release plans for technologies we depend on and so forth. We might call this our roadmap, though strictly speaking even this is projection. Still, these are intentional, and though obviously not deterministic, we will invest in making this future unfold, so it is a more likely future than most other futures we could speculate about.
But then we also want to overlay these plans with projections about how current trends might unfold. So we have moved beyond "product roadmap" turf into more reasoned projections and educated guesses about the future of the markets we compete in and the technologies that might well (re)shape these markets.
Why? Because our architecture has to see us through that world of tomorrow—when the system will be fielded, and evolved. Today's world may well be a better predictor of that world than any other prediction we might come up with. But we still need to think about what the future is likely to bring. Even if we had a perfect crystal ball, we couldn't build all of the functionality we will come to need into our first release—if we want to release in competitive timeframes, anyway. And we don't have a crystal ball, so we can't build to meet every contingency. But it also doesn't make any sense to act like a proverbial ostrich, bury our head in the sand, and ignore likely threats and opportunities. Not if we want to be great. Remember—foresight!
"Programming is the act of deciding now what will happen in the future."
James Coplien and Neil Harrison, Organizational Patterns of Agile Software Development, 2004.
When my daughter runs full-speed down
long, steep stairs, I am reminded that some people are simply more
prescient than others. In part, it is the school of hard knocks, and
in part it is a tolerance for the ambiguity we face dealing with
fuzzy futures. But the architect, as leader into the future, has to
do some scouting out of that future terrain she's leading her team
into, and we've found that roadmaps, projections and scenarios help
the architect take a long view. Not to attempt to take care of every
detail of tomorrow today, for that is so futile as to be a sign of
insanity! We carry a spare tire in the car, but not a spare battery.
It is a matter of finding the balance, for our project.
"Think like a person of action, and
act like a person of thought."
It's Not About the
"Think like a person of action, and act like a person of thought."
Howard Behar, It's Not About the Coffee, 2007
We have treated group (graphical) history and roadmaps/projections as two separate activities and artifacts. But working with groups across time, I've started to realize that having a rolling horizon, where today's present is updated with events as they unfold, and become part of the tracked history, is useful. Of course, I'm talking only about keeping architecturally significant events on our rolling radar. So, Vista was announced. The Vista delay was announced. (This was earth shattering enough that I remember when and where I was! Indeed, I was facilitating an architecture workshop in the Indianapolis Marriott—the one where Greg Russell impressed me as an all-round outstanding person and resonant architectural thinker.) Vista was released. ... If your products or applications were impacted, how did you track and make visible the values, dependencies, and risks associated with these unfolding plans, resets, and events?
With regard to projections, let me just say Dana was an unusual (Indiana University) computer science student in that he scouted out and spent a summer in Philadelphia studying under Russell Ackoff, the systems-thinking (anti-)guru. He has since spent 3 decades in the software industry, but that early influence was formative and Dana had already earned the title of architect at HP in 1987. Anyway, "projection" has Ackoffian residue.
[5/8/08: There's a nice example of a Graphical History done by Dave Sibbet of Grove here (Evolution of the Internet) and here (HBR Management Ideas and Practice). See also National Semiconductor's vision here and VizThink's Context Map here.]
3/18/08 Structure and Behavior: Take 3 (or is it 4?)
Since it is "Brain Awareness Month", it is interesting to note that much the same structure (the human brain) can give rise to very different behaviors (yours versus mine, for example).
I have also been thinking, prompted by Booch's "architecture is irrelevant" and Amr Elssamadisy's "refactoring is waste" pieces, about how quite different structures can support largely similar behaviors. This is most striking at the poles: kludgy code versus well-structured code. So we can make statements like "it is largely irrelevant to the user whether our architecture is service oriented or not." But such statements are generally true only within a range of assumptions about required behaviors. This became clear to me early on, when we wanted to interrupt a long print job to insert a high priority rush job, and that couldn't be done with the pipe-and-filter architecture that was the dominant design for printer firmware. This new "use case" (or variation on a use case) couldn't be accommodated by the existing architecture, and required a fundamental redesign if we were to elevate this piece of behavior to "must" status in our feature scope. For a narrow range of assumptions about required behavior, we can accommodate that behavior with a variety of structures—from structures that are highly tuned up and specific to the behavior, to structures that are more flexible and could support more variance or spread in behaviors. Indeed, it is when we throw variance at the architecture, due to evolving needs over time, or different product variations at a point in time, that the structure becomes very important—even for relatively simple systems. Add to that demands like (operational) scalability and reliability, and structure really factors.
"Not only are a
system’s desired operating modes influenced by its architecture, but
so are some of its failure modes. Thus an architecture that permits
only one path between elements may fail if a leg of any path breaks.
All of a tree below a broken node is isolated from the rest of the
Edward Crawley, Olivier de Weck,
Steven Eppinger, Christopher Magee, Joel Moses, Warren Seering, Joel
Schindall, David Wallace and Daniel Whitney, "The
Influence of Architecture in Engineering Systems,"
Edward Crawley, Olivier de Weck,
Steven Eppinger, Christopher Magee, Joel Moses, Warren Seering, Joel
Schindall, David Wallace and Daniel Whitney, "The
Influence of Architecture in Engineering Systems,"
A system could be coded in such a way that it is very tightly bound to a particular workflow, forcing diffuse changes to the code to accommodate variations in the workflow. Duh! But it makes the point that structure can bound the range of possible system behavior in ways that become quite obtrusive and frustrating to the business as it seeks to adapt to changes in the competitive landscape. If we don't throw some variation at the architecture, assessing the impact using interaction models, thought experiments and reasoned arguments, we can be blinded to coupling to the particular workflow "the" customer has expressed in the requirements. But I already reacted to invoking YAGNI at every turn. The architect has to use judgment to balance exigency and structural robustness. Use lightweight, "Pareto" tools (even models on flip charts or whiteboards) and keep track of assumptions and rationale along with your decisions. Apply that uncommon common sense. Time to move on.
"Common-sense in an uncommon degree is what the world calls wisdom." -- Samuel Taylor Coleridge
3/18/08 Architecture and Agile
Scott Ambler's commentary/essays: Agile Architecture: Strategies for Scaling Agile Development
If you read Scott Ambler's Agile Architecture: Strategies for Scaling Agile Development, you'll see that the piece has progressed a fair amount—at least it has since I read it a couple of years ago. He still has that common practice/agile practice table that I find offensive—Scott says it is common practice for architects to place themselves on pedestals. Common practice? Hmmpf! And he's updated his references and suggested reading and doesn't recommend this journal. Hmmpf again. Oh, yeah. I don't like that he sets up architects for target practice. OK, Scott Ambler is doing good, important work that will make a big difference in software engineering because he appeals to a broad base. But context matters, and some projects are vastly more complex than others, and some architecture efforts are strategic and broad scoped, so not everything should be tarred with the same brush.
One point worth pondering: the deeper the team goes on detailed concepts, the higher the cost of changing the overall concept. The more quickly we can expose the overall concept to tests of desirability and fit (determining whether this is the "right" system), the more confidently we can launch teams to work concurrently on different components or services. Remember, development concurrency (more teams working in parallel) is the path to "agility," meaning speed to market with a new or enhanced product or service, for larger, more complex systems.
We can cover good distance by simply asking ourselves what kinds of requirements changes we are concerned about:
We want to get a handle on our dominant uncertainties, just as we want to get a handle on our dominant complexities. The first helps us with "right system," the second with "built right."
Once we have a better sense of where our uncertainties lie, we can formulate strategies for resolving them. It is generally a good idea to figure out how to deliver increments of value that build out the vision, give us opportunities to refine and elaborate the vision, and even to ditch the vision and start afresh, if that is what it will take to ultimately be successful. But we can learn so much from rapid modeling cycles with active stakeholder involvement, that we can get rapid and cheap course correction even before we launch teams on sprints, or other incremental/evolutionary development process styles.
This paper (from systems engineering) is interesting: "Agile SYSTEMS ENGINEERING versus AGILE Systems engineering," by Reinhard Haberfellner and Olivier de Week, INCOSE 2005.
3/19/08 Animals in Translation and Bad Smells
Dana picked Animals in Translation back up and was telling me another story from it; this time about the way that Temple Grandin audits slaughter houses. She says that while most auditors audit documents, she audits the animals. You can change the documents to mislead, but animals don't mislead. Why inspect for dirty floors, because problems with filth will show up in the animals? And so forth. She has a small set of questions about the animals, that are key questions that reveal answers to many questions. And she looks at direct evidence.
This reminded me of Booch's questions that he uses to assess "how a project smells" when he goes in to do an architecture archeological dig:
from my notes from Grady Booch's Yahoo! talk (May 2007); the wisdom is Booch's, any mistakes in transcription are mine
The "smells" language reminds me of James Navarro's advice to me in my early days at HP. Of course, it also reflects the "bad code smells" language that Kent Beck apparently introduced into our parlance. Some of the smells in Mika Mäntylä's Taxonomy of "Bad Code Smells" apply just as well to architecture—we apply heuristics along these lines looking at responsibilities of components in conceptual architecture, and again as we delve further in logical architecture, designing architectural mechanisms and key interfaces. Of course, these are along the lines of Booch's guidelines:
The fundamentals never go out of style:
from my notes from Grady Booch's Yahoo! talk; the wisdom is Booch's, any mistakes in transcription are mine
Our contribution here is the recognition that much more of this can be done with visual models and reasoned arguments than is typically done in projects (whether they do BDUF or "agile architecture"). We don't just see this modeling as getting everyone on the same start page (although this is critical), but as a way to quickly, cheaply, participatively improve our architecture designs so we eliminate the smell at its architectural source.
I learned early on that the people and the process (not the documented process, but what people are actually doing) give me huge hits of insight very quickly that hours and hours of pouring over architecture documentation (what there is) and structural analysis tool output, and so forth, only serves to confirm. If I ask a question about how a piece of behavior is supported, and it takes 5 people to answer the question, I've learned a lot. If I'm in a meeting and first one architect, and then another, and then another, are called out because there's an ops crisis, I've learned a lot! But it is also true that I learn a lot from the artifacts. If I ask to see the architecture and I'm shown a picture of the infrastructure that tells me volumes! If it comes out from under a pile of stuff in the architect's drawer, or is a smudgy vestigial sketch on a white board, that tells me a lot. I'm not dismissing the details by any means; it is just that often the details obfuscate the fundamentals. Going in, one wants to know if everyone is shooting in the dark.
3/19/08 The Architect Also Codes Debate
I realize I have already logged my reaction to the insistence in some quarters that architects must code, but found myself returning to it as I'm working on clarifying my experiences and thoughts around organizing to support good, right, successful architecture. Now, for technical leads and architects on smaller projects, I agree—although often where architects then put their attention is prototyping with new technologies. While it is essential to stay on top of technology directions and assess the capabilities new technologies bring, this doesn't address the root concern which is losing touch with the project demands and realities. Still, we have to recognize that the architect has to juggle many competing demands.
On the innovation side, there is staying abreast of emerging capabilities to assess their relevance to the business, through
And there is being inventive in applying capabilities to create value for "the customer" or "the business" (read "proxy for customers across market segments and future customers, as well as shareholders"). And there is building these seeds of a vision into a shared vision that the organization embraces and acts intentionally to realize.
On the project execution side, there is being an effective technical leader—being credible, making decisions, coaching, solving problems, and defending the structure and evolving it as necessary. The organization can accomplish big, ambitious, complex things by providing the context and technical strategy within which many (dozens, even hundreds of) developers can innovate and create. But to do that effectively, minimizing waste, takes architecture, and it is a big job for big systems. Such a big job, that the architect may be squeezed for time in development cycles, persuading and influencing, coaching, collaborating, addressing misunderstanding, resolving conflicts, and, when needed, updating the architecture. And that's just talking about the architect's work with developers. The architect also has to work with the management team, keeping them appraised and keeping their enthusiasm for the effort high, even while keeping an eye on the future and providing input to strategic direction.
So, for big projects we have to realistic about the desirability and feasibility of giving the architect critical path coding responsibilities—if we expect the architect to stay on top of all that is demanded to ensure "right system, built right." We could give the architect some coding responsibility just to keep her hand in the game, but we have to ask ourselves what we want to accomplish—how many if-then-else statements does a person have to write in a lifetime? Surely we want the architect to stay abreast of the most challenging problems? These she has to take on, pair program to see them through, or at a minimum review the approach taken. Remember, too, if she solves every challenging problem, she's neglecting the aspirations and personal development of her team.
In all this, it is just as well to bear in mind:
Yes, these were remarkable people. Yes, these are extreme cases. My point is only that while the exception does not prove the rule, we should not let the general rule exclude the exception. Moreover, we don't expect a composer to play every instrument in a symphony orchestra. Composing is it's own domain of competency, takes its own kind of attention. Architecture does too.
"The person who sweeps the floor should choose the broom."
Howard Behar, It's Not About the Coffee, 2007
Architects need to be remarkable people. The need for great architects is becoming a widespread concern, and more and more organizations are looking at development programs for architects because simply having coding experience, though essential, is not enough. I would like to remind you that I am still waiting to hear how you would like us to help you meet your personal development goals as you progress on the architect career path.
For historical antecedents to this discussion see also ChiefArchitect and ArchitectAsKeeperOfTheFlame and the discussions that follow. Michael Feathers expressed some struggle with how to word this, but I like where it goes:
"it takes real authority to come into a situation and change it so that the authority is no longer needed." — Michael Feathers
[3/26/08: I suspect that a whole lot of this debate would just evaporate if we consistently prefixed our discussions of the architect role and architecting process (e.g. how much ceremony is needed) with a description of the context—much like a patterns template. For complex, large-scale systems are intrinsically different than small-scale systems and this difference shows up in the construction roles and process; likewise, broad-scoped efforts (like product family architectures) are intrinsically different from narrowly scoped efforts. The mantra: context matters. Do you need to hire an architect to design your dog-house? Your ice fishing hut? Your 3-level home? Your local hospital? By the time we're talking hospital architecture, we're talking a team including lead architect and structural engineers.]
3/19/08 A Quiet Backwaters Place
Yelp! I hadn't realized it, but flaming comments and threats have apparently chased Kathy Sierra right out of the blogosphere! I'm so sorry to hear that! Kathy Sierra set the bar in terms of insight (take for example her featuritis post), upbeat energy and willingness to interpret life in positive terms (take for example her angry can be bad for your brain post). But it just goes to show how the information explosion has raised the din to a roar; I didn't even notice that I hadn't visited Kathy's blog in over a year, and I liked Kathy's blog!
We saw Horton Hears A Who last the weekend (no, I'm not too old for that), and it led to a discussion about the battle between good and evil that is played out in our everyday lives. The big evils like overt violence and spam are easy to identify and shun. But the smaller battles, like the battle not to grouse at a day-dreamy kid who every... single... day... has... to... be... told... to... do... the... same... thing getting ready for school or bed; the battle not to say something hurtful like, "you're 8, you're smart, I just don't get what's so hard about this daily routine." The tension between the American dream, Easter magic and materialism versus environmental footprint, carbon footprint and global warming.
Our grousing and our insults hurt. And our lack of attention adds up.
The electronically connected socially networked, blogging, twittering world is so new; perhaps we'll develop a code of conduct that helps us overcome the "mind blindness" and group reinforcement that leads to a lack of social restraint and subjugates common decency.
So, anyway, I'm reminded that holding forth in a quiet backwaters place has its advantages! I have less impact on other people's thinking, but the tradeoff is that I have much more freedom to impact my own thinking!
3/20/08 Happy Vernal Equinox!
3/20/08 Lessons from Product Development
In the product development space, a product platform is not the compute platform, but rather a shared platform architecture and set of common components that support a family of product variants. These may hit different points on the price-performance spectrum, or differ along some other dimension of market segmentation such as usage situation (e.g., clinic versus hospital). In software, the SEI has popularized the term "software product line." In other engineering disciplines, "product families" and "product platforms" have been a focus of research and publications. Of course, product family and product platforms have a history in product development in industry, reflected in books like:
and I still have (to buy and) read
3/22/08 Circle of Excellence
Once again, Grady Booch has given me good grist for my thought mill. Earlier this month, I talked about creating a personal circle of excellence, and Mr. Booch has offered us a prototypical example of reflective self-assessment and characterization:
"I am not defined by what I do, nor do I do just one thing. At the moment, I'm a chief scientist and a Fellow and a software architect and a project manager and a programmer and a researcher. I'm also a mentor, lecturer, consultant, software archeologist, theorist, methodologist, developer, pragmatist, pioneer, mediator, historian and visionary. In my working career I've been a mower of lawns, a scooper of ice cream, and a singer of songs. By act of Congress, I was once even an officer and a gentleman. I'm a good friend and confidant, a godfather, and a loving husband. I listen well and play well with others. More abstractly, I'm a child at heart, a warrior, a servant, a leader, a dreamer, a lover, a believer, and most of all an awe-struck seeker."
Grady Booch, Software Architecture Handbook blog, March 18, 2008
Mr. Booch is sincere, and sincerely admired. It's that "awe-struck seeker" that most appeals to me. For "awe" and "seek" are two properties I value highly. It speaks to a positive, affirmative approach to the world, seeing the magnificent, being open to delight. And it speaks to being investigative, an explorer in all the worlds, the world of nature, the constructed world, and the worlds of thought, creativity and spirituality. "Authenticity" is on the pop radar, but I would say that Mr. Booch's qualities are authentic. I mean, he collects stuffed giraffe; you just know he's a child at heart. But really, in my (admittedly limited) interactions with him, I've concluded that he is a very nice, very smart guy who has managed not to become arrogant despite his impact on the software field.
Now here's the important part. We are what we believe we are. I once wrote this for a friend:
I love your art, what you reflect.
The insight is more important than the words: we are constantly in the process of creating ourselves in the image we hold of ourselves. We strive to be consistent, authentic to ourselves, and to the image of ourselves that we project for others to embrace.
Mr. Booch says he is not defined by what he does. I get that he rebels against being defined by a job label. Still, I beg to disagree. What he does, says a lot about his qualities and interests, and what he does, focuses his attention and, given finite time on this earth, conscribes what he becomes. I started a "movies I like" page and there explained the tag line ("what I see, defines me") that I created for my "other interests" page. Anyway, I make so bold as to disagree, because we must choose carefully what we do. It is like system scoping, but with respect to who we are and what we are capable of! It is this insight that drives Steve Jobs to his exceptional focus on excellence:
"We don't get a chance to do that many things, and every one should be really excellent. Because this is our life. Life is brief, and then you die, you know? So this is what we've chosen to do with our life. We could be sitting in a monastery somewhere in Japan. We could be out sailing. Some of the [executive team] could be playing golf. They could be running other companies. And we've all chosen to do this with our lives. So it better be damn good. It better be worth it. And we think it is."
Steve Jobs, Fortune magazine, March 7, 2008
The saddest thing is looking back and realizing that in the hurly-burly of the day-to-day we just kept taking the next step, and the next step, none of it adding up to much. Mr. Booch has lived a life of "ands," including publishing 5 books, and he's not done. That is inspiring! But it takes focus, work towards a vision that refines who we are and creates the person we wish to be.
Now, this is all probably too mushy for you. 5 9's of the time it is too mushy for me! But I do want to encourage you to spend your 5 minutes for this year thinking about your vision for yourself. Thinking about your personal qualities, what you like and admire in yourself. Project yourself out two to five years, and think about how you would like to be seen, who you would like to be, what you would like to have done and be capable of doing. And then think about how to get there. Take notes. And if you care to share your personal development plan with me, then you will influence our ability to help you reach your vision—for yourself, but also for your organization.
Also, if you're a senior/chief architect leading other architects, then it'd be helpful to us, if you'd share your vision for architects in your organization, and what you believe would help them strive for excellence. It takes a lot of people to create a great system these days, and being a great architect means working effectively with and through other great people. Set them on a path to greatness, and you rise higher too.
[3/26/08: And if you're thinking that "Life is brief, and then you die" is not your style (indicating that you're not 40 yet), then how about this from Jeff Bezos:
"So it really was a decision that I had to make for myself, and the framework I found which made the decision incredibly easy was what I called — which only a nerd would call — a "regret minimization framework." So I wanted to project myself forward to age 80 and say, "Okay, now I'm looking back on my life. I want to have minimized the number of regrets I have." I knew that when I was 80 I was not going to regret having tried this. I was not going to regret trying to participate in this thing called the Internet that I thought was going to be a really big deal. I knew that if I failed I wouldn't regret that, but I knew the one thing I might regret is not ever having tried. I knew that that would haunt me every day, and so when I thought about it that way it was an incredibly easy decision. And I think that's very good. If you can project yourself out to age 80 and sort of think, "What will I think at that time?" it gets you away from some of the daily pieces of confusion. "
Jeff Bezos interview, Academy of Achievement interview, last revised 0ctober 2006
That is the best example of desired state interviewing performed on oneself that I have come across yet. I'll definitely be using that one in our Architectural Leadership workshops, if not others!]
3/22/08 Shipping the Prototype
This evening Ryan decided he wanted a toy marlin that is bigger than him. Given that the whole thing was improvised—he drew the marlin out on big paper and I guesstimated what it would take to make it in 3-d—he was buoyantly happy at the result. In this case, the prototype will do.
3/23/08 Happy Egg Day
So, Google doesn't commemorate this runner-up to Christmas on the retail calendar; I wonder why not? "Happy Easter" may not be "pc" in a multi-cultural world—but not celebrating chocolate? Why, that's absurd! My kids learned the phrase “sugar rush” from school friends this year, and the phrase has produced more “bouncing-off-the-walls” than the sugar itself did in previous years! J That's the power of naming something. It's worth bearing in mind, like when you're thinking of principles.
If you recognize Easter in a spiritual way, I hope you have a happy celebratory day. And to us all, I wish a day of reconciliation and tolerance for diversity; a happy, gentle day.
(I noticed it was ok for Google to commemorate St. Patrick's Day. Was that because it might also be known as "hit the pub" day? Well, I liked Google's egg hunt in 2001; that was creative.)
3/23/08 Great Collaborations
Here are some examples of great collaborations that I wanted to keep track of (good metaphors):
and among performing artists:
I'd like some examples from software... so I need to spend some cycles reflecting on and researching this. And I'd welcome input—if any good stories or well-known examples come to mind, just message me on my ruth at traceinthesand interface.
I'm doing some research to create a Graphical History of Software Architecture, in part to demonstrate the technique, and in part to "practice what I preach" as we do another round of reflecting on our part in the "helping architects be great" big picture. In the process, I came across this lecture slide-set from De Montfort University in the UK. Naturally, I really like the recognition that it gives to the Bredemeyer approach. But it calls it "Dana Bredemeyer's approach" and I confess I am partly to blame for the general ascription of our approach to Dana. When Ryan and Sara were babies I played a less visible but key role creating Bredemeyer's Resources for Architects website and training materials; yup, that was our way of dealing with the off-ramp/on-ramp choices around having babies and a career—I kept working, but because I wasn't willing to travel away from my babies/toddlers I focused on branding Bredemeyer. In those days, when I traveled I did it with a nanny and toddler entourage, and that was expensive but I did it to stay active in the field, albeit not as active as I am now.
Anyway, architect practitioners and practitioner-oriented universities give more credit to our contribution than those who follow more formal publication/academic dominance hierarchy versions of the history of our field—like this one. For example, it neglects the Visual Architecting Process in the architecture methods band because it has been documented on our Resources for Architects website (so freely accessible since 1999) rather than in a formally published book (too long we have tarried). We were at the early head of the wave that recognized that the internet is where technical people "live," and freely putting high quality useful information on architects' desktops via the internet was going to have broad and powerful reach. Of course, VAP has been taught in our workshops for that long, and its predecessor was taught inside HP. In 1998, Rick Kazman, Paul Clements and Linda Northrop met with us at HP and we shared what we were doing inside HP with them. That was when we were still living in Moss Beach (on the California coast) and Rick Kazman joined us for dinner at our home. Gosh, those 10 years went quickly!
Turning over some historical rocks, I found:
3/27/08 Turning over more Rocks: Software Architecture History
Scouting out some practitioner history, I pinged Bruce Anderson at IBM. Bruce is, in my assessment, a consultant par excellence for he embodies what I believe is the essence of consulting—being devoted to the success of others. Bruce is credited by Grady Booch as having inspired his Software Architecture Handbook work. Indeed, Bruce put together an OOPSLA Birds of a Feather session titled "Towards an Architecture Handbook" in 1990, and followed that up with OOPSLA workshops on that topic during the next few years. Now, I see his current email signature tagline reads "from strategy to services and components." Even in his signature, he is educating and leading thought.
Academic-flavored treatments of our field's early history (SEI and Kruchten, Obbink and Stafford) talk about Dijkstra, Parnas and Brooks, with nary a mention on Melvin Conway—you remember, of Conway's Law fame. This is what Melvin says:
"In 1967 I submitted a paper called "How Do Committees Invent?" to the Harvard Business Review. HBR rejected it on the grounds that I had not proved my thesis. I then submitted it to Datamation, the major IT magazine at that time, which published it April 1968."
Practitioners are empiricists; ample proof lay in our repeated and collective experience; fortunately this was recognized by Datamation. Anyway, Conway's paper talks about systems, subsystems and interfaces, the system design process, the classical "systems image their design groups" and the tendency of large system [structures] to "disintegrate" during development. It doesn't use the word architecture or architect, but it is clearly about system design. So, methinks Conway has been snubbed twice, first by HBR, and then by the historical treatments of our software architecture field.
Ok, so this indicates something really important: everyone has only a partial view on history, and to build up a better picture we need to involve others who have diverse viewpoints. So, I'd like to beg, plead, coax, cajole, anyone who has an important pointer here to please do share it with me! If the history we produce is missing something important, you are to blame if you don't point it out!
What follows below is not my view of history! It is only a collection of references to papers that I think are historically significant to our field of software architecture. But I believe our history is much richer than the paper trail left by formal publications. In building architecture, we would find it pretty silly not to reflect the prototypical buildings of each architectural genre or style in a historical timeline. We would also find a history written in terms of the books/papers that were published or the conferences held, a trifle one-sided. So, we need a view of the systems, the people, and the practices that stand out as milestones in our history; one that complements the view we get reading Krutchen, Obbink and Stafford's “Past, Present and Future of Software Architecture” IEEE Software paper (March/April 2006). While that paper is really valuable, it focuses (by conscious choice, announced in the opening paragraph) on academic and formal contributions to the field and neglects the practitioner history. Given that focus, the article gives short shrift to the history of architecture in practice, and leaves a hole in the story of our field. It is that history that I would like to shed some light on, and welcome and encourage your help in doing so.
An incomplete but growing list of papers that are historically important to the field of software architecture:
Hmmm, this sounds like a reference to architecture by counter-example to me! And from 1962! If American Girl is anything to go by, I'm of historical vintage myself, but not that historical!
A Qualities Link I didn't Want to Lose
Land R., "Improving Quality Attributes of a Complex System Through Architectural Analysis - A Case Study", In Proceedings of 9th IEEE Conference on Engineering of Computer-Based Systems (to appear), 2002
3/26/08 Defined by Job Title
Last night, I overheard an Indiana University professor say that he is a "mathematician, not an educator." I was quite taken aback. It would be one thing to say he's not a school teacher, thereby distancing himself from the poor state of math in our public schools. But for someone who is a lecturer to say he is not an educator is very illuminating—sadly so. It says "math research is where my heart and head is; lecturing just pays the bills, and I don't stake any part of my identity on my role as a lecturer." Russ Ackoff, in contrast, calls himself an educator, in contradistinction to being a guru. On the rare occasions that I've been referred to as a guru, or my work as "teachings," I've shuddered in embarrassment, knowing how undeserving I am of the kindness intended by the unfitting appellation. But if ever there was a guru in software or in business, I don't expect that "followers" would accept his thoughts without question. On the contrary, I believe we admire most the people who make us think because they surprise us out of our preconceptions and make us look at something in a new way; well, that is true for me.
Dan Pritchett (this is a classic) and Werner Vogels (this is a classic) are among those I look to. Dan has set a great example, one that is by all accounts consistent with what is being established as culture for eBay, in taking serious time to simplify his code—and blog about it. For systems with much of a life expectancy (like eBay's), we really need to elevate the importance of structure, so that it can compete successfully with what is perceived as "customer-facing" value delivery.
3/27/08 Pareto the Early Cycles
Here's some advice I gave recently:
Start practicing the VAP tools. Do a Competitive Landscape for your current project and see how that changes what you say and do with your customer and your project team. Just do it in 20-30 minutes and see what you come up with. When the opportunity presents itself, do a Competitive Landscape exercise with your (extended) team. Notice what happens when you do it alone and the value you get from that, versus the value when the group does it.
Likewise, create an Influence Map. Create a Technology Roadmap/Trend Projection, write up some scenarios and look for risks. Do these in small 20-30 minute bursts for the project you’re working on, so they help you meet your project goals and demonstrate to you the value of using these tools at the Pareto level (20% investment for 80% gain). Put your Technology Roadmap up in a team space, with a marker or post-its so people can add to it. Notice what happens as this becomes a shared work-product that the team keeps alive.
Similarly, start collecting and crafting stories. Create a personal introduction story and use it at the beginning of the workshop. Take the vision statement for your current project and make it come alive. A “cereal box” with a features list is scope-on-a-box. It's one way to convey the end state, but we want it to be more compelling than that. What are the stories about how a customer’s life will be better? And so forth.
Each time, notice what difference this makes to you—to your understanding, and to what you do and say. The more excited you are about these tools, the more infectious you will be teaching them. Enthusiasm persuades.
For more on any architect role-related topic, try googling your keyword or phrase of interest (e.g. “stories” or “architect sketch”) on site:ruthmalan.com.
This is generally speaking the advice I give to architects in our workshops, except for the last item (I'm very shy about my journal, and though it is referenced on a slide here or there, I don't personally recommend it in workshops; Dana mentions it, but it is rumored that he likes me).
Technical people really have to overcome a tendency to overwork something (this is certainly true for me). But striving for perfection while we're still getting a lay-of-the-land feel for the strategic issues we need to deal with, is a fools game for which the rules aren't even invented yet. We have to force ourselves to move quickly, to get that quick Pareto hit, and move on, because there is so much more ground to cover! I would far rather architects did most everything in VAP at the 20% level, than that they picked one or two things to do at the 80% level. For the first broad-brush strokes iterative VAP cycles give a lay-of-the-land that informs thinking in critically strategic ways, surfacing dominant uncertainties and complexities, and laying the groundwork for addressing differentiating value and architectural challenge. Subsequent cycles are focused by what we learn in the early cycles, and that is where we start to carefully choose what techniques and models will best help us define the problem and design our approach. At that point, our challenges and goals should direct our attention, and the process offers help but we, the architects on the project, need to decide where to put attention based on the unique priorities, context, requirements and architectural challenges we face.
3/27/08 Context Matters
Dana and I tell stories from Surely You're Joking Mr. Feynman, which I've only listened to (most of) but need to get a hardcopy for when I want to read snippets in workshops. One of my favorites is about the importance of setting context. But I have a little story that demonstrates by counter-example. The other day we decided to replace our VAP poster hours before Dana needed to board a plane with it. Naturally Murphy intervened—our scanner acted up. I asked my assistant to run over to Kinkos to scan the images. In all the scurry, I neglected to tell him the purpose and the urgency. He went off to Kinkos, found their scanner busy and came back to the office, thinking that his time was too valuable to waste waiting around for the scanner to free up, so he'd just go back later! What might have been a reasonable decision under other circumstances, was a very damaging decision under the peculiar circumstances that day. Now, this little story is sort of like "leadership lessons I learned from my kids" (or books I read to my kids). I know, I tell those too. And I know my reading on some people's credibility meter plummets at such times. But I learned a lesson from it, because it demonstrates that in small ways and big, context matters, and even one of the most enthusiastic purveyors of this message can forget to set context!
So, we need to set context. In the large, with competitive landscapes, business strategy and intent, system use context, system infrastructure context, all these external contexts. And we need to set technical context in terms of goals and strategies, principles, down the line to service, component and interface "contracts."
Why? When we set context, we empower people to make better decisions when the unexpected happens. And the unexpected will happen. So the good leader finds ways to efficiently convey context—goals and objectives, but also the contextual information that makes these goals makes sense. If we know that safety is the concern, we can deliver a 2-engine jet that is as safe as a 3-engine jet, but costs less.
3/27/08 And Organization Matters
I heard this commercial for the "Buxton over-the-shoulder organizer." (Not my choice; I was held captive in the waiting room while my kids entertained the dentist.) Anyway, the Buxton organizer (cut sound first—audio plays automatically) caught my attention—the value proposition? Ever lost your keys in your purse? I once worked with a guy who was going for a sex change and as part of the process he had to "be" a woman for a year. The first thing that happened when he started to dress as a woman was that she lost her keys in her purse! I kept saying "did you look in your purse?" and she kept saying yes and dismissing me, looking everywhere else a second and third and fourth time. Eventually she found them in her purse. I once lost my passport in my purse, but that's a different story—one about assumptions. So anyway, the value proposition is "a place for each thing;" a place for your cell phone, a place for your keys, glasses, whatever—pockets to organize everything.
For a long time I've been telling architects that the Architecture Decision Model is an organizer like kids' cubbies at school or lockers in locker room, or the old room-key-and-mail-pigeonholes in the hotel lobby in older movies. That is, it provides placeholders for all the different kinds of decisions an architect (or team of architects) makes.
A framework like this provides a place to log decisions, as well as insights, ideas, questions, risks and issues, to keep track of thought. This is important, because if you organize your decision making to address dominant uncertainties and complexities, prioritizing to reduce risk or prove out a cornerstone approach or key architectural mechanism, you may find yourself thinking about, modeling and making detailed logical architecture decisions before you've completed architecture strategy (a.k.a. meta-architecture), for example. The decision model provides organization for the decisions, without forcing a sequence to making them.
Organization helps with communication (providing a navigation structure) and keeping the architecture current. We know where to put decisions (and related explanations, justifications/rationale, issues, etc.), and where to look for decisions. It also provides a place to put issues and ideas that relate to other areas than our current immediate focus, so we can keep track of them without being defocused by them.
In short, no more decisions lost in the purse. No more ringing phone that you can't answer because you can't find it in the clutter.
And if you're dismissing this as just about woman's purses and process stuff, please give me your attention for one more paragraph. The Conceptual Architecture is an organization model too. Any time that we explicitly apply the principle of separation of concerns, one of the benefits we get is better organization. So the Architecture Decision Model does that at the highest level, but the Conceptual Architecture does that too. We factor cohesive responsibilities and assign them to an architectural component (e.g., component, module or service). We name the component and it becomes the pigeonhole for related responsibilities. As we work through our behavioral models, we might decide we have unrelated responsibilities that we're attempting to force-fit into an inappropriate pigeonhole, or our pigeonhole may be getting overstuffed, so we refactor.
Basically we're trying to establish one place for a decision, and have it be a place we'll find, the next time we're looking for it.
3/28/08 Towards A More Visual Trace
A week or two ago, I stumbled on Hugh Macleod's piece on creativity. Two things struck me:
Inspired, I started carrying a little sketch book in my purse. And while waiting in line, in airports or playing taxi service for my kids, I started experimenting with conveying ideas with just a few lines and a black pen. I kinda liked my "architecture serves behavior" sketch—it seems like the germ of a cartoon series!
Now, for some time I've been thinking that we try too hard to be "perfect" with our models, and we artificially make that too high a barrier for ourselves. Scanning in a hand-drawn model and including that in our document is good enough for many purposes. It conveys a "work-in-progress" feel that can be just right if we want to get input to improve our architecture.
I think it is an important skill for an architect—to conceptualize and express abstract concepts. Throwing in a little humor helps too.
3/28/08 As CIOs Get a Place at the Table, Expect To Be Along For the Ride
This is interesting: "The CIO Profession: driving innovation and competitive advantage." It is a report done by IBM, along with Harvard and MIT, and it confirms a trend we've been noticing:
"Senior management increasingly recognizes technology as central to innovation and competitive advantage. As a result, more and more CIOs are gaining a prominent seat at the table in their executive teams and playing an active role in strategic business decisions."
Center for CIO Leadership, October 2007
As this trend continues, architects will be in demand to deliver on the CIO's mandate to create innovation and competitive advantage through technology.
3/28/08 Sjaak Laan
I've been remiss—Sjaak Laan has had a link to my journal for months, and I forgot to thank him! Sjaak, thanks, you're an architect and a gentleman, and you will forever bear the distinction of being the first to link to this journal in your links/blogroll sidebar. Sjaak is yet another of those Dutch architects; The Netherlands is truly a great country for architecture—where in this case I mean software and systems architecture, of course.
For the rest of you, just think, you could have had that distinction. Ok, so http://www.ruthmalan.com/Journal/JournalCurrent.htm will always link to the current issue. There, no excuses. If you blog, let me know when you've linked here, and I'll thank you!
3/30/08 Thinking Around the Edges
Dana is just back from Beijing; he says the new terminal just opened and is amazing—so big it will have it's own weather! Some of the folk there mentioned that Johnnie Chung Lee (from CMU) prototyped a low-cost interactive whiteboard using a Wiimote and DIY light-pen (see the video on YouTube):
"Since the Wiimote can track sources of infrared (IR) light, you can track pens that have an IR led in the tip. By pointing a wiimote at a projection screen or LCD display, you can create very low-cost interactive whiteboards or tablet displays. Since the Wiimote can track up to 4 points, up to 4 pens can be used. It also works great with rear-projected displays."
Johnny Chung Lee, Wii projects
As architects, we need to keep our trend sensors turned on, even when we're generally heads down cranking a project out. Interactive user input technologies got a big boost with Wii, because it made interactive relatively cheap and it is rapidly becoming part of mass expectation. Johnny Chung Lee writes:
"As of September 2007, Nintendo has sold over 13 million Wii game consoles. This significantly exceeds the number of Tablet PCs in use today according to even the most generous estimates of Tablet PC sales. This makes the Wii Remote one of the most common computer input devices in the world."
Ok, so where does this go? It doesn't matter in the least, unless it does, and then it is transformative. It is the job of the architect to figure this out! Are these changing expectations around input devices going to reshape our world? In what ways? Does this impact our value proposition? Is there a chance to shape a market by leading, or can we afford to wait and see, to follow?
3/30/08 Still Turning Over Rocks: More History
"In 1961 or 1962, Fred Brooks approached Jerry Weinberg at the IBM Systems Research Institute in New York and asked him whether he thought the term architect was suitable for software systems people. At the time, Weinberg had already been incorporating ideas from architecture in his courses on system development. Brooks was concerned about the fidelity of the analogy, but it seemed to hold in their ensuing discussions as they fleshed it out. From then on, the term became a fixture of the discourse on software design."
James Coplien, Reevaluating the Architectural Metaphor: Toward Piecemeal Growth, 1999