A Trace in the Sand
Online Architecture Journal by Ruth Malan
I also write at:
Trace in the
By Topic (May)
- Do architects need
5/1/06 May Day
It is Workers Day, and it reminds me how fortunate I am to work in a knowledge economy where I can be creative and do work I truly enjoy!
In April, I dealt with subjects related to the role and skills of the architect. Some of this writing was packaged up into essays on the Resources for Architects web site. Some may appear in the forthcoming GEAO Journal article on the architect competency framework. But if you are interested in the architect role, you will find useful links, and discussion of various aspects of the role, in my April journal writings.
5/2/06 Do Architects Need to Code?
In part, architecture is about enabling many people to work on a complex system, and still achieve something that is coherent. So the architect needs to have technical insight that comes from technical experience, and the architect needs to be good at achieving alignment with the technical direction, and getting buy-in to the architectural decisions that shape the system and resolve the big challenges. This takes persuasion and influence, skills we associate with leadership and organizational politics.
In an autocratic culture, we may be able get away with less of the politics and influence and rely more on the authority vested in the architect role. But in the highly collaborative, consensus-oriented cultures that characterize most of the businesses we work with, the broader the scope of the architecture, the more the architect has to rely on leadership and organization politics skills to influence and persuade, align and get buy-in. Of course, if the architect has no credibility with the development community, all the political skills in the world will not overcome this deficit.
Several blog posts (e.g., Ramkoth, Johanna) have levied criticism at architects who don't code, and there's not a whole lot of empathy for the tension that the architects themselves are placed under. Many architects I know would love to have more time to code (in their day job, too), but they have to carefully choose where to spend their bandwidth. The architect knows he could do a good job of the code, but if he was writing that part of the system, and that part, and that part, pretty soon he would run up against even his own high productivity ceiling and have to face the fact that complex systems cannot be built by one great architect/developer alone. Then you have to take more people on board. Then you need to communicate to them to set context and align what they do. Then you need to take more people on board because that communicating meant you had less time to personally code, bringing down your own coding productivity ceiling... But if you do a good job as an architect then you raise the productivity ceiling of the team. And so it goes.
So, many architects or technical leads absolutely, yes, still write code and this is important. But this may not be true of senior architects responsible for complex system architectures, with a number of architects and numerous developers reporting to them for technical direction. This is a big job, and leaves little time for writing code, creating tension between wanting to keep a hand in the game to retain skills and credibility, and the technical strategy setting, problem solving and communicating, advocating, defending and explaining that also takes time. Many of these senior architects ask how they can stay in touch better with the technical frontier and maintain the respect of developers they work with. My best answer is to refer to those senior architects that I have worked with, who have retained the respect of their development teams even though they no longer have the bandwidth to cut (critical path production) code. They are extremely smart. They pay attention to many sources, inside and outside their organization. And they pair up with developers to solve challenges they face. This pair programming provides the architect a chance to coach/demonstrate by doing, and to problem-solve hands-on, while being able to move on to other issues when that challenge is dealt with. The architect's role is to abstract and extract, create models, bring their own and others expertise to bear on creating and validating the architecture, synthesize practices to make common across the team, find the mechanisms to use consistently across the system. And this role should be seen as valuable and necessary to the team's success. The alternative is to see the architect as a threat, constraining, interrupting, intervening but walking off with the title and salary. This is why we say goodwill is the real silver bullet—again, if your team lets you do a good job as an architect, then you raise the productivity ceiling of the team.
Architects are very much about expanding the creative capacity of the team, and achieving system integrity, system-fit-to-purpose and system-fit-to-context. Everyone plays a role in creating successful products and applications; not architects alone, not developers alone, not testers alone, not managers alone, nor marketing and business analysts alone. Yes, architects have an important role to play, and we need to let them play that role! In large projects, the architect's role should not be undercut by challenging the architect to load up on coding responsibilities and simultaneously keep the architecture solid and up-to-date. Something will have to give, and it will be the architecture. Just because a project enters the construction phase does not mean that the architect's role is done and now the architect can take on a fully-loaded developer role. If we are trying to do less waterfall and more agile modeling and development, then we have to recognize that we are spreading architecture work out through the entire construction phase.
When construction begins we should not view the architect as a freed resource that we can pull off onto another architecture project nor, for that matter, should we move the architect back into a full-time developer role, loading her up with coding responsibilities. The architect is needed through construction, and through all the releases, to explain and defend the architecture, and to modify it when the architect is persuaded this must be done. During construction, the architect is needed to explain, mentor and coach, help problem-solve, to watch for misapplication of the architecture, to watch for obstruction of the architectural intent, and to learn!
To get the benefits of architecture in terms of decreasing lifecycle costs and increasing productivity (and agility) and predictability, we need to make sure the architecture is protected from accommodations that weaken the structure and integrity of the system, and improved as faults are discovered, and that the architecture as documented keeps pace with the architecture as implemented.
Building a sky-rise is technically and organizationally complex. Man-hour for man-hour, building software is probably even more so! In software, from start to finish, we are creating as we go; we are not just executing on a plan. But we do need a plan to provide the context within which we can allow more people to create. Architecture in our software world is very much about enabling more effective collaboration. Collaboration that scales up to the kinds of numbers that we rely on to get complex products and applications shipped, has to be enabled. We can't just assume it will happen. We can't expect it to happen in the large, without processes and tools to facilitate the communication and co-ordination involved. By tools, I don't necessarily mean computer-supported tools, though they could be. I mean, at least, the architecture diagrams and documents explaining and justifying the architecture decisions and approach. But all these architecture documents cannot stand alone. Collaboration is about personal involvement, and people are best at activating personal involvement—people with good leadership, persuasion and influence skills, that is. Yes, good to great architects! Architects who understand that it is not just about the technical soundness of the architecture, but about getting the development community actively and effectively engaged in a collaborative effort with customer and business value as the goal.
And in his rather timely way, The Pragmatic Architect has a synergistic post on his blog. This is the kind of blog I find really exciting, because it tells the story of an architect's personal journey.
[See also Should Architects Code? Bill Barr, May 2007]
[5/7/10: Commentary: Ok, that post was circa 2006--it is getting attention because I highlighted it in a journal post this month, and it was picked up by Rockford Lhotka and Bob Familiar (thank you for the posts, the digg, the shouts and the tweets). Rockford's comments are very insightful:
"So I think that blog post
has a lot of very valid points, most notably the idea that you can't
expect to pull the architect out of their role and have them code -
that'll cost you the architect.
-- Rockford Lhotka, comment on blog post titled "Do Architects Need to Code," 5/7/2010
Indeed, it is important to highlight those points! And I still (4 years later, and despite all we've learned in the interim) think it is also important to increase awareness that architects working on projects of different scale/complexity and at different levels of scope/seniority play different roles. Let's face it, working across large groups, aligning large groups, making it possible for the work of many dozens, even hundreds of developers, to come together in a system that is cohesive and delivers the intended value, takes time--serious time--so we need to be a careful with universals like "of course architects need to code." Senior architects who are working at broad scope are dealing with so many competing demands on their attention that day-to-day production code responsibilities generally isn't such a great idea. If you go to the extreme, and think of the chief architect, all the reasons that we need the chief architect role compete with this demand that the architect code. At the extremes, it is easy to see. Small project--of course the architect should code. Very large project--not so easy to find enough time in the day, making critical path code not such a great idea and non-critical path code may not be where the architectural challenges lie. Many architects use prototyping as a way to stay in touch with the advancing frontier, but there is always a need to reserve some bandwidth for staying in touch with the production code. My point was that by pair programming through challenging pieces of the system, the architect can immerse but then step away again, leaving that part of the system in the hands of someone who is fully up-to-speed on it. Naturally reviews (architecture, design and code) are other vehicles not just for ensuring system integrity but also for raising challenges and issues with the architecture. But really, the net-net is that staying integrated with the technical teams is important all round. All the while acknowledging that the architect has important responsibilities other than critical path feature delivery. Enabling that, yes. Distinctly!]
5/2/06 Architects and Requirements
Ron Jacobs, of Microsoft, has two pieces of work worth a visit. One is a video clip on Requirements and Constraints (some heavy breathing and camera shake which I only mention so you bear with it, for it is a good introduction to requirements and constraints, and nice goal metaphor). The other is a "beyond bullets" style presentation on Architecture and Architects. Thanks to Mike Platt for pointing to Dick Hardt's outstanding presentation on Identity 2.0. Presentation style to aspire to!
5/3/06 Software as Collective Work Product
I was just renewing my subscription to Harvard Business Review, and since I was there, was scanning through the archive of back issues on topics that interest me because they are important to my effectiveness and the effectiveness of other architects. One article that immediately jumped out because it resonates with my journal entry yesterday, is Katzenbach and Smith's "The Discipline of Teams" (HBR, July 2005). They define teams as groups that produce a collective work product, and collective work products as those that reflect the joint, real contribution of team members. Software, whether for products or to support services, is generally a collective work product. But sometimes the groups that produce these work products become so large we need to organize the work so that we can have multiple teams, and teams of teams, all effectively, with personal investment, working on the collective work product.
I have been recommending Katzenbach and Smith's book, The Wisdom of Teams (1993), to architects ever since I came across it in 1994. Their HBR article, "The Discipline of Teams," is worth a read. Some managers get uncomfortable at their turf being encroached on when I make points like this, thinking that it is the manager's job to set the vision and tend to the care and feeding of teams. And some architects get uncomfortable, thinking that they would far rather this was a manager's job so that they could just be left in peace to worry about how to achieve consistent SSO across federated applications integrated via web services, or whether this resource manager mechanism will scale as multiple chasses are added...
Software teams are about two things: getting more expertise so we can build better products and applications, and getting more bandwidth. Yes, these are two things:
Architects who recognize that they are more effective when they enable their communities to be more effective, are well on the road to being great. It is not just about the architect making technically solid decisions. It is about enrolling others, and empowering them, to contribute their best. It is about inspiring and motivating, it is about generating goodwill, and it is about enabling co-ordination so that all the pieces will be the best they can be, and the system, composed of the pieces will be the best it can be. (And of course, by "best" I mean best fit to the need taking into account constraints and tradeoffs that have to be made.)
Yes, the great architect is technically great. If it were just about the people side, then we could have stopped at technical project managers and we would not have seen software architects emerge in all kinds of organizations and industries over the past decade or so. There is a strong technical dimension to the role. You can't just slice-and-dice a system any old way and parcel the pieces out to teams and individuals to build and then recompose the pieces into systems that deliver the services with the intended characteristics or qualities! Even if you start with a common vision, but only have a rough cut at divvying up the system, then you either have endless unproductive bickering about what the pieces mean and who must do what, or you have people going off and making their best guesses at what they were meant to do and maybe, eventually, through immense hard work and mass self-sacrifice, get to something that works. But this path is littered with failures!
So, it takes experience and talent to create a great architecture—architecture that clearly identifies the structures and mechanisms that must be built to deliver the differentiating value propositions and foundational services of the system. And it takes other skills to help the communities involved make their best contribution to the success of the system.
The architect who understands that he must get the great architecture that is in his head into the heads of the team, is the architect who begins to get the full scale of the challenge! And no matter how great we think the architecture in our heads is, it will be greater if we include others in the process of creating it. We will understand better what the architecture needs to be, to meet the needs of all who have a stake in it. This includes those who will create it, and those who have a perspective on the value this system could, and should, offer.
In workshops, architects have commented on how productive their teams are, creating a draft of key aspects of the architecture in just 4 days. They ascribe this productivity to working with top-notch peers. This is certainly true, but it is also true that we insist on working through that fuzzy front-end team alignment stuff (context, vision, strategy) before we start to "peal the onion" working on layers of requirements and architecture specification, and validating all the way.
5/3/06 Resume Selection Criteria and Architect Skills
When sorting resumes to staff an architect position, I would look for experience and successes that would speak to the characteristics of the architect. And in submitting my resume, I would not say " I am great at handling organizational politics." I would say, "I was lead architect for a product platform that supports 6 to 12 product releases per year, where the products span an order of magnitude variation in memory footprint and a price range from $90 to $500." I would let my successes speak for my skills. If I said I was great at organizational politics, I'd expect the hiring team to want to barf! But I could not achieve these results unless I had solved the technical challenges in my path and I was truly great at influence and persuasion, as well as communicating, consulting and coaching.
But if I was posting a job description for a platform architect for my product line, and if I wanted to have the right person self-select to apply for the position I had open, I would make it clear that I needed someone who was capable of meeting the strategic and socio-political challenges inherent in creating a product platform to be used by different product groups focused on different market segments all with unique customer needs and constraints. And I would look for very strong domain knowledge, and technical agility and adeptness at picking up new technologies, as well as considerable actual experience developing systems like this.
And if I was giving performance feedback to one of my platform architects, and I perceived that he was coming up short on influence and persuasion, I would recommend setting personal goals for improving in that dimension, look for a mentor to help the person, and point to some books and articles on the subject, including The Tipping Point and Never Eat Lunch Alone.
And, as an architect, I view organizational politics as an area I need to get better at. I am not a natural connector and salesperson (in the Gladwell sense) , but when it comes to architecture I rise to the occasion, but could still stand to do better, much better! So I scan for help, and set goals, and practice what I learn, and try to get feedback so I can measure my progress against my goals. I'm not trying to be nauseating! I'm trying to be more effective at getting a variety of people, at various points in the project, to contribute their best work to the architecture effort I'm leading, and to gain and sustain sponsorship throughout the life of the architecture. This is part leadership, and part understanding networks of influence and using persuasion most effectively.
5/3/06 Architecture and Garden Sheds
What we say about architecture and the role of the architect assumes a context in which architecture is needed. In the building space, we wouldn't call in an architect to design our garden shed. But if we were building a sky-rise, or even a 4-story apartment building, we would not contemplate undertaking the project without an architect!
When we talk about the qualities of great architects, or the activities in the architecting process, we are assuming that architecture and an architect are needed. If it is a simple system (small in scope, and simple in the technical sense), and we nonetheless go through all the steps of architecting, then we must realize that we are doing this for reasons other than immediate necessity. A good reason to do it is to learn, or because we want to create a culture that values and understands models. Or we might do this to prepare for adding complexity later.
But when the system is complex, we need to get to the point where we would simply be aghast if it were not architected! And be aghast if we were asked to renovate or add to the system and there was no architecture specification to use in planning and designing our changes.
5/4/06 Architecture and Collaboration
Today, if we were just to do a good job of replacing our current system with a modernized, cleaned-up new system, we would face considerable challenges. But we also need to be forward looking, and put in place enabling infrastructure and innovative features that will make the system competitive when released (generally in 1 to 2 years time), and see it through a set of iterations that broaden our product set or add features and value over time.
This means that as architects, we need to collaborate on many fronts, to get good ideas, to understand technical challenges, and create good, right, and ultimately successful architecture. The architect has to manage a balance between fast-track, closely-controlled and focused individual work, and opening up to influences from outside and inside the organization. A key skill is partnering; partnering outside and inside the organization to understand both how technology can be used to gain competitive advantage and to identify and manage the risks associated, and get the job done.
5/4/06 The Stool with Three Legs
An architect in a leading energy company told me that their organization was being set up as "a stool with three legs:" business, architects, development. It was the first time I'd heard it put that way, although we are seeing quite a number of organizations setting up architecture practices within IT, so there is a separation from the developer community. Whichever way we go with organization structures, there will be upsides and downsides. If we create an architect group, we give the group identity, and we allow the group to focus on developing their area of expertise, to share and learn from one another. We make it clear where to go for architecture help. And we face the danger of creating an architecture "ivory tower."
Nonetheless, in the building architecture field we see this. Architects work in practices. Construction workers work for construction companies. There are areas where this separation will become stronger in the software field. We already see implementation being outsourced quite extensively. We sometimes see the architecture being outsourced too, but, in an area that is core (in the Moore sense, that is, it contributes to competitive advantage) architecture is more apt to be kept in-house.
In either case (architecture practice as separate group from development, or outsourced code development), the architects are going to lose touch with their development roots. This places the architect in a tough spot! Technology is changing all around us. Staying up on the deep ramifications of these technologies is tough. But then, this is true for developers too. They are getting very deep experience with what they are working on, but the relevance of that experience will shift over time. The architect needs to be watching for these shifts, scanning the technology space and maintaining outside relationships to do so. And the architect needs to stay in touch with the current set of development challenges, directly by prototyping and experimenting early on, and possibly implementing pieces of the system during construction, or indirectly by involving key developers in the design of aspects of the architecture, and validating the architecture with stakeholders who can help improve, and prove, its efficacy.
We view validation as an activity that is undertaken early, and then often, as the architecture is iterated on, refined, and matured. Early validation exercises are undertaken with "friends" of the architecture effort, those who will take part in the exercise with a view to helping improve the architecture, not to bringing it down in flames before it has even half-baked.
5/4/06 Organizational Politics
From time to time I have wondered whether we should use the term "organizational effectiveness" rather than "organizational politics." (See, for example, the Organizational Effectiveness book section on our site. That was created in 2001, and evidences how long I have been in this quandary.) As a technical community, we tend to have an uneasiness around "politics," as if it is inherently distasteful. I generally want to call it what it is rather than reverting to some more comfortable euphemism, but I am finding that I want a term that includes organizational politics and some other areas like partnering that combine better into "organizational effectiveness" than "organizational politics." I know at least some in Microsoft considered the term "organizational agility" but they settled on "organizational dynamics" in the competency framework that is alluded to in their certification requirements. Still, I think neither works nearly so well as organizational effectiveness.
5/4/06 Architects and Requirements
I was (re)reading Rob van Ommering's essay titled "Things to do in Denver When You're an Architect." Rob has a section titled "Things I know I Didn't Do (But Should Have...)" and the first item is:
"Talking to requirements managers: in our organization there was a unidirectional flow of information between requirements and development teams. This is a pity, since developers may help to invent new features (at least there should be an interaction)."
We see this so much, that I have taken it up as something of a crusade, as you'll note in my journal entries over the past couple of months! We work with architecture teams and individual architects across many industries, both in IT and in product development. And Rob's situation is very common, and, yes, it is a pity. We recommend that architects bridge this divide, but many architects we work with see the gap as being bigger than they are. They have to spend energy overcoming so many other barriers that this one is just too much for them to take on.
5/4/06 in the US is 04/05/06
If you missed 01:02:03 04/05/06 (whether that was last month in the US, or today if you use dd/mm/yy), take heart! In less than one hour it will be 06/05/04 03:02:01!
5/4/06 03:03:03. Ok, if you missed that, you only have to wait 100 years... I tried to tell you in time!
5/4/06 Performance Contracts
I was investigating some work on leadership and came across an article on Flow. I mentioned to Dana that I'd leapt from leadership to "Flow" to Patagonia's plan to create biodegradable shirts! And you know what he said? "Flow, what do you mean flow? Are you talking about Mihaly Csikszentmihalyi's book?" And he even pronounced it right (Mee-high CHICK-sent-me-high-ee)! I had never heard of Flow, nor Mihaly Csikszentmihalyi. I was impressed! Apparently if you read Flow you will remember its author Mihaly Csikszentmihalyi ten years later and be able to connect the word "Flow" to him.
Now, this is interesting:
In "The Discipline of Teams," Katzenbach and Smith ((HBR, July 2005) identify clear goals as important ingredients for high performing teams.
Csikszentmihalyi believes that "flow has several necessary preconditions. These include having clear goals and a reasonable expectation of completing the task at hand. People must also have the ability to concentrate, receive regular feedback on their progress, and actually possess the skills needed for that type of work." (Flow:The Art of Work, by Ann Marsh, FastCompany, August 2005)
This underscores much in our architecting process, where we emphasize creating an architecture vision and architecture strategy (objectives, scoping and priority setting, principles) which are all about setting clear and achievable goals. Then we work on understanding the architecturally significant requirements (functional and system qualities/properties/non-functional requirements and constraints). These are all about getting more and more clear about our direction and goals, and setting priorities and scope taking into account the constraints we must work within, to ensure that our goals are achievable.
We have been emphasizing these activities because they are important to team alignment and motivation. Getting great software built is as much about the social dynamics of the team as it is about the code. XP and agile approaches are a lot about, ahem, flow.
Jimmy Nielssen is asking some important questions on his blog in a series he's titled SOA-Q.
5/5/06 Non-functional Requirements
David Ing has a great (insightful humor) post on non-functional requirements called "The Performance Question."
5/5/06 Software Design
Jack Reeves, "What is Software Design?" C++ Journal, 1992.
James Shore, "Quality with a Name," April 5, 2006
5/5/06 New Entry in the Blogosphere
I've landed in the blogosphere. I took Kevin's recommendation and used WordPress (afterall he's using WordPress). I checked, and www.traceinthesand.com wasn't taken (that surprised me! doesn't anyone else want to leave a trace in the sand? make difference, that sort of thing?). Now all I have to do is think of something to say!
5/6/06 First Blog Post
I decided to start "at the beginning," with a discussion of what software architecture is, so that we have some context for subsequent discussions about the role of the architect. I will craft the blog post here, before putting it in the blogosphere, and will borrow from earlier journal entries, so don't be surprised to see some repetition in what follows:
5/6/06 What is Architecture?
To set the context for subsequent posts, I thought we'd start with the topic of "what is software architecture?" Bass, Clements and Kazman's definition of software architecture (in essence, the high level structure of the system, described in terms of components and their externally visible properties and the relationships among them) has been very influential. While this definition has resonated with many people, the continuing discussion indicates some remaining uneasiness with definitions proposed so far. Rather than propose another definition, we accept the Bass et. al. definition and focus on the central concerns that software architecture addresses. (You might also like to take a look Chapter 1: Software Architecture: central concerns and key decisions.)
Complexity and Cost of Change
We need architecture to manage complexity and cost of change. Grady Booch puts it like this: "all architecture is design, but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change." Cost of change goes up as scope of impact increases, so this definition covers decisions relating to system decomposition as well as those addressing cross-cutting concerns. Cost of change is also high if a significant chunk of the system has to be revised or rewritten, so this speaks to challenging pieces of the system.
Big Rocks First
We use the "big rocks" metaphor (see slide 6 and 7): If we start with the big rocks, then add the pebbles, and last the sand, we fit them all into our jar. And yes, we could even add water. The point is not that we can always ask developers to do more! The point is that if we start with the sand, add the pebbles, and then try to add the big rocks, we cannot fit them all in the jar. To fit them all in, we must start with the big rocks. Architecture is about getting the big rocks in place first.
But what are the "big rocks"? The architectural elements—the components and their relationships, yes. And architectural mechanisms addressing cross-cutting concerns or systemic properties, yes. Big rocks bear a high cost of change, yes. Is there more?
Architecture Implements Strategy
The architect of early generations of HP OpenView said "Architecture is the translation of business strategy into technical strategy." This definition focuses on the strategic nature of architecture. What is significant, in this view, is driven by what is strategic, and what is strategic determines how we will compete. How we will differentiate dictates the big things we must get right, what hard problems we will tackle, where we will innovate, and where we must be ahead of competition. And it allows us to accept good-enough along those dimensions where we are not trying to create competitive advantage. Of course, even "good enough" may be challenging, especially when taken in conjunction with where we aim to differentiate.
The business or product strategy needs to establish what differentiating value we will deliver to our customers, our shareholders, our partners in the value network and our people. The architect needs to assess what capabilities will deliver this value, and further, what services and properties must characterize these capabilities in order to deliver this differentiating value. Architecture is about designing system capabilities that deliver the value propositions and reinforce the identity of the system (application, product, product-line, etc.) in alignment with the business strategy.
Architecturally significant decisions are those that must be made by the person or team who has influence and perspective across the system in order to deliver on the strategic objectives of the system.
Architecture Balances Differentiation, Complexity and Cost
The architect needs to balance the need to differentiate, with the lifecycle cost of the features and quality attributes we pursue. What we need to do, just to be in the game, constrains what we can accomplish in order to distinguish our products or services and business. For example, in many systems some level of security, scalability, and disaster recovery are threshold attributes not our avenues for differentiation. So we must develop mechanisms (authentication, encryption and firewalls; load balancing; failover, etc.) to address these challenges. Just delivering the base level of features and qualities is challenging, given the threshold set by intense competition in most markets.
Others will have a say in what our opportunities are to differentiate (e.g., marketing). And others will have a say in identifying the challenges we face to build and field systems in our domain (e.g., development). The architect needs to play a role in balancing what we would like to do, with what we are able to do given our resources and capabilities.
Architecture Expands Our Capability
Moreover, the architecture needs to play a role in increasing what we are able to do, and increasing the value of what we attempt to do, by allowing focus. Focusing attention, enabling specialization and separation, understanding where outsourcing or licensing can be leveraged, reducing complexity and scope where it is not essential to our value proposition. Knowing what to focus on and knowing what we can ignore—both are key to success.
Architecturally Significant Decisions
So architecture helps us manage complexity and cost of change, and deliver differentiating value in alignment with our business and product strategy. Architecturally significant decisions are those that the architect (or architecture team) needs to make in order to address these concerns (strategy, complexity, and cost of change). They generally include the high-level decomposition of the system and address cross-cutting concerns. What we do is driven by our business strategy, how we do it is driven by cost of change.
Furthering the State of our Understanding
If you have a definition of software architecture that you would like to add, I encourage you do so on the Resources for Architects site or the SEI site. That way your definition will be placed in the context of its predecessors, and your contribution will be all the more evident. And by all means also add your definition as a comment to the post.
Whatever else you might add, I hope that in the commentary on this post, you will share as concretely as you are comfortable with given the public forum (and your level of identity disclosure),
To set context for your observations, it would be very helpful if you would provide some background, such as
Don't worry if you repeat something that has been said. What challenges you face will certainly overlap with challenges others face. What is interesting is what is common and what is different across architecture efforts.
5/7/06 Architecture is Semantics-Free?
Well, after I spent my weekend trying to get what architecture is about articulated (ok, so I also went to the Farmer's Market, planted flowers and veggies with my kids, went running through the woods with my family, read a lot, thought more, wrote a bit, etc.), I came across this statement by Martin Fowler:
"I think SOA has turned into a semantics-free concept that can join 'components' and 'architecture'." (July 1, 2005)
Gosh, my bubble is quite burst.
Well anyway, you can make me feel better. See my blog post here, and do leave a comment. Naturally, some nice words would be good (you can practice your leadership and motivation skills on me, I don't mind) but what I'm really interested in is triggering some useful sharing of what concerns you are addressing through architecture. What challenges do you face, that you believe are uniquely solved by architecture?
Sharing experiences is a huge value. If it is something we "already knew" it is valuable for the validation it provides. If it differs from our experience and perception, it helps us to see how context also shapes meaningful differences in what other people perceive. What they believe is quite likely well-justified in their context, given their experiences, constraints and opportunities. Our experience causes us to draw conclusions that shape our beliefs so that when we make assertions they may seem off-the-wall to someone in a quite different context, yet be completely, obviously, self-evidently, correct to us!
There are a range beliefs out there about architecture, from "semantics-free concept" to "high-level structure of a system..." to "components and contracts" to... And it ranges from "just pretty Powerpoints used to impress management" to "a blueprint that allows distributed teams to work effectively on a collective work product that assembles smoothly to yield predictable services and service levels." Context matters. What architecture needs to be for one project may be different than for another. And what architecture is allowed to be for one project will be different than for another!
5/8/06 Modularity and "tolerance of uncertainty"
I was chasing down modularity as an important aspect of architecture, and came across Baldwin and Clark's work (again, but it has been a while). They make the point that a modular architecture is "tolerant of uncertainty" and "welcomes experiment" in the design of modules, so that modular architecture leaves our options open. We know this (e.g., see introduction), of course, but I do like the way they put it.
Well, it is not quite so simple, but within the scope of possibilities that the architecture leaves open, a modular architecture has the property that well-contained modules can be changed with less impact (less cost of change) on the rest of the system.
The term marketecture is often used to talk about the idealization of the architecture of the product or product set that is projected "up and out." It is used somewhat wryly by developers because it is rather different than the reality that the development world sees. Why is it different? The system evolves through a series of accommodations in response to customer needs, competitive threats, and emerging opportunities in conjunction with the ever-present pressure of the rush to release.
Can we take off the pressure to release? Should Apple have delayed the release of Aperture? Should Microsoft just let the Vista team release when it is ready, and save the team all the angst of missed releases? If we don't have a target, we don't know what we are shooting for. Should we make our target public? If we do, we certainly make it clear that it is going to be painful to move the target, so we'd best do due diligence in scoping the feature set and get on with getting the work done. But what if, despite our best efforts, we miscalculated? Should we:
For a first release, with competitors nipping at our heels, it is arguably the right move to ship even if the product isn't quite solid yet, as long as it does have a compelling value proposition along with (minor) outstanding bugs. At any rate, I'm glad the Vista and Aperture decisions weren't my call!
But, can we get back on track, if we see we are off-target on the ship date? Probably not. Managers are indicted for making the mythical man-month error over and over. And technical people tend to err on the mythical-engineer side. Gene Krantz (director of NASA mission control during the Apollo program) reputedly said "Engineers will wildly overestimate what they can do in one year and wildly underestimate what can be accomplished in ten." (Elsewhere, Bill Gates is credited with being the first to say something along these lines, but wikipedia gives this honor to Joseph Licklider who put it in a footnote in a paper in 1965.) Engineers tend to think that the most effective way to get a truly great product shipped is to have 2 to 4 truly great designer/developers working on it. Regardless of the fact that it would still take 50 top-talent-person years to complete?
If we want to create truly exciting products that evolve through a series of iterative refinements, some of which are internal milestones and some of which are externally released, we need to maintain a clear architecture so that we can push that mythical man-month millstone out of our way! We simply have to get better at enabling people to be productive whether they are in a 4 person product team or a 40 person product team, or even a 400 person solution team. Which is not to say we should think we can ramp up from 20 people to 150 in a matter of weeks! Just a rough computation and we see that would mean each of the 20 would, on average, have to bring 7.5 people up to speed on the project! That means dead halt for those already up to speed on the project and slow creep up the learning curve for those new to it.
See also "Tech's New Headache: Feature Creep," by Kenji Hall, Business Week, April 12, 2006, at http://www.businessweek.com/globalbiz/content/apr2006/gb20060412_598932.htm
See also "Why we all sell code with bugs", by Eric Sink, The Guardian, May 25, 2006.
Modularity and what we can learn from Trek
I have a Trek bicycle, but with Shimano gears and brakes, and Bontrager frame, wheels, tires, and pedals, the ineffable Trek quality cannot be pinned down to any component. The success factor here: modularity with clear interfaces, and clear design requirements pushed down to the modules so that excellence at the level of the overall system is in good part achieved through the composition of excellent pieces. Bicycles have been around a long time, and there is a dominant design. That allows innovation to be focused on the components and subsystems, with less frequent revolution in the overall decomposition. Together these 3rd party pieces following a standard decomposition (2 wheels, frame, braking system, gearing system, etc.) nonetheless yield a system that is unique and differentiated. Modular components driven to excellence in of themselves, plus a clear sense of the differentiating system qualities, yields a competitive product.
In the software world, in the pressure of the moment we allow ourselves to accommodate the architecture. Every compromise to the modularity of the system increases the future cost of change. But we live in the present. We have to get out of this singular mindset, and part of the way out is to have an architect with the clear and well-known charter to preserve the structure of the system—to defend modularity and ensure that the modular pieces do indeed work together to deliver the functions and properties required.
Feature set teams or storyline teams (or whatever else you call them in your group) tend to work across the layers in a system. This work partitioning is good for driving out features that users can respond to and give feedback on, so they can be improved. Good. But it also makes it important to empower an architect to defend and preserve the architecture, and yes, evolve the architecture when, on balance, that is the best route.
It is the architect's responsibility to create (with collaborative input from developers) the architecture, identify the modules, and address the design decisions that cross module boundaries and involve multiple modules to deliver functionality and system properties.
Objects, then components, now services, are all hyped because they promise to address the build-from-parts need in our software world, at ever greater levels of granularity. As if that will solve our problem... We have to have good parts—good in the sense that they really fit our need. I acknowledge that I am fundamentally, completely biased here, but I can see no other course than through getting better at architecture, and better at protecting and preserving our architecture, and better at investing in evolving and regenerating our architecture. And better at investing in what it takes to completely revolutionize our architecture, even though that means we have to do so with a whole new set of people, which is counter-intuitive (see Rechtin's book Why Eagles Can't Swim, 2000).
5/10/06 Continuing the Modularity Discussion: Parts (on their own) are not the Panacea
Russell Ackoff uses this exercise in seminars (and we have borrowed it from him):
Collect together a team of the best automotive design engineers in the world. Assign them the task of selecting the best car component of each type. Will they be able to create the world’s best car from these components?
No! Even if they could all plug together, they are designed to optimize with respect to different property or quality goals, and different relative priorities among the system property goals.
So parts, on their own, are not the panacea or proverbial silver bullet we have been looking for. But parts that have a clear role in a well-architected system, offer promise.
The caveat: can we create good, even great architectures, when what we are doing is innovative? If the software we are working on is helping us to deliver competitive advantage, then along some dimension, we are necessarily looking to innovate, so we are necessarily pushing beyond what we know well.
But consider again, the Trek bicycle. A lot has stayed the same, and yet everything is different, from the bicycle I had as a graduate student in Palo Alto 15 years ago! And it is not just because my budget is different now. It is also because competition has driven Trek to innovate and leverage: modularity enables Trek to use 3rd parties best-of-breed parts within their architecture, along with their own parts, to create a unique offering on the market.
5/10/06 Architects by Title and Architects by Role
The architect (and for larger projects, the architecture team) is responsible for architectural decisions, that is decisions that need to be made by the person or small group that has the perspective and authority to make decisions that are in the best interest of the overall system, taking into account both business strategy and the lifecycle cost consequences of these decisions.
In some organizations, "architects" give advice that the development teams can choose to take into account or ignore. In this case, they have the title but not the role. They are internal architecture consultants. They help bring consistency and help raise the bar of design practice in the organization. But they are not, generally, the architects of the systems being developed in these organizations.
In these organizations, and in organizations with no formal architect title, developers make the architectural decisions, and hence "developers" are playing (some of) the role without the "architect" title.
By bringing the title and the role together, and giving the person with the title the chance to fully play the role, we put the organizational structures in place that allow us to start to have the conversations and set organizational expectations about behaviors so that we can get the benefit of architecture.
Experience tells us that without having someone (or a team, for larger projects) responsible for the architecture, the architecture is no-one's priority, and it gives way to what is on the priority plate—features that need to be released SOON!
5/10/06 HelpMatch Strategy and Architecture
A small group of architects in the Indianapolis Architects Community has been working on the HelpMatch initiative. We have an interesting situation, in that there is no existing organization, so even our enterprise architecture is greenfields! And we are in a situation that will reverberate for many enterprise architects: there is no business strategy. But for us, there is no business as yet. So we get to craft the business strategy, not just influence and interpret it.
So most of our work to date has been on the strategy, using our "Identity, Value Propositions/ Capabilities" strategy framework which leverages important strategy tools like context maps from Grove and strategy maps from Kaplan and Norton.
We started by creating a Context Map. We have also created Context Maps for HelpMatch during the Software Architecture Workshops in Indianapolis (context map, other models) and Las Vegas, and the Role of the Architect workshop in Indianapolis.
We worked on Identity/Value Propositions/Capabilities descriptions for competitors, and this gave us a better sense of how to distinguish HelpMatch from other organizations and services out there.
"Competitors" in the HelpMatch World
Internet-based Donor Match Programs
Inkindex allows manufacturers and retailers to donate excess inventory and get tax benefits from donating (via Inkindex) to non-profits. The company can claim the donation at least at cost rather than having to sell the inventory at below cost just to move the goods. It works like this:
Individual donors can enter their telephone area code and get a list of requests from non-profit organizations that share that area code. The get the contact information for the non-profit and from there they need to contact the non-profit to arrange the donation.
Corporations can make direct product donations via Inkindex directly to non-profit groups, or indirectly through barter (what they donate gets bartered for cash, goods or services that the non-profit needs) or through auctions (donated products are sold through auctions and the proceeds are donated to the non-profit).
Non-profit organizations submit product donation "wish-lists" online. Donations are matched to non-profits through two routes: 1) Individuals view wishlists and contact non-profits directly, and 2) Inkindex member corporations add product donation listings online. Inkindex automatically matches donations with "wish-lists," and notifies organizations of matched donations via email.
The Inkindex strategy overview
in the table above,
Market Gap is HelpMatch Opportunity
Within the US, at least, the internet is ubiquitous enough that even without additional infrastructure (like call-centers), we could do a lot more to match those who need help to those who are willing to provide it. When individuals help individuals you remove the distribution logistics burden that has caused the big institutions to steer clear of donations of used goods.
Alternatively, we can support the distribution logistics by supporting collaboration within networks (either existing community groups, corporate response groups, or ad hoc groups) that will provide the organizational/man-power side of the distribution system.
HelpMatch Exploration: Turning a threat into an opportunity
The seed of an exciting idea (sponsors) was planted during the Indianapolis Role of the Architect, and it jelled with an idea (networks) that we were getting excited about at the end of the Indy architects meeting in March. Al de Castro and I took up this idea in the April meeting, and this is where we went with it:
Donor Concern: how do I know my donation is going to someone with authentic need? How do I know my goodwill is not being taken advantage of?
Opportunity: Provide authentication through sponsor networks. A sponsor will form a HelpMatch domain or network of individuals or groups who are vouched for by the sponsor. For example:
Sponsors could be allowed to set policies which would define:
HelpMatch could also set policies, for example, setting trust levels
Trust levels may be limited within HelpMatch domains, or generalized across all of HelpMatch.
The Role of Network Sponsors
A sponsor is an individual or organization who wants to help, and who sets up and uses a HelpMatch-supported network to do so.
HelpMatch would then provide a portal to the sponsored networks, and donors and potential recipients could search for and register with a network, and then donate to, or find help within, the network.
After Hurricane Katrina, the Indiana University Student Housing Office started to look for interim housing for parents of IU students from the hurricane-stricken region. People could call IU and offer to put up hurricane-displaced family members of IU students. Effectively, IU was vouching that these family members were indeed bona fide displaced families. IU got the word out about its efforts through local public road and the local newspaper. The response rate was good, and more people offered to open their homes and take in families than were needed.
This sponsor network is already a feature of our informal nationwide disaster response system. But how could we better support it? What if IU could use HelpMatch to set up an IU-sponsored network? This would promote the network, but they would also use traditional media like radio and newspapers, and emerging media like blogs and Craig's List. IU could allow family members from the disaster-stricken areas to register for assistance through the network based on their relationship to IU students whose home address is in the disaster area. They could then enter their needs, or they could search through the donations database (where offers of temporary housing would be a kind of donation). Or students could come to an IU office and working with IU staff, register their families and enter the needs their family has communicated to them.
Implications for HelpMatch Capabilities
Establish Identity: HelpMatch needs to establish an identity in the marketplace that sponsors want to be associated with. We need to create a snowball effect, where non-profits and corporations want to get on board.
HelpMatch System Capabilities:
5/10/06 Comments on the Strategy Process
Our primary focus is on differentiating value. What do our "customers" value? What are their concerns, needs, goals, and how does this translate into value we could provide? What can we do uniquely, that will provide differentiating value?
What opportunities have our competitors left on the table, and how do these opportunities relate to their identity and capabilities and how do they relate to our identity and capabilities. If the opportunity is more aligned with their identity and capability set, we need to consider what it would take for us to compete if they seize the opportunity too, and we should see if there are other opportunities that are better aligned with our identity and capabilities.
We also need to look at our value network, and consider what opportunities that presents. And we need to think about building our value network, to reinforce the value we deliver and further differentiate.
5/11/06 Do We Need Architects?
I'm asking this because I get asked it, not a lot, but from time to time. Of course, I have an opinion. I recognize my opinion is contaminated. For more than a decade I have been working with a biased set of people, namely architects and managers who have realized they need to pay more attention to architecture. But everyone is biased in one way or another.
So let's just look at this statement:
Experience tells us that without having someone (or a team, for larger projects) responsible for the architecture, the architecture is no-one's priority, and it gives way to what is on the priority plate—features that need to be released SOON!
Is it true that we can't expect designer/coders to simultaneously defend and mature the architecture and cut feature set code to the fast pace of the release drumbeat?
In the product world, many, many, many projects are in the situation that the system is generally poorly structured, with increasing concerns about their ability to keep shipping releases on time. In the IT world, it is called the maintenance burden, and in company after company maintenance activities absorb the biggest share of the annual IT budget.
Here and there we have teams who believe they can create and preserve a good, clean architecture without having designated architects, and I'm sure they can, since they are obviously sensitive to the need for the balance. But the evidence tends more to reinforce the claim that we do need architects, and an organizational culture that supports architects, to change the outcome that we keep ending up with.
5/11/06 HelpMatch Gathers Momentum
The Indy Architects meeting made good progress. I was reminded that Freecycle needs to be included in the competitive analysis. Kurt had pointed us to it some time back. At the time, I went to freecycle.com which is an entirely different thing than freecycle.org! Of course, there is more I need to write up, just to capture work we have already done, and the work we did today took us leaps ahead.
I added Freecycle to the competitor table. Freecycle.org is very much like Craig's List in that it is like a classified ads system online, except that it is for finding a taker or a giver for free goods (usually used).
We think we can address the trust barrier--namely addressing the donor concern around abuse of trust if just anyone can pose as someone impacted by a disaster and make off with donated stuff.
This allows us to present a very exciting value proposition: people who lose personal property, homes, and livelihoods can be helped by others all over the nation (at least, and, in time, all over the world) by matching help to need, removing the logistics burden of individualized help, and providing mechanisms to protect the giver from fraudulent abuses on the recipient side.
I really believe we have an exciting set of value propositions that make for very, very exciting architectural challenges.
This can really make a difference in the world, and it is really exciting to work on something this IMPORTANT to mankind! The need for this system is already there, with tsunamis, hurricanes, mud flows, earthquakes, and other disasters leaving bereft people still recovering months and even years later. And the next large-scale disaster is looming, what with cities on fault lines, cities in the shadows of volcanoes, dense populations in tsunami zones, terrorism, wars, avian flu or other pandemics, global warming (more hurricanes and tornadoes?), and more.
So, what are you doing to get involved? Yes, you! Keep up with the Indy group's HelpMatch progress and participate in what we're doing. You can review work done, add to it, work ahead of us on some piece of the architecture, refine and improve pieces that have been drafted, or backfill pieces that so far we have only brushed over. You can do similar things differently, and add that way. Or you go about it entirely your own way, and show everyone how you do it.
To help in two ways at the same time, work on HelpMatch and share your work. By doing the latter, you contribute to a shared body of practice that we are trying to encourage to help us build architect community and knowledge.
Right now you can publish HelpMatch artifacts on the Resources for Architects web site, but we are also putting in place a HelpMatch web site. The latter will have the benefit that it will not be a vendor site, although vendor's can sponsor the site. But the Resources for Architects site is intended to be a comprehensive resource for architects and it has the advantage of being pretty well known in the architecture space. It is a vendor site, but it is a technology-neutral vendor at least!
5/11/06 HelpMatch Networks
The following outlines some possibilities for sponsored networks in HelpMatch:
5/14/06 Mother's Day and Staying Out of the Way
It is Mother's Day and my kids cooked breakfast and bought it to me in bed. Very sweet—they're 6 and 8, and it was a big leap to let them cut the bread and use the stove entirely on their own while I stayed in bed. I gave some guidelines: which hotplate on the stove to use and what temperature setting (I strategically asked for scrambled eggs, thinking forward to cooking temperature and safety). My only rule was "no fighting." (If they start arguing they lose track of details like where the knife is...)
The only anxious moment came when I heard plates: clatter, clatter, clatter, clatter, clatter... I stayed in bed. It was hard. But it was their big treat for me, and I didn't want to spoil it by intervening in the end-game. It turned out my son was getting the plate he had painted himself, and that was on the bottom of the stack of 16 or so dinner plates... He moved each one, one at a time. No accidents. The scrambled eggs and toast were great, if just a tad cool—all that plate moving had it's cost, but the hand-painted plate was the perfect way to present a kid-cooked breakfast!
Sometimes, we just have to stay out of the way.
Lead, follow, or get out of the way. Contribute your best effort, and let go where you are not needed.
For the architect, getting out of the way on decisions that are not architecturally significant means you free others up to be creative, move quickly, and learn. And you keep your architecture minimalist, so that the architecture itself doesn't get in the way!
For decisions that someone else is leading on, demonstrate good following. That doesn't mean you can never play devil's advocate, but watch your behavior and don't do it all the time! It doesn't mean you have to "go along' with everything, switch off, damp the passion, mute your qualms, ... either. There is a time for divergent, out-of-the-box, creative thinking where you explore opportunities, but project leaders need to start to drive toward convergence at some point, or the project will never be done. This happens early in a project when the scope is explored. And it happens all along the way, in smaller, more focused areas as challenges crop up, options are sought and weighed, and choices are made.
Good following means maintaining goodwill, contributing your best effort as appropriate to the phase: is this exploration or is this time for convergence to decisions and execution on decisions? If you don't agree with the direction and decisions, you have to do some soul searching. Is it just you, just what you care about, or are things really on the wrong course for the business? If you put your own personal interests aside, and truly believe bad decisions are being made for the business, then you have to figure out what to do about it. But you first have to consider this: if you were looking at this issue from the perspective of the bigger picture, would you come to the same conclusions? And if things are not done the way you think they should be, what is the likely result? Will it be good enough? Often good enough but done in time is a whole lot better than perfect but takes too long, and endless arguing may get you closer to your idea of a perfect solution, if you win the argument, but how much does it slow things down to be tied up arguing? Often, arguing is really important to seeing a better way to do things, that no-one was considering. But there comes a time to stop arguing and move forward. Has your leader decided that time has come? Or is your leader too caught up in the arguing and someone else needs to call time and remind the leader?
Social factors, yes. Getting software built is a social process, because it involves collaboration on a collective work product. Goodwill greases the skids of collaborative work.
5/15/06 HelpMatch "Competitor" Red Cross/Red Crescent
If we place ourselves in the future when HelpMatch exists, then HelpMatch "competes" with others who provide assistance to those in need. It competes for supply of donations. Hence it competes with local non-profits like Goodwill and Salvation Army, or even garage sales for the local non-profit schools. It competes for cash donations (so that cash donations could cover shipping costs, or setting up assistance centers like internet kiosks in disaster areas). It competes for attention, and attention is needed to drum up cash and other donations. So it competes with Red Cross and Habitat for Humanity and others.
We have to think about this. We do not want to take away from Red Cross and other institutions playing a vital role following disasters. And we want to provide an avenue for people to help people in those weeks and months following disasters where they need to rebuild their lives. So we need to be clear about the value propositions of "competitors" and the value proposition of HelpMatch. The value proposition of HelpMatch must be clear, and compellingly different from those of its competitors, not to out-compete them but to provide something that is needed but not already part of the landscape. (This is a somewhat like Blue Ocean Strategy, but for a philanthropic reason.) The only reason to get excited about HelpMatch is if it is clear it will have huge value to mankind and that value is missing until something like HelpMatch is brought into existence.
5/15/06 Architects on a Pedestal? or Architects for Target Practice
I was reading Scott Ambler's piece on agile architecture. I was running along pretty well ... until I came to the table summarizing "common practice" and distinguishing it from "agile practice." Here Mr. Ambler has positioned himself as attacking "common architecture practice" by choosing words that slight, like "pedestal" and "hands dirty."
For your ease of reference, here is Scott Ambler's comparison:
A lot is asked of architects, and respecting talented architects should not be disparaged; respect is earned the hard way in software, whether it be for talent and contribution as managers, developers, testers or architects. Architects, as technical leaders, need to be credible, experienced, smart; esteem is not misplaced. Besides, when the architect is held in high regard by the team, this does nothing to diminish the other team members--not unless the architect is arrogant, and arrogance and competence, with its corollary of respect, are not joined at the hip!
As for traveling light: Architecture models and documents play many roles. They help the architects think about the many, many challenges they have to balance and address. They help the architects get input, create a shared mind-space where others can play a role in seeing the alternatives being considered, pose new approaches, throw challenges at them, and advance the understanding of the system and what it must do. And the models document decisions, so these decisions can be explained and shared. And they create a record of decisions, so these decisions can be referred to (and defended and amended if necessary) during all phases of the life of the system. How much documentation you need is context dependent. Tomorrow can take care of itself better, if it is clear what we did today! Code as architecture, code as model of the system, just does not scale as a communication tool when you're trying to build and evolve 100,000 to million++ LOC systems.
Some suggest that it is an unnecessary overload to model twice, once in architecture diagrams and again in the code. Yes, it does mean we have to keep the architecture models (and supporting explanations, rationale, etc.) up-to-date and that takes conscious additional effort, which is hard to motivate since code is what our organizations typically value and measure progress on. But it is not duplication of effort. Architecture is used to orchestrate the social work of building and evolving complex systems and architecture is used to tackle hard design challenges and make tradeoffs while it is relatively cheap to play out various alternatives and assess their relative efficacy. Architecture is architecture. It is good for what it is good for—thinking about, designing and communicating the overall structure of the system and the important architectural mechanisms.
Both aspects of architecture are important: the overall structure and the mechanisms. If we design the system to be composed of components (or services, or whatever we call our architectural elements) then we can use components as units of innovation and experiment, units of change. But then we need to understand the role these components play in the system, and what their interfaces are so we can access the services provided by those roles. And, if we are going to carve up the system into pieces, we need to address the properties of the system as a whole and the properties that cut across various pieces, so that the system will have the behaviors and properties our stakeholders care (most) about.
In complex systems we need a role, that of architect, that specializes in addressing the structural soundness of the system, paying attention to a host of concerns like performance under normal and peak load (like seasonal spikes in transaction volume) and performance under hard-to-predict conditions produced by the interaction of applications using shared resources, security (like SSN and identity management across integrated applications), and so on. This is not just system decomposition or "navigation views," but attention to cross-cutting concerns.
Can we do this without architecture documentation? To get a cheap chuckle in workshops, I make the point that the only thing harder than getting anyone to read the architecture documentation is getting the architects to write it! So why bother? Well, as an architect you need to model. It is a critical thinking tool for complex systems. Besides, your head very effectively encapsulates your thinking, unless you speak, draw and write. So you at least need to speak and draw, to create a shared mental space where others can influence and get access to your thinking. And now we have a chance to make it "our" thinking, our architecture, because it is shared, not just in one head.
Can we stop at conversations and drawings? Even for more complex systems, that can be good enough to get going, and in any event is irreplaceable, no matter how complete our specification. But conversations as the only architecture communication vehicle, rely on the person(s) who has the architecture in their head being there. What, you never want to go on vacation? You never want to go home at 5pm even once a week to take your son to soccer practice? You always want to be working on this system? Really, for the next 10 to 20 years? What if you get hit by the proverbial bus, or poached by a start-up offering you shot at CTO of the next big thing? And now how about this: do you remember what you (personally, or the collective you, the architecture team) were thinking when you made one choice over another, three years ago? three months ago? three days ago? And just how did this mechanism work anyway? Why did we think this would work? Documentation creates a record of our decisions, for ourselves, for our teams, and for the future. And documentation explains and justifies our decisions. [9/20/11: Sharing the thinking about why the system is the way it is, how it works conceptually, what boundaries need to be preserved, keeps the team in command of its architecture. Conversations help, but documentation make the reasoning accessible in different ways, asynchronously but also to deeper cogitation.]
Does this mean we need to create a truckload of documents? The US Constitution was just 4 pages. Truly, elegantly minimalist. Still, 85 essays (The Federalist Papers) were written to explain, justify and motivate various aspects of the Constitution, to get the buy-in needed so that all 13 states would ratify the Constitution. And Madison took verbatim notes during the convention that Summer, and these are available to us still today. We need to be minimalist in what decisions we make. And we need to be clear about what we need to communicate, to whom, and how best to do that. As architects, we need to take the long view when we do this. Yes, there are hard choices to make to allocate our finite bandwidth over pressing demands, but we do have to weigh in the strategic not just the tactical.
Can we do this in advance of coding? Do we know enough about our system to do so? Architecture is also about risk management: identifying risks and strategies to decrease or avoid them. In very novel spaces, we may take a quick-and-dirty, learn-and-fix, approach. Quick cycles through modeling and experimenting through code helps us grow our understanding and reduce risks. Whether we create an architecture upfront, or along the way, we need to get more disciplined about building systems with structural integrity (soundness, consistency, etc.) and leaving a trail of models and supporting explanations to make evolution of the system more a matter of engineering than heroism. We should be advocating not deprecating models and documents as we try to inch towards a software culture that produces higher quality systems that deliver (just as much) gain without (quite so much) pain.
Architecture is not a problem we face once and then never again. If we do a good job, we don't have to think about it all, all the time. But if we keep innovating even within the boundaries cast by the architecture, we will need to keep the architecture views up-to-date. And, yes, since we don't have a crystal ball, and even if we did, we couldn't solve all the future's problems today, we will find that we have to change the architecture.
We can take care of tomorrow's problems tomorrow only if we don't postpone today's due diligence until tomorrow! If we create a kludge today, that is where we start tomorrow.
The pace of change is so fast that we heartily want to be agile. But we have to bear in mind that agile means different things to big and to small projects. Big projects may be the only way to be agile (quick and responsive) when you want to create something really big. Remember the Empire State building: 7 million man-hours, but completed in 16 months! That was extreme agile! And when we go after big things, like a next generation platform for our product line, we need to get extreme agile on a larger scale than the agile and extreme communities are generally talking about! Yes, extreme agile on that scale takes more upfront architecting to allow the kind of planning it takes to co-ordinate the effective contribution of more teams, of teams, of teams. That kind of architecting has many of the good properties of architecting that Scott Ambler talks about in his (generally very good) "agile architecting" essay, which, by the way, is actually very much the kind of architecting that is in fact in common practice—when architecting is in practice!
The problem is not the architecting style that is in "common practice". The problem is that a good many projects are not commonly practicing architecture! And many of these are using the trendy "agile" banner to authenticate their omission. Instead of slighting architects by associating disparaging images with architects in general (common practice), it would have been better to head the columns "Bad Practice" and "Agile Practice" or something like that.
5/17/06 Mentoring Architects
Dana and I have been informally mentoring architects, and formally working with architecture teams, for more than a decade. We have decided it is time to roll up some ideas we've been playing with for some time, and offer a formal (so we know how to price it) mentoring program that will help individual architects set personal goals and make accelerated progress towards meeting them. Here's the blurb I roughed up so that I can start to get reaction and feedback:
Many businesses already know that senior management takes special talent and skills, and they invest heavily in developing and nurturing that talent. Architecture forms the platform for strategic advantage. A business that wants to lead into a new era of strategic advantage through architecture needs to pay the same kind of attention, make the same kind of investments, in its architects. It is not enough to have a program for identifying junior architects and helping them to transition into that role. We need to groom talented architects for senior architect roles that are as demanding as executive management positions in terms of their "soft" skills, yet also require technical depth and systems design experience and talent.
We are offering a fast-track program for architects that is analogous to programs that are offered to promising managers being groomed for the executive track. It requires the commitment of the business to nurturing its architect talent, and commitment from the architect to stretch beyond current job demands to learn concepts and skills that will help the architect do better on the job today, and advance to senior architect roles with broader and more strategic scope of impact.
We are offering a limited number of mentoring opportunities to individual architects. When you sign up for the mentoring program, you will be assigned a personal coach. Your coach will meet with you and help you assess your strengths and weaknesses, and set goals for your professional development. These goals will help you and your coach tailor a program that will help you move closer to your goals, enhancing your on-the-job performance. Your coach will review and give input on your current architecture work, and help you look more broadly to develop skills you need to advance as an architect. Your coach will help you sort through what to read, where to focus, what to practice, and give you safe, private feedback that will help you see and value your strengths so that you can lead and empower others.
This program is most helpful to architects who want to advance more quickly to senior architect positions, and senior technical management positions with an architectural leaning. We have created this program because quite a number of architects that we have worked with are now in chief architect, CIO, and CTO positions. They have reached these positions because they have the right qualities for these roles, not because we "retrofitted" them with these qualities. But at the same time, they have found our architecting process and our focus on architects skills to have been most useful to them, and they are strong advocates of our work within their organizations.
It is not intended to be a big program,
but rather one that is focused on just a small number of promising
architects. Organizations have realized they need architects. The next
big "ah ha" is realizing that architect talent can be developed and
honed, just like management talent, but it takes investment and
attention. The proposed program aims to groom promising architects for
success by helping them be more successful on their immediate projects
and directing their attention and helping them navigate through building
skills that will help them be more effective technical leaders and
advance rapidly on the architect career path.
5/17/06 Architecture Handbook Hiatus
Grady Booch changed the software world with his OOAD work, with huge ramifications for everything that software touches. He is poised to change the world again with his architecture handbook work, but he will be taking some forced downtime to undergo heart surgery at the Mayo Clinic on May 30. I wish Grady and his wife strength through the preparation and wait, and hope for a speedy recovery. I selfishly hope that Grady can return soon to work on his Architecture Handbook with the silver lining of no travel interruptions for quite some time to come!
5/17/06 What we can Learn from Building Architects
Ok, at the risk of someone (probably me) drawing a cartoon of a building made up of software packages... I just have to return to this topic... I was surfing on something else, and quite by accident came upon the AIA site, that's the American Institute of Architects (building architects, that is)... I don't know why I hadn't thought to go there, but I'll take this as a happy accident.
At any rate, here's a quote right out guidance to building architects:
"Winning now requires a complex blend of political savvy coupled with bulls-eye content, psychology of audience appeal, people skills, and influential strategies."
and the advice on how to present to sponsors is generally relevant to us!
Ok, intriguing. Oh, and the site offers a
Primer on Project Delivery: "To help architects and contractors achieve quality projects that fulfill owner expectations and are delivered on time and on budget, the AIA and the Associated General Contractors of America (AGC) formed a task force to produce a primer."
Hmm, so my neighbor wasn't the only one who's home building project was "released" several months late.
Do look at these excerpts from the Architect's Handbook:
I like this: "Design brings to the surface the major considerations inherent in a situation. It is a process that is both problem-seeking and problem-solving." from Ten Key Factors that Affect Any Design.
5/18/06 OK, I'll come clean
I mentioned I'd stumbled on the AIA site and that I hadn't thought to go there before. I've always somewhat liked the building architecture metaphor, and somewhat been suspicious of it. My real discomfort has been around the huge amount of attention that is needed directly in the software architecture space, attention that is diffused if I pay any attention outside the space and, besides, the Sewell's (over?)did that "draw parallels from the building architecture space thing" already... I pay attention to a variety of spaces as it is, from creativity to leadership to ... to system modeling to ... to resource management and performance tuning.
But my late evening fly-by on the AIA site was quite interesting; interesting enough to return to with prime early morning attention!
Because we're working with Ben Ponne on revamping and updating the Architect Competency Framework, I was intrigued to see that, even though building architecture is a pretty mature field with well-established university programs and formalized body-of-work (like the Architect Handbook), they are investigating some of the same territory that we investigate. For example, see the piece on "fast-track to must-have status."
As you've seen, I have been waxing (in)eloquent about the architect's role during the construction phase. Hello, what's this? It's an issue in the building architecture field! In that field, architect up-front only also issues! Here's a piece encouraging architects to get back in the game of construction, though not directly with a hammer and nail, but maintaining intellectual control throughout so that compromises and adjustments to stay on schedule and within budget are made quickly by the architect! Here are some excerpts:
"Think about it. Replace adversarial relationships with teaming. Make changes during construction to improve the project without fearing a costly change order or delay claim. Solve problems on the spot to keep the job moving and reduce risk of suits or claims over design errors and omissions. Experienced design-builders say, “When you have total control, you have less risk.” The traditional method of turning over control of your plans to a group of low-bidding contractors, some of whom may be looking to exploit any error, omission, or ambiguity in those necessarily imperfect plans and specs, certainly has its risks. Doesn’t retaining control make more sense?"
"From 1909 until 1978, the AIA forbade its members from design-build due to a perceived conflict of interest. We’ve come a long way. In September 2005, the AIA Board of Directors adopted policy statement No. 26 on Alternative Project Delivery methods that says regardless of the delivery method used, the AIA 'believes that an architect is most qualified to lead' and the AIA 'advocates that architects should be retained in that role.'..."
"Great architecture has always been the result of a great designer. For centuries those great designers also led the construction team. Names like Brunelleschi and Jefferson are synonymous with “master builder,” or capomaestro, the one who led the entire process from design through construction. But over the years we have become a fragmented profession. Engineering, though part of architectural education and licensing, is nearly always delegated to a consultant."
from "Taking the Lead in Design-Build" by G. William Quatman, Esq., April, 2006
5/18/06 HelpMatch and Disaster Recovery
The AIA identifies these "Three Stages of Disaster Assistance:
5/19/06 Blogs and the Feedback Revolution
Blogs have gained speedy popularity because they enable a community of interest to form around a topic, led by someone who is passionate enough about the topic to blaze a thought path for others to react to. The essayist gets feedback, which is good for motivation, but also stimulates thought. Blogs provide the opportunity for people to get into the discussion, so it is not just unidirectional. It is somewhat unfortunate that the momentum to interact in that forum, though, often comes from disagreement rather than agreement and extension.
And when something ominous looms, like Grady Booch's impending open-heart surgery, and Scoble's mom's evidently inevitable impending death, then people who usually are quiet observers on the sidelines start to show up with words of support and recognition.
5/21/06 HelpMatch and Networks as Agents of Change
It occurred to me that HelpMatch networks will become a very dynamic way to raise awareness for need, and build support for change at the network level. For example, someone who cares about what is going on in Darfur/Sudan could start up a network that concerned others could plug into to find out how they could help, find out about rallies and ways to put pressure on those who set policy, find out about aide, and so forth. HelpMatch networks could integrate the best of blogs, the best of LinkedIn, and so forth.
Software people, architects and engineers, this is a big thing we can do! The idea is here and growing. Become involved. This can make a difference in the lives of those that really need it. And it can make a really big difference in your life, now by engaging in something really important, and later, if you ever need help recovering from a disaster.
5/22/06 Just Say "No" to Your Architect
But Nick goes on to say "And then, for the folks who work inside the fantastic and sometimes naive boundaries defined by their architecture, the snooping of the architect becomes meddlesome, irritating, and unwelcome." We do need to invite component owners to look at their own motives, and consider whether this stance is in the overall interest of the business. If you want the architect to understand and stay current with your particular set of challenges, yet you want to shield your work from the architect, then you send your architect out wandering in a barren desert just as surely as management does when they have no (communicated) strategic direction! Snooping is unwelcome. But is it snooping, or willingness to partner, explore, learn and mentor? Ah, goodwill!
It is also not true that the architect needs to "stay out" of the components. What is true, is that the architect needs to be hands-off on what is not architecturally significant. But the lines of architectural significance aren't simply drawn at the boundaries of the components. Life is not so simple. The architect needs to watch for what is architecturally significant, and engineers need to watch for what is architecturally significant and bring this to the attention of the architect. Yes, we need goodwill and partnership, not turf protection, to reach team goals.
When issues surface that challenge the architectural approach, the architect needs to listen, and the development team(s) needs to advocate their position. The architect needs to balance the positions of different teams, or individuals. The architect needs to make the call if the teams/individuals cannot reach a consensus position, or if the architect believes that their position is not in the overall best interests of the system, balancing short-term exigencies and long term strategy. Decisions that are part of the architecture need to be adhered to. If not, we don't get what the architecture promises, and we take the short road to project chaos.
Of course, if you have an incompetent architect, you're in a bad way, no matter what! But in truth, do you really have an incompetent architect or a peevish team (or team member)? Times are a-changing, and there are some who like the good old cowboy days where everyone got to make decisions based on local exigency and survival through this present skirmish. It is hard to give up power. This is why Mandela and de Klerk were awarded the Nobel Peace Prize!
5/22/06 Architect's Linkblog
If you haven't stopped by lately (or subscribed to the RSS feed), definitely check in on Kevin's Architect's Linksblog. I always find Kevin has scouted out interesting news and articles. He visits the broad scope of topics relevant to an architect (he is one), from technology news to the hard and soft aspects of architecting.
5/22/06 Architecture Documentation
I was reading back through our Meta-Architecture chapter and I realized it does a pretty good job, and the piece on documentation covers some important ground. The process overview chapter needs to be written afresh... but I'm reasonably happy with the Meta and Conceptual chapters. I'm working on two fronts on the writing side—the Logical Architecture chapter and GEAO Journal article on the Architect Competency Framework. And of course I'm busy on other fronts besides. We're getting momentum behind our architect coaching program, and that is truly exciting. I'm also setting up some 1-day architecture events through the Summer months. All these, and ongoing project work, keeps me busy. Which gets me back to the point I was going to make: I need to scale back to focusing on the GEAO article so that means explicitly tabling the Logical chapter for a little. In the meantime, if you have any thoughts on Logical architecture in the context of SOA, I'd be interested in reading them.
5/24/06 SaaS and Services
When I was working at HP Labs in the early to mid 90's (I then moved over to HP's internal consulting group that was, at the time, called the Software Initiative) we got excited about the notion of pay-per-use software services, and we saw downloadable and composable software services as a wave of the future. It should come as no surprise, since Martin Griss was the head of our department, and we were working on the "Flexible Software Factory" project and one of the issues in reuse is "who pays." It is still an issue, and keeps coming up, even in the financial sector which is pretty aggressive about staying on the leading edge of technology and development practice.
One way SaaS potentially simplifies all this is by externalizing the market for software services. It is somehow easier to set up pricing models when the transaction occurs across business boundaries. I hear architects talk about setting up pricing models internally, but it is a hard sell. Still, as the SaaS paradigm gains momentum, we will not only learn how to architect and build software as services better (dealing with issues like application and data security including integration issues like identity context propagation; performance monitoring, load balancing and alerts, etc.) but we will establish better practices for running SaaS businesses (dealing with issues like pricing, metering and billing; as well as selling, servicing, versioning, certification, and hosting, delivery, infrastructure security, etc.).
5/24/06 HelpMatch in the Financial Marketplace
The Economist has an article on bankless banking through "community" networks and the Internet! I checked out Prosper.com and it is exciting as an analogy for HelpMatch. In particular, Prosper has this concept of "groups" that seems to be analogous, but different, from the HelpMatch networks concept. Group leaders tell a story to get lenders to join their group, and borrowers to borrow from them. So the idea has precedent, but it is not (yet) in place for disaster aid, plus it is architecturally very interesting.
5/24/06 All Heated Up over Building Analogy
Goodness, people sure do get riled up over using building architecture as an analogy to software architecture (see The Software Construction Analogy is Broken, and comments). I never thought analogy implied the source and subject were "identical" or "the same in every way" or even "the same in most ways," but rather that we can usefully apply lessons and information from the source to the subject domain. I think this is a true statement: Complex buildings have been around longer than complex machinery. So we are applying some lessons from the design of complex buildings to the design of complex systems (physical and software).
Complex buildings require many people, and many person-hours, to build. This takes co-ordination, not just at the scheduling level, but at the content level. For complex software systems, this content-level co-ordination is all the more important because the content is innovative and highly experimental, so all is not known in advance. We have been doing this co-ordination on-the-fly, person-to-person, but our systems are becoming ever more complex at the same time as our development organizations are becoming more distributed (with telecommuting, outsourcing, farm-shoring, off-shoring, near-shoring, etc., etc.!). So we need to develop ways to partition work so that the experimentation and innovation can still happen, yet we can get products released on relatively short (and ever shortening) timescales driven by global competition. And that's just the decomposition/work partitioning angle. The building architect designs a sound and aesthetic structure that meets functional demands. The system architect is also chartered with structural integrity and intentional design to meet system quality requirements, while meeting functional needs.
The problems are not the same, the techniques are not the same, but there are lessons that are useful at an abstract level. I don't think it is particularly useful to dwell on the analogy, but at the same time, getting all hot under the collar about borrowing the architect/architecture role/concept and leveraging it in the software (and other systems) space seems like misplaced energy.
5/24/06 Architects and Bricks, Language and Tom Peters
I was happy to be able to quote from Simon Munro's blog post titled "Do Architects Lay Bricks?" on the Role of the Architect page of the Resources site. Simon writes in a racy style (the racy dipped below the below the belt in the original post but that now is fixed). The title is a nice play on the (building) architect debate!
Even Tom Peters seems to think that crude language is ok on blogs, but there's language that spends time in the toilet and there's language that writes women out of the conversation ... and so far I have seen Tom Peters being generally careful to apply the right strokes when it comes to his female audience, as long as they tolerate toilet talk that is. Personally, I am not inclined to see it as a sign of an evolved intelligence, but rather as a code of entry into some kind of club that I never did "get" the point of... but I realize that is another one of those personal biases I carry around as baggage.
Why do I mention Tom Peters? Well, I'm sort of expecting I'm going to have to buy his "Design" book, and have to actually read it, and so forth... because it's going to be talked about in management circles and architects are going to have to know what the management folk think design is a la Tom Peters... But on a cursory scan at Barnes and Nibble (as my kids call it), it appears that amidst all the buy-me-get-me jumpy type and photos (that caused me to not buy it right off) there seems to be some points I might want to stop and consider. And at under $10 for a full-color book(let), the price is amazing. Even a DK travel guide costs at least twice that!
5/25/06 Growth Crisis, Architects and Architecture
Actually, in truth the debate around architecture and the role of the architect is not so widespread as the attention it gets might signify. Many, many organizations have well-established architect roles, titles and organizations.
Nowadays, the companies that come to us for advice on setting up the architect role tend to be those that have grown quickly from startup to success over the past 2 to 5 years and are facing the challenge of entering a new life stage. They have all the culture of the young upstart company, but their engineers have seen it through the start-up phase, and after throwing long hours at bringing the company to success, they are now ready to have family lives, and their spouses are insisting on it, and they are (and feel) entitled to it. But all the factors that made it so successful as a start-up—the aggressive, "now"-focused management team, the do-what-it-takes engineers—are at odds with getting to an even-keeled, tempered, approach to getting good follow-on releases shipped and broadening the product portfolio through a rationalized product platform (software product line) approach. Such companies have a difficult shift to make. They've gone from small to medium-sized in a flash, and suddenly meetings and consensus and process are the bug-bears of the day!
In the glory days, a few smart, energetic people drank from the same chalice of vision and that was enough to propel them speedily through the first incarnation of their product. But success breeds customer feedback, demand for new features, and on and on, and suddenly the organization is caught up in ambitious growth, and architectural compromise after compromise... And before you know it, the company is successful, it has 200 developers, and they and their code are falling all over each other!
5/26/06 OpenOffice and Extensibility
Yesterday I went spelunking in OpenOffice territory. I realized that it is very hard to do, for there is no handy map and I'd need a personal guide or I'd have to explore with my flashlight and boy that is slow going and hard to see the big structures and remember my way!
But I also realized that OpenOffice is extensible in way that I want to pay attention to and learn from. Now, I also want to get HelpMatch ratcheted up to the point where I can use all my connections to get more great minds engaged in it (I say more great minds not referring to myself, naturally, but to Barry Crist, Kurt Kirkham, Al de Castro and Jeff Price and Gene Shin, in the Indy Architects Community). And Ben Ponne is waiting on a new draft of the Architect Competency Framework, and I need to get that Logical Architecture chapter finished, and on top of all that we need to earn a living... so there's all the client work that takes precedence over all else.
So why did I head out on a brave excursion into the OpenOffice underworld when there's so much else to be done? Well, HelpMatch of course! OpenOffice is a nice example of a complex, and highly successful Open Source application development project. Good component model, great extensibility mechanisms.
Following up on the Architect's Linksblog link to this Leadership blog and from there to Tony Manning's work on strategy, I took a look at chapter 1 of Tony Manning's book. Right on page 1 there's a point very applicable to architects as they create architectures:
"Win 'votes': You have to exist in harmony with many stakeholders, and you have to persuade some of them (especially your people, your suppliers, and your customers) to volunteer their imagination and spirit to your cause."
Manning also says:
"The first task of a leader is to provide a clear point of view – 'There’s the hill we’re aiming at … these are the results we want … this is how we should conduct ourselves … here are our priorities … this is what we’ll do to get where we want to go.' This is the context in which people work.
The ongoing task is to focus and inspire them. We all know that 'what gets measured gets managed.' But we conveniently forget that it’s only what is spoken about – constantly, passionately, consistently – that will be either measured or managed. Talk about the right issues in the right way to the right people, and extraordinary things happen; but get the conversation wrong and you’re sunk."
Tony Manning, Making Sense of Strategy, Ch. 1, p. 15.
5/27/06 Architects are Writing their Job Descriptions
In many, many organizations, architects are writing their own job descriptions. I mean this quite literally! And they write their project charter, and they determine how they should be evaluated. Odds are, if you are called an architect, you're in this situation. But that is not really new. I've been writing my job description ever since I started working as a software engineer. If you're bright and hugely, off-the-charts, productive, you're probably going to have a lot of say in what you do and how you should be measured. Well, that' s been my experience, and that is what I'm seeing broadly with architects.
Even in organizations that have had the architect position in place for years (though they may call it tech lead, senior scientist, senior or chief engineer, etc.), people in these roles are actively determining what the role means, and how they ought to be measured. The frontier that we work at shifts, so it is not that managers don't understand what architecture is and ought to be and so on; it is more that what architecture is and needs to be shifts as our organization's architecture capability matures. So managers are actually allowing their organizations to be more adaptive when they let their architects set the agenda and associated expectations.
5/27/06 Deep Smarts
I've been looking at the book reviews for Deep Smarts (Leonard and Swap, 2005) on Amazon, trying to decide whether to buy it or not. Anyway, it reminded me that I've been wanting to say that intelligence is an important attribute of a successful architect. "Deep smarts" may be a way of articulating that it is more than intelligence, but a way of cutting through to what is important and accumulating knowledge that matters. Well, I still haven't decided on the book. So far, it hasn't made it onto the list. I need to check if there's an HBR rendition summarizing the main points.
Allan Hoffman at Monster.com is researching the software architect role for an upcoming article in Monster.com, and he asked me to give input. Here are his questions (in italics) and my responses:
A. What are the key elements differentiating the role of a software architect from related roles (project manager, programmer, and so forth)?
The role of software architect particularly comes into play for reasonably complex projects. These require the co-ordination of a fair number of people as they create, improve and extend a collective work product. This is not just a matter of scheduling and workload balancing. Software construction is very far from route manufacturing assembly. Rather it is a process of invention and inventive evolution.
Software architecture has to provide a context for this innovation, creating boundaries within which programmers can work with a high degree of freedom, while nevertheless delivering a system with a coherent identity, with parts that collaborate to produce services with service levels that not simply emergent, but rather the result of a reasoned, goal-oriented process.
Software architects are the people with the skills and experience needed to design these architectural elements that become units of work, and units of innovation and experiment, containing the cost of change through the initial experimental stages, and throughout the evolution of the system.
Software architects need to have real system development experience. But they are not just technically great. They also need to have skills that are more like those of managers, because they need to lead various communities (including developers, but also managers, and others), persuading, consulting, mentoring, and so on.
Software development is innovative work; getting people to work towards something with a coherent identity and integrity is a matter of getting people to voluntarily contribute their imagination and best effort to working towards a common vision, within a common setting of goals, principles and overall system structure, and with a good idea of the commitments their piece of the system must contribute to the overall system purpose.
Setting system goals is strategic, motivating and aligning people to achieve them is leadership, helping people understand them and be successful is consulting, navigating all the relationships and networks of influence to get things done in organizational settings is organizational politics. The architect must have all these skills, or partner very, very well with someone who does!
B. Is there a typical career trajectory for becoming a software architect?
I haven't formally kept statistics, but this is my sense:
Software architects typically have a degree in computer science and work as programmers for 5 to 8 years, before they are considered for architect roles. Within 3 or 4 years, promising developers are often playing tech lead on some chunk of the system. But this is a gross generalization.
Talents vary, and opportunities vary. I know some great architects who majored in mathematics, for example. The main factor is programming experience on real production systems. I know some architects who got there after just 3 or 4 years programming. And I know lots of architects who are 40+ who had some 15 years of programming before getting the title of architect, though often they have been playing the role without the formal title. There are still organizations today that avoid the architect title/role.
I recommend you take a look at my essay titled "Quest for Great Architects" which quotes from and links to a number architect job postings, and makes points about the skills these hiring managers are looking for, for different types of software architect job (from junior architect to senior architect working across broad organizational scope). See http://www.bredemeyer.com/Architect/QuestForGreatArchitects.htm
C. What advice would you give a young techie who believes he/she would like to become a software architect?
Spend time on our Resources for Software Architects web site (www.bredemeyer.com), understanding what software architects do, and what skills they need to have, and think about whether you like to do this kind of work and have these kinds of skills. A perfectly good alternative is to focus on being a super-programmer. Very technically focused people are vital to success too, and should be rewarded for growth along technical specialist dimensions. The architect career track is for people who want to design systems, and who are good at solving system design problems; that is, design problems that impact the achievement of system goals, addressing cross-cutting concerns, and enabling the system to deliver on the business strategy as it applies to that system.
Then, if you still want to be an architect, try to find opportunities to get depth and breadth of experience. Try to get to work on a project from early design all the way through the first few releases. Try to get more responsibility; for a component, for a project team, etc.
Your technical base is what will qualify you for an architect role. Your leadership, strategy, organizational politics/effectiveness skills, partnering and consulting skills will help you be successful in the role, and hiring managers will, more and more, be looking for these broad skill sets. Broaden your reading, look for a business mentor, not just a technical mentor, seek role models who are good leaders, and so forth.
5/29/06 Honorable Distinction: Per Henrik Lausten
Per Henrik Lausten has the (I think) very honorable distinction of being the first blogger to link to my Trace in the Sand blog. And he is complimentary about our Resources for Architects site. We get indirect feedback when people come to our workshops because they like the work we have on our web site. But we get very little direct feedback on the web site, so it is heart-warming and noticeable when someone takes the trouble to say something nice.
5/30/06 VAP Testimonials
Well, this is a rewarding time: Per Lausten's blog post yesterday, followed by some very nice customer feedback on VAP (the Visual Architecting Process) today. We want to submit VAP to The Open Group for inclusion in their list of recognized architecting methods for IT Architect certification. Naturally, they put out some hoops to jump through, and one of these is "user testimonials."
I don't like to bother architects for testimonials, and rather let our work speak for itself; word-of-mouth keeps our business going, but prevailing upon architects to play a role in our marketing seems a bit much! Still, in this case I had to ask, because I feel that having VAP recognized for the certification process will help architects who are using VAP—otherwise architects would have to submit VAP for recognition. This is not onerous, but if we do it, we save architects who are using VAP that step on the certification path. Anyway, I pinged a small set of architects asking for testimonials and they have started to come in.
Most of the feedback we get is implied. For example, we have a growing number of architects who have taken our workshops several times. This is most emphatically not because they are slow learners! It is because when they start up new projects, they want to start out having their team learn VAP and they know our workshops are not just training classes, but really effective team start-up events where the team gets onto the same page in terms of vision, process and visual language and lexicon, and they work quickly and effectively through a rough cut at the architecture. They are by no means done after the 4 days of the workshop, but they have stakes in the ground that make it easier to see where the next round of work needs to be done to assess and address risk, innovate, and iterate to mature the team's vision and architectural approach. Anyway, we don't need these architects to wax on about the value of VAP; their actions speak volumes for the value they perceive.
5/30/06 EA and the Strategy Conversation
I was reading through "Updating the Clinger-Cohen Competencies for Enterprise Architecture", and in Appendix A they quote the CIO Council Federal Enterprise Architecture Version 1.0 as saying:
"An enterprise architecture (EA) establishes an agency-wide roadmap to achieve the agency's mission through optimal performance of its core business processes within an efficient information technology (IT) environment. Enterprise architecture are 'blueprints' for systematically and completely defining an organization's current (baseline) or desired (target) environment. ..."
There is this notion, even in the business world, that Enterprise Architecture is about IT efficiency, and Jeanne Ross even goes so far as to say that enterprise architects do not need to know their company's business strategy, but only "how your business is organized—how integrated and standardized its information and processes are" (quoted by Christopher Koch in a sidebar to his article titled "A New Blueprint for the Enterprise," CIO Magazine, March 1, 2005).
If business strategy isn't communicated to the enterprise architecture (EA) team, it can do a good job given where the business is today, based on what the team knows today, and based on the team's best guesses at capabilities that need to be built to ensure competitiveness into the future. The EA team members are smart, and they've been around this business block a while, so their best guesses are likely to be really promising avenues for the company. But relying on best guesses, even the best guesses of smart, well-intentioned, business-savvy architects, is not exactly the most robust route to business-IT alignment!
If senior management isn't setting the direction, then others are. If the company does not have a stated, shared strategy, then where it is headed is a matter of where the real strategy makers take it. Yes, the real strategy makers may be people in marketing, product development, whoever has a say in what actually reaches customers and how effectively it does so. And yes, this includes the EA team, if the executive team never talks to the EA team about the strategy.
This strategy conversation may happen through the CIO, but it is strategically useful to the company if it sees the CIO and the chief enterprise architect as complementary and includes both in the strategy conversation. The CIO is responsible for IT; the chief enterprise architect is responsible for architecting business capabilities. The chief enterprise architect is the bridge between IT and the business, responsible for business capabilities not simply automation of processes. When we see it this way, it is clearer that EA is not just about efficiency, but about differentiation.
IT supports the essential functions of the business. We have to satisfy business imperatives; these are the things we must do given the kind of business we are. We have to do these at least to the level of competitive parity. As enterprise architects, we want our business strategy to help us delineate where we are striving for differentiation and where we need simply to be good enough.
Information technology has changed so fast that systems put in place just a few years ago are already out-of-date, yet we are still limping along with core systems that are seriously, debilitatingly geriatric at 20 or so years of age! This means that part of the strategic agenda has to include modernizing systems that may or may not be central to our differentiation strategy, but core to our business.
Of course, even with this in mind, enterprise architects need to be thinking strategically: they need to think about the organization's path into the future and what it will take to be competitive not just in the short term but into the future, as far out as it makes sense. For example, if we set out to modernize key systems, we need to take into account what it will take to be at competitive parity when the systems are fielded and through their foreseeable effective lives. Thus we may appear to over-invest if we take just the near-term into account, because we are investing for parity over the major portion of the life of the systems.
IT affects the "quality of life" of much of the organization. When FrontPage crashes, I am vexed at losing an hour of writing, but when the "credit check" service fails due to an unmitigated single point of failure and the business loses mortgage customers, that causes a lot of stress for many different people.
5/31/06 Project Wipe-out: Big Failures
Architects help manage risk. Some say: We learn from mistakes. I say: The best mistakes to learn from are someone else's, so here are some big failures to learn from!
Software project failure:
Software system failure:
Win some, lose some?
Bugs, Slugs and Work-arounds
Learning from failure:
Gems and Keepers
The most precious gem found on this mining expedition:
"...a club that began in 1945 when engineers
found a moth in Panel F, Relay #70 of the Harvard Mark II system.
I might keep this one around for when my kids are teenagers:
Editorial Comment on Software System Failures
These horror stories (I have to force them out of my mind every time I drive my Toyota!), neglect several points of note:
Even when these failures cost nothing more than hours, it is a big drain on productivity, and a high stress burden. The gain is high. The pain is sporadic, and usually hits below our pain-versus-gain tolerance threshold and we grumble a bit, but put up with it.
Still, ignoring the pain is not a very resourceful way to approach our future health!
Software development is a complex, human endeavor. Even the very smartest, best people make mistakes. It is a hard truth, but bugs are hard to eliminate—entirely. Some of the approach to bug control is architectural:
And some of the approach to bug control is through process (as poisonous as that might be to some developers):
And some of the approach to bug control is through management discipline:
On Software Project Failures
See Steve McConnell's "Classic Mistakes Enumerated" for a well-researched treatise on mistakes to avoid on software projects.
[6/2/06: I just got the June issue of CrossTalk in the mail, and low and behold the theme is "Why Projects Fail."]
5/30/06 Tacit IT
"... IT networks are now a commodity, but IT supported tacit networks can be a competitive advantage."
Christopher Koch, "Can you do 'Tacit' IT?, IT Strategy blog on CIO.com, December 15, 2005.
5/30/06 Microsoft Moves to Dominate
Apparently Microsoft and IBM are dueling for the hearts and minds of architects, and Microsoft is making some big plays. Here's the current informal "scorecard":
Microsoft has an advantage in the push to make its presence pervasive, because it has explicitly decided to knit webs of community influence within the IT space:
"... the company is setting its sights on developing a sense of community among users, and especially among the powerful 'IT professional' category.
Microsoft isn't doing this on a whim. According to extensive internal studies, a robust and responsive community is the No. 1 satisfaction driver among IT pros."
And who better to do this with than architects, for they top the technology food chain!
This means that we get a lot of really great architects on both sides of the Microsoft-IBM domination wars getting sponsored by their business to share their thinking. This sets a context in which everybody else has to scramble for a piece of the attention pie. So we have a whole smorgasbord of great thought being aired with generally free access—often without even the intrusive requirement of creating and keeping track of yet another login and password.
It is changing the name of the information game, and changing the influence channels. I still like Harvard Business Review and MIT's Technology Review. I'm not in a rush to squeeze out all attention space for solid articles, sifted through a stringent selection process, in juried journals. But between the web sites, webcasts and the blogosphere, there's more than enough to cause the sane person to take cover from the information explosion.
5/31/06 Grady Booch is on the mend!
Thanks to Grady's blog, we know that Grady's heart surgery yesterday went well, and that he is already out of ICU! Something big like that gets one to look at the priority short-list, and his brain-child, the Software Architecture Handbook, has to be right at the top of the list along with other close members of his family! So, I'm confident it'll be getting attention even before the 4 week recuperation period is over.