A Trace in the Sand
Online Architecture Journal
by Ruth Malan
|
I also write at:
HelpMatch:
-
HelpMatch Wiki Trace in the
Sand
2006 - June By Topic (May) - Do architects need
to code? - Spelunking in the OpenOffice underworld
Input to Monster.com article
on Architect Career BY TOPIC (previous months)
HelpMatch Case Study: Architect Role
-
Architect Career Ladder Architect Skills Persuasion: Strategy Architecture Players in the Architecture Epidemic: Dealing with Complexity: Design Authority Risk Updates to Resources for Architects web site March updates:
|
May 2006 5/1/06 May Day It is Workers Day, and it reminds me how fortunate I am to work in a knowledge economy where I can be creative and do work I truly enjoy! In April, I dealt with subjects related to the role and skills of the architect. Some of this writing was packaged up into essays on the Resources for Architects web site. Some may appear in the forthcoming GEAO Journal article on the architect competency framework. But if you are interested in the architect role, you will find useful links, and discussion of various aspects of the role, in my April journal writings. 5/2/06 Do Architects Need to Code? In part, architecture is about enabling many people to work on a complex system, and still achieve something that is coherent. So the architect needs to have technical insight that comes from technical experience, and the architect needs to be good at achieving alignment with the technical direction, and getting buy-in to the architectural decisions that shape the system and resolve the big challenges. This takes persuasion and influence, skills we associate with leadership and organizational politics. In an autocratic culture, we may be able get away with less of the politics and influence and rely more on the authority vested in the architect role. But in the highly collaborative, consensus-oriented cultures that characterize most of the businesses we work with, the broader the scope of the architecture, the more the architect has to rely on leadership and organization politics skills to influence and persuade, align and get buy-in. Of course, if the architect has no credibility with the development community, all the political skills in the world will not overcome this deficit. Several blog posts (e.g., Ramkoth, Johanna) have levied criticism at architects who don't code, and there's not a whole lot of empathy for the tension that the architects themselves are placed under. Many architects I know would love to have time to code (in their day job, too), but they have to carefully choose where to spend their bandwidth. The architect knows he could do a good job of the code, but if he was writing that part of the system, and that part, and that part, pretty soon he would run up against even his own high productivity ceiling and have to face the fact that complex systems cannot be built by one great architect/developer alone. Then you have to take more people on board. Then you need to communicate to them what to do. Then you need to take more people on board because that communicating meant you had less time to personally code, bringing down your own coding productivity ceiling... But if you do a good job as an architect then you raise the productivity ceiling of the team. And so it goes. There may be junior
architects or technical leads who "own" components and absolutely, yes,
still write code. And senior architects responsible for complex system
architectures, with a number of junior architects and developers
reporting to them for technical direction. This is a big job, and leaves
little time for code reviews, let alone writing code. Many of these
senior architects ask how they can stay in touch better with the
technical frontier and maintain the respect of developers they work
with. My best answer is to refer to those architects that I have worked
with, who have retained the respect of the development team even though
they no longer have the bandwidth to cut code. They are extremely smart.
They pay attention to many sources, inside and outside their
organization. And they pair up with developers to solve
challenges they face. Their role is to
abstract and extract, create models, bring their own and others
expertise to bear on creating and validating the architecture,
synthesize practices to make common
across the team, find the mechanisms to use consistently across the
system. Architects are very much about expanding the creative capacity of the team, and achieving system integrity, system-fit-to-purpose and system-fit-to-context. Everyone plays a role in creating successful products and applications; not architects alone, not developers alone, not testers alone, not managers alone, nor marketing and business analysts alone. Yes, architects have an important role to play, and we need to let them play that role! In large projects, the architect's role should not be undercut by challenging the architect to load up on coding responsibilities and simultaneously keep the architecture solid and up-to-date. Something will have to give, and it will be the architecture. Just because a project enters the construction phase does not mean that the architects role is done and now the architect can take on a fully-loaded developer role. If we are trying to do less waterfall and more agile modeling and development, then we have to recognize that we are spreading architecture work out through the entire construction phase. When construction begins we should not view the architect as a freed resource that we can pull off onto another architecture project nor, for that matter, should we move the architect back into a (full-time) developer role, loading her up with coding responsibilities. The architect is needed through construction, and through all the releases, to explain and defend the architecture, and to modify it when the architect is persuaded this must be done. During construction, the architect is needed to explain, mentor and coach, help problem-solve, to watch for misapplication of the architecture, to watch for obstruction of the architectural intent, and to learn! To get the benefits of architecture in terms of decreasing lifecycle costs and increasing productivity (and agility) and predictability, we need to make sure the architecture is protected from accommodations that weaken the structure and integrity of the system, and improved as faults are discovered, and that the architecture as documented keeps pace with the architecture as implemented. Building a sky-rise is technically and organizationally complex. Man-hour for man-hour, building software is probably even more so! In software, from start to finish, we are creating as we go; we are not just executing on a plan. But we do need a plan to provide the context within which we can allow more people to create. Architecture in our software world is very much about enabling more effective collaboration. Collaboration that scales up to the kinds of numbers that we rely on to get complex products and applications shipped, has to be enabled. We can't just assume it will happen. We can't expect it to happen in the large, without processes and tools to facilitate the communication and co-ordination involved. By tools, I don't necessarily mean computer-supported tools, though they could be. I mean, at least, the architecture diagrams and documents explaining and justifying the architecture decisions and approach. But all these architecture documents cannot stand alone. Collaboration is about personal involvement, and people are best at activating personal involvement—people with good leadership, persuasion and influence skills, that is. Yes, good to great architects! Architects who understand that it is not just about the technical soundness of the architecture, but about getting the development community actively and effectively engaged in a collaborative effort with customer and business value as the goal. And in his rather timely way, The Pragmatic Architect has a synergistic post on his blog. This is the kind of blog I find really exciting, because it tells the story of an architect's personal journey. [See also Should Architects Code? Bill Barr, May 2007] 5/2/06 Architects and Requirements Ron Jacobs, of Microsoft, has two pieces of work worth a visit. One is a video clip on Requirements and Constraints (some heavy breathing and camera shake, but a good introduction to requirements and constraints, and nice goal metaphor). The other is a "beyond bullets" style presentation on Architecture and Architects. Thanks to Mike Platt for pointing to Dick Hardt's outstanding presentation on Identity 2.0. Presentation style to aspire to! 5/3/06 Software as Collective Work Product I was just renewing my subscription to Harvard Business Review, and since I was there, was scanning through the archive of back issues on topics that interest me because they are important to my effectiveness and the effectiveness of other architects. One article that immediately jumped out because it resonates with my journal entry yesterday, is Katzenbach and Smith's "The Discipline of Teams" (HBR, July 2005). They define teams as groups that produce a collective work product, and collective work products as those that reflect the joint, real contribution of team members. Software, whether for products or to support services, is generally a collective work product. But sometimes the groups that produce these work products become so large we need to organize the work so that we can have multiple teams, and teams of teams, all effectively, with personal investment, working on the collective work product. I have been recommending Katzenbach and Smith's book, The Wisdom of Teams (1993), to architects ever since I came across it in 1994. Their HBR article, "The Discipline of Teams," is worth a read. Some managers get uncomfortable at their turf being encroached on when I make points like this, thinking that it is the manager's job to set the vision and tend to the care and feeding of teams. And some architects get uncomfortable, thinking that they would far rather this was a manager's job so that they could just be left in peace to worry about how to achieve consistent SSO across federated applications integrated via web services, or whether this resource manager mechanism will scale as multiple chasses are added... Software teams are about two things: getting more expertise so we can build better products and applications, and getting more bandwidth. Yes, these are two things:
Architects who recognize that they are more effective when they enable their communities to be more effective, are well on the road to being great. It is not just about the architect making technically solid decisions. It is about enrolling others, and empowering them, to contribute their best. It is about inspiring and motivating, it is about generating goodwill, and it is about enabling co-ordination so that all the pieces will be the best they can be, and the system, composed of the pieces will be the best it can be. (And of course, by "best" I mean best fit to the need taking into account constraints and tradeoffs that have to be made.) Yes, the great architect is technically great. If it were just about the people side, then we could have stopped at technical project managers and we would not have seen software architects emerge in all kinds of organizations and industries over the past decade or so. There is a strong technical dimension to the role. You can't just slice-and-dice a system any old way and parcel the pieces out to teams and individuals to build and then recompose the pieces into systems that deliver the services with the intended characteristics or qualities! Even if you start with a common vision, but only have a rough cut at divvying up the system, then you either have endless unproductive bickering about what the pieces mean and who must do what, or you have people going off and making their best guesses at what they were meant to do and maybe, eventually, through immense hard work and mass self-sacrifice, get to something that works. But this path is littered with failures! So, it takes experience and talent to create a great architecture—architecture that clearly identifies the structures and mechanisms that must be built to deliver the differentiating value propositions and foundational services of the system. And it takes other skills to help the communities involved make their best contribution to the success of the system. The architect who understands that he must get the great architecture that is in his head into the heads of the team, is the architect who begins to get the full scale of the challenge! And no matter how great we think the architecture in our heads is, it will be greater if we include others in the process of creating it. We will understand better what the architecture needs to be, to meet the needs of all who have a stake in it. This includes those who will create it, and those who have a perspective on the value this system could, and should, offer.
In workshops, architects have commented on how productive their teams are, creating a draft of key aspects of the architecture in just 4 days. They ascribe this productivity to working with top-notch peers. This is certainly true, but it is also true that we insist on working through that fuzzy front-end team alignment stuff (context, vision, strategy) before we start to "peal the onion" working on layers of requirements and architecture specification, and validating all the way. 5/3/06 Resume Selection Criteria and Architect Skills When sorting resumes to staff an architect position, I would look for experience and successes that would speak to the characteristics of the architect. And in submitting my resume, I would not say " I am great at handling organizational politics." I would say, "I was lead architect for a product platform that supports 6 to 12 product releases per year, where the products span an order of magnitude variation in memory footprint and a price range from $90 to $500." I would let my successes speak for my skills. If I said I was great at organizational politics, I'd expect the hiring team to want to barf! But I could not achieve these results unless I had solved the technical challenges in my path and I was truly great at influence and persuasion, as well as communicating, consulting and coaching. But if I was posting a job description for a platform architect for my product line, and if I wanted to have the right person self-select to apply for the position I had open, I would make it clear that I needed someone who was capable of meeting the strategic and socio-political challenges inherent in creating a product platform to be used by different product groups focused on different market segments all with unique customer needs and constraints. And I would look for very strong domain knowledge, and technical agility and adeptness at picking up new technologies, as well as considerable actual experience developing systems like this. And if I was giving performance feedback to one of my platform architects, and I perceived that he was coming up short on influence and persuasion, I would recommend setting personal goals for improving in that dimension, look for a mentor to help the person, and point to some books and articles on the subject, including The Tipping Point and Never Eat Lunch Alone. And, as an architect, I view organizational politics as an area I need to get better at. I am not a natural connector and salesperson (in the Gladwell sense) , but when it comes to architecture I rise to the occasion, but could still stand to do better, much better! So I scan for help, and set goals, and practice what I learn, and try to get feedback so I can measure my progress against my goals. I'm not trying to be nauseating! I'm trying to be more effective at getting a variety of people, at various points in the project, to contribute their best work to the architecture effort I'm leading, and to gain and sustain sponsorship throughout the life of the architecture. This is part leadership, and part understanding networks of influence and using persuasion most effectively. 5/3/06 Architecture and Garden Sheds What we say about architecture and the role of the architect assumes a context in which architecture is needed. In the building space, we wouldn't call in an architect to design our garden shed. But if we were building a sky-rise, or even a 4-story apartment building, we would not contemplate undertaking the project without an architect! When we talk about the qualities of great architects, or the activities in the architecting process, we are assuming that architecture and an architect are needed. If it is a simple system (small in scope, and simple in the technical sense), and we nonetheless go through all the steps of architecting, then we must realize that we are doing this for reasons other than immediate necessity. A good reason to do it is to learn, or because we want to create a culture that values and understands models. Or we might do this to prepare for adding complexity later. But when the system is complex, we need to get to the point where we would simply be aghast if it were not architected! And be aghast if we were asked to renovate or add to the system and there was no architecture specification to use in planning and designing our changes. 5/4/06 Architecture and Collaboration Today, if we were just to do a good job of replacing our current system with a modernized, cleaned-up new system, we would face considerable challenges. But we also need to be forward looking, and put in place enabling infrastructure and innovative features that will make the system competitive when released (generally in 1 to 2 years time), and see it through a set of iterations that broaden our product set or add features and value over time. This means that as architects, we need to collaborate on many fronts, to get good ideas, to understand technical challenges, and create good, right, and ultimately successful architecture. The architect has to manage a balance between fast-track, closely-controlled and focused individual work, and opening up to influences from outside and inside the organization. A key skill is partnering; partnering outside and inside the organization to understand both how technology can be used to gain competitive advantage and to identify and manage the risks associated, and get the job done. 5/4/06 The Stool with Three Legs An architect in a leading energy company told me that their organization was being set up as "a stool with three legs:" business, architects, development. It was the first time I'd heard it put that way, although we are seeing quite a number of organizations setting up architecture practices within IT, so there is a separation from the developer community. Whichever way we go with organization structures, there will be upsides and downsides. If we create an architect group, we give the group identity, and we allow the group to focus on developing their area of expertise, to share and learn from one another. We make it clear where to go for architecture help. And we face the danger of creating an architecture "ivory tower." Nonetheless, in the building architecture field we see this. Architects work in practices. Construction workers work for construction companies. There are areas where this separation will become stronger in the software field. We already see implementation being outsourced quite extensively. We sometimes see the architecture being outsourced too, but, in an area that is core (in the Moore sense, that is, it contributes to competitive advantage) architecture is more apt to be kept in-house. In either case (architecture practice as separate group from development, or outsourced code development), the architects are going to lose touch with their development roots. This places the architect in a tough spot! Technology is changing all around us. Staying up on the deep ramifications of these technologies is tough. But then, this is true for developers too. They are getting very deep experience with what they are working on, but the relevance of that experience will shift over time. The architect needs to be watching for these shifts, scanning the technology space and maintaining outside relationships to do so. And the architect needs to stay in touch with the current set of development challenges, directly by prototyping and experimenting early on, and possibly implementing pieces of the system during construction, or indirectly by involving key developers in the design of aspects of the architecture, and validating the architecture with stakeholders who can help improve, and prove, its efficacy. We view validation as an activity that is undertaken early, and then often, as the architecture is iterated on, refined, and matured. Early validation exercises are undertaken with "friends" of the architecture effort, those who will take part in the exercise with a view to helping improve the architecture, not to bringing it down in flames before it has even half-baked. 5/4/06 Organizational Politics From time to time I have wondered whether we should use the term "organizational effectiveness" rather than "organizational politics." (See, for example, the Organizational Effectiveness book section on our site. That was created in 2001, and evidences how long I have been in this quandary.) As a technical community, we tend to have an uneasiness around "politics," as if it is inherently distasteful. I generally want to call it what it is rather than reverting to some more comfortable euphemism, but I am finding that I want a term that includes organizational politics and some other areas like partnering that combine better into "organizational effectiveness" than "organizational politics." I know at least some in Microsoft considered the term "organizational agility" but they settled on "organizational dynamics" in the competency framework that is alluded to in their certification requirements. Still, I think neither works nearly so well as organizational effectiveness. 5/4/06 Architects and Requirements I was (re)reading Rob van Ommering's essay titled "Things to do in Denver When You're an Architect." Rob has a section titled "Things I know I Didn't Do (But Should Have...)" and the first item is: "Talking to requirements managers: in our organization there was a unidirectional flow of information between requirements and development teams. This is a pity, since developers may help to invent new features (at least there should be an interaction)." We see this so much, that I have taken it up as something of a crusade, as you'll note in my journal entries over the past couple of months! We work with architecture teams and individual architects across many industries, both in IT and in product development. And Rob's situation is very common, and, yes, it is a pity. We recommend that architects bridge this divide, but many architects we work with see the gap as being bigger than they are. They have to spend energy overcoming so many other barriers that this one is just too much for them to take on. 5/4/06 in the US is 04/05/06 If you missed 01:02:03 04/05/06 (whether that was last month in the US, or today if you use dd/mm/yy), take heart! In less than one hour it will be 06/05/04 03:02:01! 5/4/06 03:03:03. Ok, if you missed that, you only have to wait 100 years... I tried to tell you in time! Oh yeah, I should have an RSS feed. Sorry! 5/4/06 Performance Contracts I was investigating some work on leadership and came across an article on Flow. I mentioned to Dana that I'd leapt from leadership to "Flow" to Patagonia's plan to create biodegradable shirts! And you know what he said? "Flow, what do you mean flow? Are you talking about Mihaly Csikszentmihalyi's book?" And he even pronounced it right (Mee-high CHICK-sent-me-high-ee)! I had never heard of Flow, nor Mihaly Csikszentmihalyi. I was impressed! Apparently if you read Flow you will remember its author Mihaly Csikszentmihalyi ten years later and be able to connect the word "Flow" to him. Now, this is interesting: In "The Discipline of Teams," Katzenbach and Smith ((HBR, July 2005) identify clear goals as important ingredients for high performing teams. Csikszentmihalyi believes that "flow has several necessary preconditions. These include having clear goals and a reasonable expectation of completing the task at hand. People must also have the ability to concentrate, receive regular feedback on their progress, and actually possess the skills needed for that type of work." (Flow:The Art of Work, by Ann Marsh, FastCompany, August 2005) This underscores much in our architecting process, where we emphasize creating an architecture vision and architecture strategy (objectives, scoping and priority setting, principles) which are all about setting clear and achievable goals. Then we work on understanding the architecturally significant requirements (functional and system qualities/properties/non-functional requirements and constraints). These are all about getting more and more clear about our direction and goals, and setting priorities and scope taking into account the constraints we must work within, to ensure that our goals are achievable. We have been emphasizing these activities because they are important to team alignment and motivation. Getting great software built is as much about the social dynamics of the team as it is about the code. XP and agile approaches are a lot about, ahem, flow. 5/5/06 SOA-Q Jimmy Nielssen is asking some important questions on his blog in a series he's titled SOA-Q. 5/5/06 Non-functional Requirements David Ing has a great (insightful humor) post on non-functional requirements called "The Performance Question." 5/5/06 Software Design Jack Reeves, "What is Software Design?" C++ Journal, 1992. James Shore, "Quality with a Name," April 5, 2006
I've landed in the blogosphere. I took Kevin's recommendation and used WordPress (afterall he's using WordPress). I checked, and www.traceinthesand.com wasn't taken (that surprised me! doesn't anyone else want to leave a trace in the sand? make difference, that sort of thing?). Now all I have to do is think of something to say! 5/6/06 First Blog Post I decided to start "at the beginning," with a discussion of what software architecture is, so that we have some context for subsequent discussions about the role of the architect. I will craft the blog post here, before putting it in the blogosphere, and will borrow from earlier journal entries, so don't be surprised to see some repetition in what follows: 5/6/06 What is Architecture? To set the context for subsequent posts, I thought we'd start with the topic of "what is software architecture?" Bass, Clements and Kazman's definition of software architecture (in essence, the high level structure of the system, described in terms of components and their externally visible properties and the relationships among them) has been very influential. While this definition has resonated with many people, the continuing discussion indicates some remaining uneasiness with definitions proposed so far. Rather than propose another definition, we accept the Bass et. al. definition and focus on the central concerns that software architecture addresses. (You might also like to take a look Chapter 1: Software Architecture: central concerns and key decisions.) Complexity and Cost of Change We need architecture to manage complexity and cost of change. Grady Booch puts it like this: "all architecture is design, but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change." Cost of change goes up as scope of impact increases, so this definition covers decisions relating to system decomposition as well as those addressing cross-cutting concerns. Cost of change is also high if a significant chunk of the system has to be revised or rewritten, so this speaks to challenging pieces of the system. Big Rocks First
But what are the "big rocks"? The architectural elements—the components and their relationships, yes. And architectural mechanisms addressing cross-cutting concerns or systemic properties, yes. Big rocks bear a high cost of change, yes. Is there more? Architecture Implements Strategy The architect of early generations of HP OpenView said "Architecture is the translation of business strategy into technical strategy." This definition focuses on the strategic nature of architecture. What is significant, in this view, is driven by what is strategic, and what is strategic determines how we will compete. How we will differentiate dictates the big things we must get right, what hard problems we will tackle, where we will innovate, and where we must be ahead of competition. And it allows us to accept good-enough along those dimensions where we are not trying to create competitive advantage. Of course, even "good enough" may be challenging, especially when taken in conjunction with where we aim to differentiate.
Architecturally significant decisions are those that must be made by the person or team who has influence and perspective across the system in order to deliver on the strategic objectives of the system. Architecture Balances Differentiation, Complexity and Cost The architect needs to balance the need to differentiate, with the lifecycle cost of the features and quality attributes we pursue. What we need to do, just to be in the game, constrains what we can accomplish in order to distinguish our products or services and business. For example, in many systems some level of security, scalability, and disaster recovery are threshold attributes not our avenues for differentiation. So we must develop mechanisms (authentication, encryption and firewalls; load balancing; failover, etc.) to address these challenges. Just delivering the base level of features and qualities is challenging, given the threshold set by intense competition in most markets. Others will have a say in what our opportunities are to differentiate (e.g., marketing). And others will have a say in identifying the challenges we face to build and field systems in our domain (e.g., development). The architect needs to play a role in balancing what we would like to do, with what we are able to do given our resources and capabilities. Architecture Expands Our Capability Moreover, the architecture needs to play a role in increasing what we are able to do, and increasing the value of what we attempt to do, by allowing focus. Focusing attention, enabling specialization and separation, understanding where outsourcing or licensing can be leveraged, reducing complexity and scope where it is not essential to our value proposition. Knowing what to focus on and knowing what we can ignore—both are key to success. Architecturally Significant Decisions So architecture helps us manage complexity and cost of change, and deliver differentiating value in alignment with our business and product strategy. Architecturally significant decisions are those that the architect (or architecture team) needs to make in order to address these concerns (strategy, complexity, and cost of change). They generally include the high-level decomposition of the system and address cross-cutting concerns. What we do is driven by our business strategy, how we do it is driven by cost of change. Furthering the State of our Understanding If you have a definition of software architecture that you would like to add, I encourage you do so on the Resources for Architects site or the SEI site. That way your definition will be placed in the context of its predecessors, and your contribution will be all the more evident. And by all means also add your definition as a comment to the post. Whatever else you might add, I hope that in the commentary on this post, you will share as concretely as you are comfortable with given the public forum (and your level of identity disclosure),
To set context for your observations, it would be very helpful if you would provide some background, such as
Don't worry if you repeat something that has been said. What challenges you face will certainly overlap with challenges others face. What is interesting is what is common and what is different across architecture efforts. 5/7/06 Architecture is Semantics-Free? Well, after I spent my weekend trying to get what architecture is about articulated (ok, so I also went to the Farmer's Market, planted flowers and veggies with my kids, went running through the woods with my family, read a lot, thought more, wrote a bit, etc.), I came across this statement by Martin Fowler: "I think SOA has turned into a semantics-free concept that can join 'components' and 'architecture'." (July 1, 2005) Gosh, my bubble is quite burst. Well anyway, you can make me feel better. See my blog post here, and do leave a comment. Naturally, some nice words would be good (you can practice your leadership and motivation skills on me, I don't mind) but what I'm really interested in is triggering some useful sharing of what concerns you are addressing through architecture. What challenges do you face, that you believe are uniquely solved by architecture? Sharing experiences is a huge value. If it is something we "already knew" it is valuable for the validation it provides. If it differs from our experience and perception, it helps us to see how context also shapes meaningful differences in what other people perceive. What they believe is quite likely well-justified in their context, given their experiences, constraints and opportunities. Our experience causes us to draw conclusions that shape our beliefs so that when we make assertions they may seem off-the-wall to someone in a quite different context, yet be completely, obviously, self-evidently, correct to us! There are a range beliefs out there about architecture, from "semantics-free concept" to "high-level structure of a system..." to "components and contracts" to... And it ranges from "just pretty Powerpoints used to impress management" to "a blueprint that allows distributed teams to work effectively on a collective work product that assembles smoothly to yield predictable services and service levels." Context matters. What architecture needs to be for one project may be different than for another. And what architecture is allowed to be for one project will be different than for another! 5/8/06 Modularity and "tolerance of uncertainty" I was chasing down modularity as an important aspect of architecture, and came across Baldwin and Clark's work (again, but it has been a while). They make the point that a modular architecture is "tolerant of uncertainty" and "welcomes experiment" in the design of modules, so that modular architecture leaves our options open. We know this (e.g., see introduction), of course, but I do like the way they put it. Well, it is not quite so simple, but within the scope of possibilities that the architecture leaves open, a modular architecture has the property that well-contained modules can be changed with less impact (less cost of change) on the rest of the system. 5/9/06 Marketecture on the Outside, Muditecture on the Inside The term marketecture is often used to talk about the idealization of the architecture of the product or product set that is projected "up and out." It is used somewhat wryly by developers because it is rather different than the reality that the development world sees. Why is it different? The system evolves through a series of accommodations in response to customer needs, competitive threats, and emerging opportunities in conjunction with the ever-present pressure of the rush to release. Can we take off the pressure to release? Should Apple have delayed the release of Aperture? Should Microsoft just let the Vista team release when it is ready, and save the team all the angst of missed releases? If we don't have a target, we don't know what we are shooting for. Should we make our target public? If we do, we certainly make it clear that it is going to be painful to move the target, so we'd best do due diligence in scoping the feature set and get on with getting the work done. But what if, despite our best efforts, we miscalculated? Should we:
For a first release, with competitors nipping at our heels, it is arguably the right move to ship even if the product isn't quite solid yet, as long as it does have a compelling value proposition along with (minor) outstanding bugs. At any rate, I'm glad the Vista and Aperture decisions weren't my call! But, can we get back on track, if we see we are off-target on the ship date? Probably not. Managers are indicted for making the mythical man-month error over and over. And technical people tend to err on the mythical-engineer side. Gene Krantz (director of NASA mission control during the Apollo program) reputedly said "Engineers will wildly overestimate what they can do in one year and wildly underestimate what can be accomplished in ten." (Elsewhere, Bill Gates is credited with being the first to say something along these lines, but wikipedia gives this honor to Joseph Licklider who put it in a footnote in a paper in 1965.) Engineers tend to think that the most effective way to get a truly great product shipped is to have 2 to 4 truly great designer/developers working on it. Regardless of the fact that it would still take 50 top-talent-person years to complete? If we want to create truly exciting products that evolve through a series of iterative refinements, some of which are internal milestones and some of which are externally released, we need to maintain a clear architecture so that we can push that mythical man-month millstone out of our way! We simply have to get better at enabling people to be productive whether they are in a 4 person product team or a 40 person product team, or even a 400 person solution team. Which is not to say we should think we can ramp up from 20 people to 150 in a matter of weeks! Just a rough computation and we see that would mean each of the 20 would, on average, have to bring 7.5 people up to speed on the project! That means dead halt for those already up to speed on the project and slow creep up the learning curve for those new to it. See also "Tech's New Headache: Feature Creep," by Kenji Hall, Business Week, April 12, 2006, at http://www.businessweek.com/globalbiz/content/apr2006/gb20060412_598932.htm See also "Why we all sell code with bugs", by Eric Sink, The Guardian, May 25, 2006. Modularity and what we can learn from Trek I have a Trek bicycle, but with Shimano gears and brakes, and Bontrager frame, wheels, tires, and pedals, the ineffable Trek quality cannot be pinned down to any component. The success factor here: modularity with clear interfaces, and clear design requirements pushed down to the modules so that excellence at the level of the overall system is in good part achieved through the composition of excellent pieces. Bicycles have been around a long time, and there is a dominant design. That allows innovation to be focused on the components and subsystems, with less frequent revolution in the overall decomposition. Together these 3rd party pieces following a standard decomposition (2 wheels, frame, braking system, gearing system, etc.) nonetheless yield a system that is unique and differentiated. Modular components driven to excellence in of themselves, plus a clear sense of the differentiating system qualities, yields a competitive product. In the software world, in the pressure of the moment we allow ourselves to accommodate the architecture. Every compromise to the modularity of the system increases the future cost of change. But we live in the present. We have to get out of this singular mindset, and part of the way out is to have an architect with the clear and well-known charter to preserve the structure of the system—to defend modularity and ensure that the modular pieces do indeed work together to deliver the functions and properties required. Feature set teams or storyline teams (or whatever else you call them in your group) tend to work across the layers in a system. This work partitioning is good for driving out features that users can respond to and give feedback on, so they can be improved. Good. But it also makes it important to empower an architect to defend and preserve the architecture, and yes, evolve the architecture when, on balance, that is the best route. It is the architect's responsibility to create (with collaborative input from developers) the architecture, identify the modules, and address the design decisions that cross module boundaries and involve multiple modules to deliver functionality and system properties. Objects, then components, now services, are all hyped because they promise to address the build-from-parts need in our software world, at ever greater levels of granularity. As if that will solve our problem... We have to have good parts—good in the sense that they really fit our need. I acknowledge that I am fundamentally, completely biased here, but I can see no other course than through getting better at architecture, and better at protecting and preserving our architecture, and better at investing in evolving and regenerating our architecture. And better at investing in what it takes to completely revolutionize our architecture, even though that means we have to do so with a whole new set of people, which is counter-intuitive (see Rechtin's book Why Eagles Can't Swim, 2000). 5/10/06 Continuing the Modularity Discussion: Parts (on their own) are not the Panacea Russell Ackoff uses this exercise in seminars (and we have borrowed it from him): Collect together a team of the best automotive design engineers in the world. Assign them the task of selecting the best car component of each type. Will they be able to create the world’s best car from these components? No! Even if they could all plug together, they are designed to optimize with respect to different property or quality goals, and different relative priorities among the system property goals. So parts, on their own, are not the panacea or proverbial silver bullet we have been looking for. But parts that have a clear role in a well-architected system, offer promise. The caveat: can we create good, even great architectures, when what we are doing is innovative? If the software we are working on is helping us to deliver competitive advantage, then along some dimension, we are necessarily looking to innovate, so we are necessarily pushing beyond what we know well. But consider again, the Trek bicycle. A lot has stayed the same, and yet everything is different, from the bicycle I had as a graduate student in Palo Alto 15 years ago! And it is not just because my budget is different now. It is also because competition has driven Trek to innovate and leverage: modularity enables Trek to use 3rd parties best-of-breed parts within their architecture, along with their own parts, to create a unique offering on the market. 5/10/06 Architects by Title and Architects by Role The architect (and for larger projects, the architecture team) is responsible for architectural decisions, that is decisions that need to be made by the person or small group that has the perspective and authority to make decisions that are in the best interest of the overall system, taking into account both business strategy and the lifecycle cost consequences of these decisions. In some organizations, "architects" give advice that the development teams can choose to take into account or ignore. In this case, they have the title but not the role. They are internal architecture consultants. They help bring consistency and help raise the bar of design practice in the organization. But they are not, generally, the architects of the systems being developed in these organizations. In these organizations, and in organizations with no formal architect title, developers make the architectural decisions, and hence "developers" are playing (some of) the role without the "architect" title. By bringing the title and the role together, and giving the person with the title the chance to fully play the role, we put the organizational structures in place that allow us to start to have the conversations and set organizational expectations about behaviors so that we can get the benefit of architecture. Experience tells us that without having someone (or a team, for larger projects) responsible for the architecture, the architecture is no-one's priority, and it gives way to what is on the priority plate—features that need to be released SOON! 5/10/06 HelpMatch Strategy and Architecture A small group of architects in the Indianapolis Architects Community has been working on the HelpMatch initiative. We have an interesting situation, in that there is no existing organization, so even our enterprise architecture is greenfields! And we are in a situation that will reverberate for many enterprise architects: there is no business strategy. But for us, there is no business as yet. So we get to craft the business strategy, not just influence and interpret it. So most of our work to date has been on the strategy, using our "Identity, Value Propositions/ Capabilities" strategy framework which leverages important strategy tools like context maps from Grove and strategy maps from Kaplan and Norton. We started by creating a Context Map. We have also created Context Maps for HelpMatch during the Software Architecture Workshops in Indianapolis (context map, other models) and Las Vegas, and the Role of the Architect workshop in Indianapolis. We worked on Identity/Value Propositions/Capabilities descriptions for competitors, and this gave us a better sense of how to distinguish HelpMatch from other organizations and services out there. "Competitors" in the HelpMatch World Assistance Programs
Internet-based Donor Match Programs
Inkindex Inkindex allows manufacturers and retailers to donate excess inventory and get tax benefits from donating (via Inkindex) to non-profits. The company can claim the donation at least at cost rather than having to sell the inventory at below cost just to move the goods. It works like this:
Individual donors can enter their telephone area code and get a list of requests from non-profit organizations that share that area code. The get the contact information for the non-profit and from there they need to contact the non-profit to arrange the donation. Corporations can make direct product donations via Inkindex directly to non-profit groups, or indirectly through barter (what they donate gets bartered for cash, goods or services that the non-profit needs) or through auctions (donated products are sold through auctions and the proceeds are donated to the non-profit). Non-profit organizations submit product donation "wish-lists" online. Donations are matched to non-profits through two routes: 1) Individuals view wishlists and contact non-profits directly, and 2) Inkindex member corporations add product donation listings online. Inkindex automatically matches donations with "wish-lists," and notifies organizations of matched donations via email.
The Inkindex strategy overview
in the table above, Market Gap is HelpMatch Opportunity
Within the US, at least, the internet is ubiquitous enough that even without additional infrastructure (like call-centers), we could do a lot more to match those who need help to those who are willing to provide it. When individuals help individuals you remove the distribution logistics burden that has caused the big institutions to steer clear of donations of used goods. Alternatively, we can support the distribution logistics by supporting collaboration within networks (either existing community groups, corporate response groups, or ad hoc groups) that will provide the organizational/man-power side of the distribution system. HelpMatch Strategy
HelpMatch Exploration: Turning a threat into an opportunity The seed of an exciting idea (sponsors) was planted during the Indianapolis Role of the Architect, and it jelled with an idea (networks) that we were getting excited about at the end of the Indy architects meeting in March. Al de Castro and I took up this idea in the April meeting, and this is where we went with it: Donor Concern: how do I know my donation is going to someone with authentic need? How do I know my goodwill is not being taken advantage of? Opportunity: Provide authentication through sponsor networks. A sponsor will form a HelpMatch domain or network of individuals or groups who are vouched for by the sponsor. For example:
Sponsors could be allowed to set policies which would define:
HelpMatch could also set policies, for example, setting trust levels
Trust levels may be limited within HelpMatch domains, or generalized across all of HelpMatch. The Role of Network Sponsors A sponsor is an individual or organization who wants to help, and who sets up and uses a HelpMatch-supported network to do so.
HelpMatch would then provide a portal to the sponsored networks, and donors and potential recipients could search for and register with a network, and then donate to, or find help within, the network. Illustrative Scenario After Hurricane Katrina, the Indiana University Student Housing Office started to look for interim housing for parents of IU students from the hurricane-stricken region. People could call IU and offer to put up hurricane-displaced family members of IU students. Effectively, IU was vouching that these family members were indeed bona fide displaced families. IU got the word out about its efforts through local public road and the local newspaper. The response rate was good, and more people offered to open their homes and take in families than were needed. This sponsor network is already a feature of our informal nationwide disaster response system. But how could we better support it? What if IU could use HelpMatch to set up an IU-sponsored network? This would promote the network, but they would also use traditional media like radio and newspapers, and emerging media like blogs and Craig's List. IU could allow family members from the disaster-stricken areas to register for assistance through the network based on their relationship to IU students whose home address is in the disaster area. They could then enter their needs, or they could search through the donations database (where offers of temporary housing would be a kind of donation). Or students could come to an IU office and working with IU staff, register their families and enter the needs their family has communicated to them. Implications for HelpMatch Capabilities Establish Identity: HelpMatch needs to establish an identity in the marketplace that sponsors want to be associated with. We need to create a snowball effect, where non-profits and corporations want to get on board. HelpMatch System Capabilities:
5/10/06 Comments on the Strategy Process Our primary focus is on differentiating value. What do our "customers" value? What are their concerns, needs, goals, and how does this translate into value we could provide? What can we do uniquely, that will provide differentiating value? What opportunities have our competitors left on the table, and how do these opportunities relate to their identity and capabilities and how do they relate to our identity and capabilities. If the opportunity is more aligned with their identity and capability set, we need to consider what it would take for us to compete if they seize the opportunity too, and we should see if there are other opportunities that are better aligned with our identity and capabilities. We also need to look at our value network, and consider what opportunities that presents. And we need to think about building our value network, to reinforce the value we deliver and further differentiate. 5/11/06 Do We Need Architects? I'm asking this because I get asked it, not a lot, but from time to time. Of course, I have an opinion. I recognize my opinion is contaminated. For more than a decade I have been working with a biased set of people, namely architects and managers who have realized they need to pay more attention to architecture. But everyone is biased in one way or another. So let's just look at this statement: Experience tells us that without having someone (or a team, for larger projects) responsible for the architecture, the architecture is no-one's priority, and it gives way to what is on the priority plate—features that need to be released SOON! Is it true that we can't expect designer/coders to simultaneously defend and mature the architecture and cut feature set code to the fast pace of the release drumbeat? In the product world, many, many, many projects are in the situation that the system is generally poorly structured, with increasing concerns about their ability to keep shipping releases on time. In the IT world, it is called the maintenance burden, and in company after company maintenance activities absorb the biggest share of the annual IT budget. Here and there we have teams who believe they can create and preserve a good, clean architecture without having designated architects, and I'm sure they can, since they are obviously sensitive to the need for the balance. But the evidence tends more to reinforce the claim that we do need architects, and an organizational culture that supports architects, to change the outcome that we keep ending up with. 5/11/06 HelpMatch Gathers Momentum The Indy Architects meeting made good progress. I was reminded that Freecycle needs to be included in the competitive analysis. Kurt had pointed us to it some time back. At the time, I went to freecycle.com which is an entirely different thing than freecycle.org! Of course, there is more I need to write up, just to capture work we have already done, and the work we did today took us leaps ahead. I added Freecycle to the competitor table. Freecycle.org is very much like Craig's List in that it is like a classified ads system online, except that it is for finding a taker or a giver for free goods (usually used). We think we can address the trust barrier--namely addressing the donor concern around abuse of trust if just anyone can pose as someone impacted by a disaster and make off with donated stuff. This allows us to present a very exciting value proposition: people who lose personal property, homes, and livelihoods can be helped by others all over the nation (at least, and, in time, all over the world) by matching help to need, removing the logistics burden of individualized help, and providing mechanisms to protect the giver from fraudulent abuses on the recipient side. I really believe we have an exciting set of value propositions that make for very, very exciting architectural challenges. This can really make a difference in the world, and it is really exciting to work on something this IMPORTANT to mankind! The need for this system is already there, with tsunamis, hurricanes, mud flows, earthquakes, and other disasters leaving bereft people still recovering months and even years later. And the next large-scale disaster is looming, what with cities on fault lines, cities in the shadows of volcanoes, dense populations in tsunami zones, terrorism, wars, avian flu or other pandemics, global warming (more hurricanes and tornadoes?), and more. So, what are you doing to get involved? Yes, you! Keep up with the Indy group's HelpMatch progress and participate in what we're doing. You can review work done, add to it, work ahead of us on some piece of the architecture, refine and improve pieces that have been drafted, or backfill pieces that so far we have only brushed over. You can do similar things differently, and add that way. Or you go about it entirely your own way, and show everyone how you do it. To help in two ways at the same time, work on HelpMatch and share your work. By doing the latter, you contribute to a shared body of practice that we are trying to encourage to help us build architect community and knowledge. Right now you can publish HelpMatch artifacts on the Resources for Architects web site, but we are also putting in place a HelpMatch web site. The latter will have the benefit that it will not be a vendor site, although vendor's can sponsor the site. But the Resources for Architects site is intended to be a comprehensive resource for architects and it has the advantage of being pretty well known in the architecture space. It is a vendor site, but it is a technology-neutral vendor at least! 5/11/06 HelpMatch Networks The following outlines some possibilities for sponsored networks in HelpMatch:
5/14/06 Mother's Day and Staying Out of the Way It is Mother's Day and my kids cooked breakfast and bought it to me in bed. Very sweet—they're 6 and 8, and it was a big leap to let them cut the bread and use the stove entirely on their own while I stayed in bed. I gave some guidelines: which hotplate on the stove to use and what temperature setting (I strategically asked for scrambled eggs, thinking forward to cooking temperature and safety). My only rule was "no fighting." (If they start arguing they lose track of details like where the knife is...) The only anxious moment came when I heard plates: clatter, clatter, clatter, clatter, clatter... I stayed in bed. It was hard. But it was their big treat for me, and I didn't want to spoil it by intervening in the end-game. It turned out my son was getting the plate he had painted himself, and that was on the bottom of the stack of 16 or so dinner plates... He moved each one, one at a time. No accidents. The scrambled eggs and toast were great, if just a tad cool—all that plate moving had it's cost, but the hand-painted plate was the perfect way to present a kid-cooked breakfast! Sometimes, we just have to stay out of the way. Lead, follow, or get out of the way. Contribute your best effort, and let go where you are not needed. 5/15/06 Lead, Follow or Get Out of the Way For the architect, getting out of the way on decisions that are not architecturally significant means you free others up to be creative, move quickly, and learn. And you keep your architecture minimalist, so that the architecture itself doesn't get in the way! For decisions that someone else is leading on, demonstrate good following. That doesn't mean you can never play devil's advocate, but watch your behavior and don't do it all the time! It doesn't mean you have to "go along' with everything, switch off, damp the passion, mute your qualms, ... either. There is a time for divergent, out-of-the-box, creative thinking where you explore opportunities, but project leaders need to start to drive toward convergence at some point, or the project will never be done. This happens early in a project when the scope is explored. And it happens all along the way, in smaller, more focused areas as challenges crop up, options are sought and weighed, and choices are made. Good following means maintaining goodwill, contributing your best effort as appropriate to the phase: is this exploration or is this time for convergence to decisions and execution on decisions? If you don't agree with the direction and decisions, you have to do some soul searching. Is it just you, just what you care about, or are things really on the wrong course for the business? If you put your own personal interests aside, and truly believe bad decisions are being made for the business, then you have to figure out what to do about it. But you first have to consider this: if you were looking at this issue from the perspective of the bigger picture, would you come to the same conclusions? And if things are not done the way you think they should be, what is the likely result? Will it be good enough? Often good enough but done in time is a whole lot better than perfect but takes too long, and endless arguing may get you closer to your idea of a perfect solution, if you win the argument, but how much does it slow things down to be tied up arguing? Often, arguing is really important to seeing a better way to do things, that no-one was considering. But there comes a time to stop arguing and move forward. Has your leader decided that time has come? Or is your leader too caught up in the arguing and someone else needs to call time and remind the leader? Social factors, yes. Getting software built is a social process, because it involves collaboration on a collective work product. Goodwill greases the skids of collaborative work. 5/15/06 HelpMatch "Competitor" Red Cross/Red Crescent If we place ourselves in the future when HelpMatch exists, then HelpMatch "competes" with others who provide assistance to those in need. It competes for supply of donations. Hence it competes with local non-profits like Goodwill and Salvation Army, or even garage sales for the local non-profit schools. It competes for cash donations (so that cash donations could cover shipping costs, or setting up assistance centers like internet kiosks in disaster areas). It competes for attention, and attention is needed to drum up cash and other donations. So it competes with Red Cross and Habitat for Humanity and others. We have to think about this. We do not want to take away from Red Cross and other institutions playing a vital role following disasters. And we want to provide an avenue for people to help people in those weeks and months following disasters where they need to rebuild their lives. So we need to be clear about the value propositions of "competitors" and the value proposition of HelpMatch. The value proposition of HelpMatch must be clear, and compellingly different from thos | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||