A Trace in the Sand
by Ruth Malan
6/30/11 This Trace
This journal contains notes I take as I explore what it takes to be a great software, systems and enterprise architect. For a sense of the span, or to dive into the journal by topic, there's a topic map that so far covers roughly half of the journal. Within subject areas, posts are chronological -- you can think of it as a map of a mind in motion. Also, they are journal posts, so they may be just a brush with the topic, a fragment of discourse, or a more extensive treatment. What distinguishes this journal, perhaps, is that it makes no presumption of boxing up some fixed authority on a matter; rather it is never done, always evolving, and part of the evolution is very much in the asynchronous conversations it draws in by reference to others within and beyond our field.
That said... For now, I've kind of fired myself from journaling. :-) We'll see if that'll mean I get some of the other things at the top of my list done.
But, since I manage my journal, I can hire myself back. I give myself, oh, 4 days. We'll have visitors over the holidays, so that'll be easier to accomplish. ;-)
7/5/11: On review, I conclude that this peculiar Trace is a vanity that really ought to be curbed. Moreover, wanting reassurance that the vanity produces its share of compensatory value only places more attention on the vanity and is a distraction from what value there might be.
So, we'll try to stray only in the vicinity of the system design concerns
that are at the center of my attention at present.
Charles Krueger's Eliciting Abstractions from a Software Product Line paper makes various points that are useful for iterative and evolutionary approaches within the Agile rubric, in addition to the focal audience for the paper (that being primarily architects of product lines).
"Salion uses the GEARS software mass customization infrastructure from BigLever Software.[2,3,4] GEARS provides explicit constructs that encapsulate and localize source file variations in a software product line. The lead architect at Salion routinely scans through the variation points in search of existing or emerging abstractions. Without the explicit and localized encapsulation of the variation points, it would be considerably more difficult to know where to look and what to compare in the search for abstractions."
That is, the architect, focusing on the code that is permitted to vary across different products within the product line or family, looks for emerging opportunities to (refactor and) craft abstractions that will better serve the ongoing evolution of the product line.
A user story/feature driven approach to work sequencing can influence the code structure, with duplication and fudgy boundaries which the architect needs to seek out and resolve earlier rather than later.
The need to find resilient abstractions that maintain sufficient identity (cohesion) and clear boundaries (well-defined interaction points or interfaces and low coupling between architectural elements) is all the greater for product lines where the value of the abstractions goes up to the extent that they enable reuse at lower cost (in terms of effort to fit to the shift in context, with related demands, that a new product introduces into the product line or family). But even in single system contexts, the need to craft resilient abstractions is high, enabling various development-facing properties such as understandability, maintain- or evolvability, testability, and also factors in operations-facing properties such as scalability and reliability.
Talking of software abstractions in such terms, we might notice that we are associating (but not identifying) crisp abstractions with modularity, in the sense that, for example, Baldwin and Clark use the term. Which is to say that crisp abstractions provide natural boundaries for modular elements. We've moved quite some distance from abstracting away from bits and bytes in programming languages to talking about abstractions when we mean code structures and design elements. But let's explore software abstractions further, focusing on interpretations that are helpful in the context of architecture.
The first two pages of Introduction to Software Abstractions: Logic Language and Analysis by Daniel Jackson (2006) is salient to the discussion here. This, from the opening:
"Software is built on abstractions. Pick the right ones, and programming will flow naturally from design; modules will have small and simple interfaces; and new functionality will more likely fit in without extensive reorganization. Pick the wrong ones, and programming will be a series of nasty surprises: interfaces will become baroque and clumsy as they are forced to accommodate unanticipated interactions, and even the simplest of changes will be hard to make. No amount of refactoring, bar starting again from scratch, can rescue a system built on flawed concepts."
"An abstraction is not a module, or an interface, class, or method; it is a structure, pure and simple—an idea reduced to its essential form."
Daniel Jackson goes on to say:
"But code is a poor medium for exploring abstractions. The demands of executability add a web of complexity, so that even a simple abstraction becomes mired in a bog of irrelevant details. As a notation for expressing abstractions, code is clumsy and verbose."
It is useful to explore and communicate at the level of abstractions, to design and evolve useful robust abstractions.
Here we have Coplien and Bornvig in Lean Architecture:
I wouldn't discount abstractions... though surely in investigating abstractions we aren't so much concerned with the concrete manifestation as the essence or idea -- a simplification of its full reality (or intended reality). [Indeed, the etymology of abstract is from L. abstractus "drawn away," pp. of abstrahere "to drag away; detach divert," carrying the early meaning "separated from material objects".]
Anyway, I prefer the tack suggested here (and referenced, for example, here):
"But the brain does much
more than just recollect
-- ☼Carl Sagan 'A Glorious Dawn' ft Stephen Hawking (Cosmos Remixed)♫ by John Boswell
That is subtly richer than the following:
"An "abstraction" (noun) is a
concept that acts as super-categorical noun for all subordinate
concepts, and connects any related concepts as a group, field, or
As we explore abstractions, generalizing and classifying may be integral to our process. For example, as Krueger describes, code analysis tools were used to look across the various products in their product line for opportunities to create shared abstractions, finding code elements that have diverged through cut-paste-modify. In designing architectures, we're looking across our experience, we're looking to experience crafted into reusable design nuggets as documented patterns, and to analogies we can use to leverage design ideas into our space, so the process of positing, exploring, and evolving abstractions in our designs has a dimension of analysis (often at the blink speed of intuition, and then more carefully resolved and validated through analysis), but also of synthesis.
This, from an earlier entry in my journal (2/21/11), prompted by a pointer to the artist Constantin Brâncuşi:
"Though just an anatomical study, it foreshadowed the sculptor's later efforts to reveal essence rather than merely copy outward appearance." -- wikipedia
"There are idiots who define my work as abstract; yet what they call abstract is what is most realistic. What is real is not the appearance, but the idea, the essence of things." -- Constantin Brâncuşi, wikipedia
I thought that was interesting. Software architecture is like that, isn't it? About finding and expressing the essential structure and nature of the thing. So we struggle with how to talk about it -- abstraction, compression, ...? To me, in advance (of the built system) we could say it is abstract (as in the mechanism of neo-modern art) though by reference or allusion (metaphor/analogy/symbol/imagery, ..., patterns) we draw in potentially huge influence and experience so compression plays a very important role. Once built (at all, and as the system evolves), compression (the very real, actualized meaning of whole chunks of the system gets compressed into the elements and mechanisms we represent) and abstraction (selectively eliding and occluding detail to reveal the essential) figure. In any event (expressing design intent or reflecting design as built), abstractions are central. I suppose, if I must, I could dance on the head of a pin and say these abstractions (entities in essential form) are compressions (they draw into compact form much meaning).
The neat thing is that whole fields of representation (including aesthetics and semiotics) swoop into relevance. :-)
Brâncuşi was, I gather, objecting to bundling his work into a class of art for which there is no evident relationship between what is real and the art; rather, he protested, his work captures the essence or the idea of the thing. It cleaves away the inessential and compresses the essential into a powerful expression of the essence of the reality. In other words, rather than convince me that his work is not abstract, Brâncuşi defined abstract for me, at least in so far as it applies to his work. For Brâncuşi was very much after identifying and capturing the essential identity and form (with its implications for function) of the thing he was sculpting. (See, for example, the evolution of his Bird in Space.) In discovering software abstractions, we can go from concrete instances to a generalization, eliminating detail from the concrete to find the more general, more abstract common form. But seeking the idea, the essence of the thing, supports more degrees of freedom in realizing concrete forms that retain the essence.
I don't mean to imply that abstractions in software architecture are equivalents of Brâncuşi's art, to be treasured for the essence they capture, but not utilitarian. Our abstractions need to support the translation into a functioning reality. And more: they need to convey and provide enough integrity under considerable entropic forces that will threaten to so morph the abstraction that its resilience and coherence gets washed out, imperceptibly with little compromises at first, but mounting none the less. So we seek abstractions that cohere around and grant clear identity (including purpose or intent as well as the quality or inherent, essential nature) with essential form (boundary and shape) and function (how it accomplishes its purpose, in strategic terms).
These abstractions can be, but are not limited to being, simplifications of reality, either in the sense of creating an abstract representation by eliminating code detail (and generalizing to fit a wider variance in contexts) or in the domain object sense of finding software abstractions by creating software analogies for entities in the real world (of the "problem space" or business domain of concern) that are simplified by ignoring elements of the real world complexity not applicable to the problem being supported through software. They may be inventions or discoveries that have more to do with the solution than the problem. There is nothing in the problem of a horseless carriage that tells one, in advance, that it must have an engine, but in exploring the "locomotion capability" options, that abstraction presents itself as an idea worth pursuing. And indeed, it is an idea that is still undergoing evolution, sometimes with quite disruptive innovation reshaping dominant design in the auto industry.
Early on, we explore and discover ideas that will give our system shape and function. As our system starts to gain mass, we want those abstractions to take clear shape, and to work together in well articulated ways so they preserve identity while contributing to the system's capabilities. This exploration and discovery, early and then ongoing, is so important to do, and to keep track of, to communicate and keep vividly in mind, that we separate out this view and call it Conceptual Architecture. Some of the abstractions are the (conceptual) components, some are mechanisms (sets of components that function in concert to produce some capability), etc.
So, anyway, you can see where my mind is at -- toying with just how to characterize this concept that is at the very heart of architecting. How do you think about software abstractions as they relate to software architecture, and what references have you found useful?
I know, we could say elements and relationships and mechanisms are at the heart of architecting. But what are these elements and mechanisms? How do we invent or discover them? Do we start (among other things) with concepts or abstractions, and is there a difference? I know, I've talked with some assumed authority (bold, it is true, but such are the demands of my role) about architecture for close to two decades. I hate to get pedantically precise about something as useful as the notion of abstraction, but it is fun and, I expect, worthwhile to enrich our concept of abstraction and its role in software architecture.
7/15/11: Where does the technology stack figure, if I'm suggesting the essential structure and key mechanisms are at the heart of architecting? Much of what we would otherwise have to define and implement as essential mechanisms are implemented in the technology stack, so we can work at a higher level of abstraction. Technology stack choices have much to do with thinking through what our requirements imply in terms of "technical requirements" (transposing requirements that arise in our solution into capabilities and properties we need in our middleware, etc.) so that we can evaluate alternatives in terms of the cohorts of mechanisms so supplied.
I am also reminded of Grady Booch's reference to Scott McCloud's "amplification by simplification" (Understanding Comics, 1993) in his on architecture column (possibly the Draw Me a Picture one) and slidesets (SoftVis2010 Endnote and Models 09 Keynote). Of course, when I talk of abstractions being at the heart of architecting I am also making reference to Grady's SCARS. Well, actually, he calls them fundamentals. [In a workshop an architect asked if we have a set of fundamental principles or heuristics with a mnemonic like CRUD, and I thought SCARS was suitable since that is what we get, you know, from experience.] I playfully suggested that Grady ought to add one -- call it compromise* -- in recognition of one of Eberhardt Rechtin's characterization in Systems Architecting: Creating & Building Complex Systems:
"The essence of architecting is structuring. Structuring can mean bringing form to function, bring order out of apparent chaos, or converting the partially formed ideas of a client into a workable conceptual model. The key techniques are balancing the needs, fitting the interfaces, and compromising among the extremes."
Anyway, returning to "amplification by simplification," it is well to ask "what are we amplifying?" Early on, we're amplifying our ability to conceive an approach. And as we build out the system, we're amplifying our understanding of, and ability to sustain the crispness of, the organizing abstractions and how they function to create system capabilities. In short, we're amplifying cognitive traction so that we can build more complex systems and sustain system integrity despite the entropic forces that mass as more minds and hands build-build-build the code of this system, and all its derivatives. Which is what this post has been about. Rats, I could have simply pointed to "amplification by simplification" -- which would have demonstrated amplification through simplification. Right there. Three words. Done. Ah but, then, would you have put Brancusi's studio on your Paris itinerary, or made a note to see the collection at the Philadelphia Museum of Art? At least I serve some purpose. Not what you expect, but better? Surely better, or you wouldn't read here. Oh, right. You don't. But, if you did, then I'd be able to tell you that I think one of the ways that we see the beauty in software is through its abstractions. The simple lays bare the mastery in and elegance of the structure and its mechanisms, the intricate tracing of its functioning is raised, as in relief, to the level of patterned form, and so forth. This works forth and back, for as we pare back to the essential form we see opportunities to simplify, to make more coherent.
7/17/11: Lots and lots of great discussion and pointers to other work around abstractions (and Rube Goldberg Machines) here: The Last Language?, Dominic Fox, 2011-07-14
7/18/11: When it comes to building architecture, we can stand back and look at the building and see how the design is realized in form and structure and we can move about in the building and experience how the architecture shapes our experience and limits and enables what we do within it. We know that what we see and experience has to do with architecture, because we have clear experience with good and bad architecture. Both in the now, and in how the architecture stays relevant over time, though with idiosyncrasies of its period that we perhaps indulge with a sense of romantic tolerance though more likely (for buildings that survive) view with awed respect and gasps at the mastery, elegance and beauty that man achieved with feat of mind and hand.
For software, it is harder to "stand back and view" the structure when what we have before us is scads and scads of lines of code -- even millions of LOC. Sure, to appreciate the implementation of an algorithm, we need to look at code. And when we are intimate with the codebase, we move naturally within it. But to a newcomer it is like spelunking -- often without even a flashlight. To see the structure, we need to see the abstractions. So the abstractions play multiple roles -- they help us create systems with better fit to purpose and to contexts, with greater integrity of design and structure. And they also help us stand back and see the beauty in the software itself. In its structures and in the dynamic behaviors it makes possible. Without these abstractions (at various levels), we're so deep in the code we're too close to it to see the larger structures nor even any intricate patterning among the smaller grained structures, and we need ways to move out, like the Powers of 10 movie, to see more the span of the system so we can take in aspects of its larger architectural form.
7/19/11: * A note on compromise. I playfully suggested that Mr. Booch add "compromise" to his set of architecting fundamentals. Playful, because it might be a compromise to add compromise to the set. Of course, I am very much in the camp that insists that the architect (and others on the core design team, if ultimate design authority is split, for example, among product owner and architect) decide where to excel, where compromise will not be brooked, and where negotiation, tradeoff and -- in the "call it what it is" sense -- compromise will be the order of the day so we get the great system we have in mind realized in the world. Politics being what it is. Constrained resources being what they are. And simplicity being, among other things, a matter of choosing what is essential and what we can slough off or play down. So, we could in our compromise principle, say "compromise and not." Then we have our C. Although... I like the C in Concerns too. Hm.
Abstraction, This Developer's Life podcast
Would the EA identity crisis go away if people just agreed to read the papers we did for Cutter? Wink. There's mischief (and a note of irony at the compromise of self-promoting if only in this small pool of hopefully-friends of my work) in that wink, but... it seems that this is a good time to read these three reports, in this order:
I wrote the last of these to articulate a (hopefully) compelling reason d'ętre for EA. That last paper does so much, from expressing how agility demands differ not just across organizations, but within an organization, depending on the point in the product and ecosystem lifecycle. It articulates how we can reconceive strategy so that it is more organic, coursing intentionality where it needs to flow in the organizational organism and creating concert among organically forming and reforming collaboration cells, while empowering responsive adaptations to local contexts. It refers to the second of those reports which deals with how to relate business strategy to enterprise architecture. It considers IT in the current context, exploring IT's opportunity to create competitive distinction through business intelligence and relationship platforms that make an organization more than a sum of its parts, even parts that are "hooked up" together. And it explores the role of architects, and especially enterprise architects, in organizations that are striving to reap the benefits of more organic responsiveness rather than traditional mechanistic, bureaucratic gate-kept flows of information and control.
Maybe you could, in the light of expositions of the identity crisis (Tom Graves provides a good one, and points to others), read the 3 reports I listed and let me know if they help clarify the enterprise architect role and its distinguishing identity, including mission and form. For one thing, different organizations are at different points of maturity (that's in the second of those reports). Some organizations are still more hierarchical, and are simply looking to bring the IT landscape across the organization into a more orderly portfolio to reduce costs and prepare for more integration and technology asset leverage. If that is the situation, then, for example, looking at what artifacts enterprise architects at other organizations produce, with a view to leveraging the form of what others are doing, may be just fine -- in the context of similar culture, structure and strategy, that is. Otherwise, if what an EA team is doing at another company is more the determinant of what you're doing than your business's strategy and strategic initiatives, that's going to perpetuate the identity crisis because what is done in the name of EA shapes perception.
Part of the role of enterprise architects in a time of flux is to create the role -- to shape expectations for the role and to deliver value that shores up expectations and relationships, and advances them. Those executive reports were written in good part to assist enterprise architects in formulating the role especially in organizations that are moving away from the simpler structural style of organizational silos and "linear dominance hierarchies" to more network collaborative styles, and providing another point of view to share with teams and management.
Still not interested in Part II of The Art of Change (on leadership)? You do know that I'm waiting for someone other than the editor and my kind friend Daniel to ask for it, don't you? Well, you could instead read Ken Robinson. I thought this was interesting:
"... this year the CEOs said they had three overall priorities. The first priority was running organizations that can respond to complexity because the world is getting more complex every day. Second was how to run organizations that are adaptable and resilient to these changes. But the top priority was how to promote creativity in organizations. The answer to these three priorities of complexity is to think differently about people and to reposition the role of leadership."
-- Ken Robinson On The Principles Of Creative Leadership, Rae Ann FeraTue, Jul 5, 2011
7/21/11: Here is a useful resource (of course you'll remember not to become slave to such a resource, but rather do what is strategically important in your context):
In the UK, women accounted for just 18 per cent of technology professionals in 2010, down from 22 per cent in 2001. ...
In the United Kingdom, just 9 percent of computer science students are female, compared to 14 percent 10 years ago. In the United States, just 18 percent of undergraduate IT degrees awarded in 2009 went to women, down from 37 percent in 1985.
-- 'Geeky' world of IT loses its appeal as a career choice, Maija Palmer, Jul 06, 2011
7/6/11 Mopping Up
The cartoon below, shown in Dean Ornish's TED talk, reminded me of the flood of this journal... and my failure to turn off the faucet. But it is a wonderful cartoon to have in the back pocket, for in such a variety of ways we ignore the problem because we're so busy working on the symptoms.
Image: from Dean Ornish's TED talk, at minute 00:48.
The next open enrollment class is in South Africa in August. It's winter there, and a good time to visit the big game reserves like Kruger National Park (because the grass is short and you can see the animals better, and the mosquitoes aren't a problem). Well, it's a thought. :-)
There's also an open enrollment class in Orlando, Florida in October -- that's a nice time to haul the family along for a visit to Disneyworld, etc.
Of course, a boondoggle is the only reason you'd want to take our class... No, that's not it. But if South Africa is on your list of places you want to see, the workshop could give you an extra reason to make it happen.
Here's the open enrollment class schedule:
Pragmatic Software Architecture presentation is up on Prezi. Even if
presentation on InfoQ, the Prezi version has a different pacing and
allows for a self-led flow (different from SlideShare). I've found it
useful to see how Joe frames and positions software architecture and the
architect role. Joe's
resources page is also well done.
I enjoyed "eavesdropping" on the twitterscussion between Kris Meukens and Doug Newdick yesterday, stimulated by Steve Jobs provocative statement: "It isn't the consumers' job to know what they want." Various paths of response come to mind:
As for Jobs on jobs -- on what Apple gets paid to do -- here's another quote:
"It’s not about pop culture, and it’s not about fooling people, and it’s not about convincing people that they want something they don’t. We figure out what we want. And I think we’re pretty good at having the right discipline to think through whether a lot of other people are going to want it, too. That’s what we get paid to do." -- Steve Jobs quoted in On Apple's connection with the consumer, CNN Money, 2008
That said, in our Getting Past ‘But’ EA executive report, I used the colorful Henry Ford quote ("If I'd asked customers what they wanted, they would have said a faster horses.") as well as this wonderful piece:
'A major auto company recently presented its “innovation road map” for the next ten years to a group of journalists and car enthusiasts. As the presentation progressed, it became increasingly clear that some members of the audience were restless. Finally, one listener stood up and said, “Many of us have already built and installed every single one of the innovations you say you are planning to develop in the next ten years. Wake up and smell the coffee! Come out to the parking lot and take a look at what we have developed and installed in our cars!”'
Eric von Hippel, Breakthrough Idea Number 6, Harvard Business Review, February 2007
Good ideas come from many places. Consumers included. Start-ups (very often forming around an idea for something the founders want in a product and see a gap in the market as an opportunity). Competitors (who may not fully see the value and bail, or fail to do what it takes to make the idea great). Vendors (who don't imagine themselves in a neighboring business). Researchers. The design team. Good product ideas are a brilliant synthesis of many good ideas, which come from many sources, including the market directly and indirectly, as well as what is happening in the technology space, in our market and beyond. Indeed, there are so many good ideas that it's a matter of saying "no" -- a lot! And also a matter of where we say "yes!" And going with it. Running that ball all the way. Without being distracted by all the no's that would coming nipping at our heels if we didn't act with considerable confident determination. I say this last, despite what Steve Jobs says:
“People think focus means saying yes to the thing you’ve got to focus on. But that’s not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I’m actually as proud of the things we haven’t done as the things I have done. Innovation is saying ‘no’ to 1,000 things.” -- Steve Jobs quoted in Steve Jobs: Get Rid of the Crappy Stuff, Carmine Gallo, Forbes, May 16, 2011
Of those who fail, there are those who do so in good part because they failed to say no and took on too much or made things too complex. But there are also those who failed to get sheer will, absolute conviction and determination, aligned around the thing being done. Of course, you might ask who am I to counter Mr. Jobs, but we can all see that he has and can assemble that stick-to-it-ness I'm talking about, and he clearly demonstrates that leadership -- evangelizing and aligning around an exuberant and determined "yes" -- is important too. This isn't just about tradeoffs but the passion and determination to achieve excellence in what we will do, and to slough off what would compete for attention and resources (ours, and later our customers) -- to prune that opportunity tree with care and determination.
The point is that Steve Jobs and Apple gets everything (well, ok, most everything) right, for them -- what they do works! But they are in one kind of business and have a leader who makes things happen across groups. We have a lot to learn from them. But there are other companies, and teams within companies, doing great stuff too. Differently. And we can learn from them.
If this is something you're interested in, I wrote a post back in February that if nothing else has some interesting links: Point Counterpoint
If you aren't sure that this is relevant to the architect, I took a stab at making a case for that here: Architects and Innovation (and elsewhere).
And if you haven't read Getting Past ‘But’ yet, why not? It deals superbly with the subject of this debate and more. Really. Wink. [Look, if I was to write the paper today, it would come out different because I've learned a lot in the interim too. Nonetheless, it is still in advance of majority thinking and it uses a disarmingly charming story to convey lessons about concurrent and agile development.]
Anyway, Doug and Kris made good points.
About the role of innovation in design and the designer in innovation,
and the value of using words that convey respect and goodwill.
7/12/11: also interesting: Innovation is About Making Predictions, Dennis Staufer, July 11, 2011
7/17/11: This is a perfect complement to this post: Problem-finding, by Tom Kelley of IDEO -- don't overlook the video in that post (and its point about observation)! :-)
Our field sometimes seems to have forgotten that "architecture is design" means architects are designers. Not the only designers of the software system, but the lead designers responsible for system outcomes -- strategic value and system integrity that assures the sustainability of that value. Moreover, if we allow ourselves to be scripted into the corner where we design only the guts of the software "machine", we lose sight of what is so important about architecture, which is the focus on defining and designing to attain (better) system outcomes -- in that iterative problem finding-problem solving sense. Better outcomes, that is, than we'd achieve by happy accident alone. Oh, of course we don't eschew happy accident. We just know that the "shoulders and ashes" (a wonderful Boochism*) of past trials and successes are there to be built upon to make things better -- somewhat intentionally and somewhat experimentally.
"I would contend that there are few truly novel software architectures. Rather, as with every engineering discipline, new systems or modified ones are [built] upon the shoulders and ashes of previous ones."
-- Grady Booch, blog entry January 27, 2004
7/19/11: Designers at Apple, despite their "we don’t waste our time asking users" 'tude, know how to crowdsource to create a vast playground -- witness the app store. Besides, before rushing to mass-emulate Apple, we might want to take note of other approaches:
7/20/11: Well, you get to have some 'tude when you achieve this:
"But I am convinced that ... the art and science of observing behavior should have a larger place in our thinking and in our curricula than it does at present."
— Daniel Kahneman, Insight: A Conversation with Gary Klein, 7/2/11
And a bigger place in product/systems development!
7/21/11: More illustration of diverse approaches working:
'So he [Bezos] encourages people at Amazon to ask "why not?" when considering whether to launch something new. "It's very fun to have a culture where people are willing to take these leaps. It's the opposite of the ‘institutional no.' It's the institutional yes. People at Amazon say, ‘We're going to figure out how to do this.'"
Jobs loves to ask "what if" and "why" questions and so do Apple employees. Lafley has devoted hundreds of hours to observing customers, just as anthropologists observe tribes, and has put specific processes in place for observing customers at P&G. Benioff is a great networker, and at Salesforce.com he introduced Chatter and other networking processes to help employees network both inside and outside the company for unusual ideas. As an exceptional experimenter himself, Bezos has tried to institutionalize experimentation processes at Amazon that allow employees to go down blind alleys in pursuit of new products or processes. By creating organizational processes that mirror their individual discovery behaviors, these leaders have built their personal innovator's DNA into their organizations.'
The DNA--People, Processes, And Philosophies--Of Innovative Companies,
Jeff Dyer, Hal Gregersen, and Clayton Christensen, FastCompany. July 21,
Vint Cerf is also pointing to the importance of yes, but coming at it from a different angle:
I was thinking about this in my days at DARPA many years ago when we were assigned a budget that we were able to spend on the projects that we had been given responsibility for. And having that budget was a way of being able to say yes because if it was going to cost money, you had a checkbook, and we were able to say yes as long as it fit within the budget. And I can't imagine a more important capability to facilitate innovation. If all you have is the ability to say no, then it's very hard to innovate."
Vint Cerf: Essential Ingredients for Innovating in Government, Vint Cerf , July 14, 2011
I also write at:
- Bredemeyer Resources for Architects
Architects and Architecture
- Todd Hoff (highly recommended)
- Anna Liu
- JD Meier
Architect Professional Organizations
Agile and Lean
Agile and Testing
Other Software Thought Leaders
- CapGeminini's CTOblog
CTOs and CIOs
- Werner Vogels (Amazon)
- Jonathan Schwartz (Sun)
CEOs (Web 2.0)
- Don MacAskill (SmugMug)
- Wired's monkey_bites
Social Networking/Web 2.0+ Watch
- Dan Roam
- David Sibbet (The Grove)
Strategy, BI and Competitive Intelligence
- Freakonomics blog
Um... and these
- CNN Money Business of Green videos
Here's a snapshot of social media participation:
This video (from Socialnomics and based on Socialnomics 3 by Eric Qualman) presents some incredible numbers:
The UN is saying we've made a start -- UN reports strong surge in investments in green energy -- but we need a green revolution: New industrial revolution needed to avert ‘planetary catastrophe’ – UN report (via Tom Graves tweet). [Of course, we have a long way to go before humanity learns some essential lessons in systems dynamics... this, for example, serves as a reminder: invasive solar energy.]
And as revolutions go, how about this for potential to disrupt:
We can't let the economy slow down -- there's just too much to do!
7/9/11 Complexity and Wicked Problems
Wicked problems that interact with or come under the compass of the system
strategy are architecturally significant. So these two posts are useful:
We informally call Dana's "architect across the system" diagram the "umbrella diagram." We also depict the forces and intentions that the architect is working with to synthesize system outcomes and decisions as the arrows of strategy (formal) and all the informal and politically charged personal agendas and local goals of various stakeholders, in addition to constraints. So I added archman with a handle to the umbrella, and leveraged my "architect raises the productivity ceiling of the team" notion, and the combination forms the sketch (right).
Jest aside though, there's a lot of dancing in the intersection going on, as the Agile community realizes architecture is needed and the waterfall/BDUF community realizes iterations are a good thing. As the two come together and find common ground -- less DUF, but not NoDUF -- there is still a desire to keep to the egalitarianism that grew around Agile. There is a danger, though, in promoting architecture as just another hat everyone wears, and shying away from having architects and "chief" architects. We need empowerment, sure. We value emergence. And intentionality. Leaders set context, ensure that a vision is shaped and matured (with all the evolution and shape-shifting that takes), foster passion and culture through selecting, creating and telling powerful stories that convey values. And so forth. The demands on the leader are more, not less, in a more empowered, more egalitarian culture -- one where:
The architect doesn't talk, he acts.
When this is done,
the team says, "Amazing:
we did it, all by ourselves!" (17)
-- Phlippe Kruchten's Tao of the Software Architect
Instead of dispensing with chief architects, we might want to look at tribal cultures rather than bureaucracies. This, from Yvon Chouinard's Let My People Go Surfing: The Education of a Reluctant Businessman, struck me profoundly when I read it several years ago:
"A chief in an American Indian tribe was not elected because he was the richest or had a strong political machine; he was chosen chief because of his oratory skills, which were invaluable for building consensus within the tribe."
Creating dialog across groups, hearing the various needs and agendas, negotiating, imagining what lies behind and beyond stated desires and frustrations and day-to-day tribulation and so on is fraught with ambiguity and uncertainty. Working with this "big buzzing confusion" and turning it into clear outcomes and landscape-shaping decisions is just not everyone's bailiwick. But even if it were, when consensus stalls (and it does), we need a leader who is entrusted with making the tough call so that the team can move forward.
7/11: I fault myself when I lapse into saying what is easy -- saying the architect is "balancing" across the intentions and agendas and needs and so forth of various stakeholders (and groups). For it is not so much a process of finding a balance, as deciding (or leading/helping the core design team decide) what to make great and what to sacrifice in order to do that. It is that process of finding the exciting "yes" that causes hard "no"s to have to be said. And it is this, this fact of having to negotiate and persuade and influence as much to deal with the "no"s as to get steady will behind the "yes" that calls for strong leadership. Strong, but facilitative and empowering leadership. We want to create the space for lots of innovation and contribution, by cleaving off the space of what we will not do. You know, that pruning the opportunity tree so that the tree is more healthy, and the fruit it bears gets good light. Organizations are fraught with power struggles and agenda pushing and it makes this whole arena a matter of influence and creating alignment. This arena of listening to the various positions and needs and integrating ideas and discarding others and finding/exploring/designing the compelling thing we will do -- all this is not what every developer wants to be involved in, nor are they necessarily experienced or adept enough at the politics. One might interject "who is?" to which I'd have to respond "well, at least in this role one has to have the passion to do it, and the ability to develop credibility, trust and goodwill enough among all the stakeholder groups." And again, this takes experience and a certain leaning, an interest in the concerns of a broader set of stakeholders, and so on. Hence the joke, but also the serious point, about the architect holding the umbrella fending off the political crud so the team can get great work done.
My umbrella diagram reminded me of OK Go's This Too Shall Pass video (you know, the one with amazing Rube Goldberg machine)... Which was on my mind, because it occurred to me, in a weak moment, it is a decent visualization of the guts of an agglomerated software system in action. Ok, a parody, but a fun one. So, if you didn't see the Adam Sadowsky engineers a viral music video from TED in 2010, it does draw out some interesting lessons about the process of creating a large-scale Rube Goldberg machine. The inversion of small and large is interesting (we do "large rocks" first, they do small things first), but both involve the same kind of sorting out what would make the whole thing fail and investigating and resolving (as best we can) that early, and figuring out sequencing to lower (market and technical) risk and cost.
That Rube Goldberg machine is amazing -- something on that scale, that
combination of precision and sense of fun, amazes us. So when I say it is a
visualization of the guts of an agglomerated software system in action, I'm also
saying that with a reminder reverberating in my own head, of the sense of awe I
feel looking at "legacy" code and envisioning the running system. I've been
toying with this tension between the precise, regular
beauty of a highly patterned structural form (like some of Escher's work,
and often forms in Nature) and the beauty we find in the "imperfect," the
irregular and the, in some sense,
not perfectly, consistently, shapen. In the product of mind, the craft of
man's hand, but also in forests and rugged
deserts and mountains or just some lichen growing on the deck. In some of
those cases the grandeur awes us. In others, it takes a discerning eye and a
different orientation of perspective, to see the beauty. We can use software to
create something beautiful, but to bring the beauty in the code to someone
outside its "tribe," we have to be able to articulate and render what we
appreciate and see.
"You are not an explorer because you can draw lines on a map. You are an explorer because you go somewhere new." -- Eric Ries
But if you draw the lines you find, creating a map of where you explore, you enable others to go there too. And... to decide if they want to.
I like how the diagram (shown larger at the top of this page) fell out. The simplicity of a subway-styled map, with the key elements and interactions. In the inner loop, we have the core of architects architecting architecture and in the bigger loops we have the interactions of the core with the context (organization, motivation and evolution). The 6 interrogatives, my serving men and yours, all interconnected in loops that interact. Oh goodness, I know it's no biggie. Just a neat little diagram that compresses an entire field... but hey, those interrogatives have been around like forever, and I have used that as the organizing model for the Bredemeyer Resources for Software, System and Enterprise Architects site since its inception (a dozen years ago). I just like the visualization. It makes connections.
It is one of those things that is hidden in plain sight until someone makes it visible. I like the arc, arch and architect in the inner loop (architects architecting architecture), and the tion in the context (organization, motivation, evolution). It gives it a sense of motion. The brain likes patterns! When one see patterns in the structure, one feels like one found a right structure.
But, hold the applause. No really.
Say what? ... too much perspective...
Ah, perspective. It's worth 80 IQ points, remember? Here's to perspective then -- from various angles.
7/12: Human fallibility (such as finding validation in spurious patterns, not to mention vanity doing a celebratory jig on the artifice so constructed) is, of course, why our process is not just iterative and incremental but fault finding, risk resolving and correction seeking. But, sigh, feedback on this journal is restricted, with scant few exceptions, to silence, so... I have to read the silence... which I must (could silence be affirmative? clearly not) interpret as... echoing my internal critics. Ouch.
Some time ago, Dana encouraged me thusly:
I love that you do it at all, I admire the bravery, and the discipline. I especially like it when you surprise or challenge me; then I know it's you out there and not me, and since the whole body of the work is such that it convinces me of the value of the whole body of the work, including the hidden parts, the I-don't-get-it-at-first parts, and the future parts.
More recently he remarked that my journal can be work to read. But of course! I make the assumption that you are smart and quick -- more so than me. So, if you think this map post is just a flippant abuse of your time, you're not reading carefully. Let's take the context I referred to, and in particular evolution. What concerns would we emerge to discover if we stopped off at that "evolution" station on our map? These are largely lifecycle concerns. What is the relationship of architecting to the development process? What are the architectural considerations at different points in the lifecycle of the system and the product line/family or solution and how do these concerns shift and evolve? What are the relationships to the strategic cycle and to the maturity of the ecosystem and the product or business services? and so forth. Ahhhhh. The I-don't-get-it-at-first-parts... Wink.
7/13: Um if you don't get recursive irony... let me remind you: if I seem to do a celebratory jig, I am teasing myself. The illusion of the jig is not the dance of vanity, but the shadow play of humility puppeting vanity... in a world where the reverse is more common, perhaps. Get it? The recursion? It is impossible to escape vanity. The humble person is vain (or self-righteous) about their humility. Etc. Which is why I find Randall Monroe's early commentary on irony and self-reference to be a stroke of pure genius. You don't follow that? Don't worry. It's not just you. But hey, there are people -- even people I very much admire -- who don't get this Buttercup Festival. Not even when I explain (how I see and thrill to) it!
But, within the shadow-play are wisps of truthiness, some of which might even be truth. All of which... I kid. Too much.
Ok, I'll get serious. I've likened process to scaffolding, and the more risky or complex and ambitious the heights we wish to attain, the more we need scaffolding. Dana has been replacing a stretch of siding on a piece of exterior wall between roof slopes on a section of the attic -- high, and tricky to work, due to standing on steep slopes, etc. He's been using his climbing gear (an agile approach to scaffolding), so he works with harness and ropes (to manage/reduce risk). [Still, it makes me realize how important he is to me and to the world when he's up there in so precarious a spot!] Anyway, he keeps relearning architectural lessons, of course -- like the importance of infrastructure, tooling and process, of sequence and thinking big picture just enough first and then intermittently, especially when, not having done it before, there's a whole lot of learning that has to happen as you go. Our tendency is so much to get started and do the first apparent thing, and then the next obvious thing, finding out too late what would have made the next 3 things a lot easier!
And in all these projects, of code and wood craftsmanship, we're continually reminded that you have to architect across the interfaces -- across the joints or surfaces where elements interact because without these interactions we don't have a system, or we have a system where all the great work in the elements is overridden by a sense of kludginess because their interaction is dysfunctional. Even when we're working with something as forgiving and malleable as code, it is harder to retrofit elegant interactions upon elements and surfaces that don't fit naturally. We can make it work, but it just doesn't hold integrity -- and less, the more we "make it work." Integrity, whether structural or ethical, is undone by inconsistency and misalignment. I was reminded of that when a Back to Nature box of crackers had a shell of plastic with a wide lip so designed that the box contained a third less crackers than suggested by the box size. Moreover, the amount of plastic used was a clear sustainability compromise -- where's the "Back to Nature" in so much plastic? Yummy crackers though. Which throws the issue of consistency and alignment with principles onto me! Misalignment bleeds across even loosely coupled interfaces. Perhaps not all the time, but in ways that can take us by surprise.
Which brings me back to process. It's all quite loopy. So we need reference structures, so we don't get lost. Ah yes, the map! And our architecture decision model.
But you know what's missing? Yep, you got it. A manifesto!
"You can't teach an old dogma new tricks." -- Dorothy Parker
So it must be that we need a new one. And if one, then many!
I kid. I kid. Hold the bullets! Dana just read that quote to me this morning, and I figured I had to find a way to use it. ;-) Ok, back to my box(es). The ones you put me in, and the ones I'm trying to articulate. Although, I'll say it again, I think Mr. Booch gave up something important when he let Rumbaugh get away with boxes. Clouds are so much more useful to start with.
As for my map, I think there's a chasm between simple and simplistic. I think my map is simple. I think it is a simple and worthwhile guide to a complex space -- complex because the interactions are not just bound in now, but shift as our understanding shifts and reshapes.
I really enjoyed and got a lot out of Martin Fowler's The LMAX Architecture post today. I can see it becoming a "go to" post, serving as a great illustration for many points I make.
7/12/11 AI Gets Creative
Interesting things happening in every field as technology transforms (or "invades") even space we might not think under threat (of creative waves of destruction) -- even literature.
So, could IBM pit Watson against Google and gain prominence in the search space? Google does a good job of pointing to places to find answers, while Watson gives answers (and possibly an answer tree) from vetted sources. I like the former, because I like to be in the driver's seat when it comes to developing my own mind, but I can see a lot of use for getting straight to the answers.
In terms of things changing, here's an interesting set of points: Grady Booch: The issues vexing developers, Ben Woods, ZDNet UK, 9 July, 2011 (via a Bruce Douglass retweet).
[I liked Jon Udell's description of
using WolframAlpha to explore a sustainability question. Um... Between
Watson and WolframAlpha and Facebook (wha? you know, for socializing), what do
we need schools for? If all they do is dull curiosity and create social
"The Special Interest Group on Software Engineering (SIGSOFT) of the Association for Computing Machinery (ACM) have awarded Mary Shaw and Dave Garlan the Outstanding Research Award 2011. Both computer scientists have pioneered the work on Software Architecture at the Software Engineering Institute of the Carnegie Mellon University in Pittsburgh." -- Michael Stahl, 7/1/11
7/13/11 Goodness Gracious!
Oh dear! Upon reflection: my journal is a rather daunting mass of words, isn't it? Ok. I'll try harder to stem the flood. Bottle up the playfulness. Be more useful.
I'll try. No promises though!
I'll be watching for Kris Meukens response to Tom Graves' post.
When I boil down the thoughts and questions that arise for me, I'm left with this one: Should we give in to the common placement of EA within IT (with related restrictions in focus and responsibility), and say BA is the new EA (but with appropriate placement not under IT) and let EA collapse into being the EwITA it so often is in practice?
Um. That's a Trojan Horse of a question, now isn't it? ;-)
7/18/11: See also: Six Reasons Why EA Should NOT be Assigned to the IT Department and our role of the enterprise architect paper from 2004...
7/20/11: I think this discussion
by Todd Biske provides useful perspective.
If I ever do something this good, you'll say something nice, right?
8/5/11: But not until then, of course.
Peter Bakker (@pbmobi) pointed his twitter-constellation to an excellent paper on subway maps. I liked the notion, but wasn't too sure how it applied, so (subconsciously) set myself the task of figuring out the utility on my own "stuff.' The result being the map that tops this page. I like using the metaphor of wayfinding in my journal, with the subway map for navigating to "neighborhoods" of concerns. So I guess, in a way, my map is a tribute to Peter Bakker, and a path he's set me on. [Still working on the data; active map to follow.]
Some time ago, in a side conversation with an architect, I stepped outside my normal reservations about self-promoting* and recommended a journal post on sketching and sketchbooks. [Except, you might want to note that Moleskines are to sketchnoters as iTouches/iPhones are to teenagers -- you wouldn't be caught with a me-too brand.] Well, anyway, sketching is something of a theme for me. Now Peter Bakker is on another quest, this time around sketches, and since his tweet stream will be vaporized in tweet-time, I'll collect a number of his pointers here:
As you will have noted from my Picture It talk (you may prefer the script) from June 2009, we also very much like Mr. Sketch markers -- you know, for drawing "big pictures" out loud. It's that "art of drawing people in" thing. Anyway, Mr. Sketch markers smell good! :-) Yes, we've been drawing pictures on walls for a long time --since long, long before this, though of course mankind has been doing so even longer. And since we're talking about power tools for enterprise, system and software architects, Koosh balls are way up there - if you're not going to take notes and doodle, then at least fiddle. When the hands are busy, the mind pays more attention.
Here are some additional links:
You might have noticed that I also collected a set of links on sketching last month. And there's the Visual Thinking and Visual Design section in my journal map. More than anyone could reasonably scan, let alone imbibe. So here's one example from that set: Architects and Analytical Thinking.
And since we're on a roll, here are some books I recommend:
Dave Gray's Visual Thinking sets on Flickr are great too! As is his Connected Company set. There's a lot of truly wonderful material in his various (and many) sets on Flickr to explore and learn from. I like to collect photos of signs too. This one struck me -- logic fail but works... for the LCD?
To draw the link from sketching (and signs) back to design (remembering that architecture is design), one of the Petroski books might appeal:
Or you might like one of my favorite (of its kind) books (it combines history and visual design in engineering):
* if you're thinking that's ironic, given that I was self-promoting right then, please notice that this journal is way, way beyond my normal reservations! Way beyond anyone's, really.
7/19/11: Hunkering: Putting Disorientation into the Design Process, Jared M. Spool, Apr 07, 2009 [so that's what it's called!] [via Peter i.e., @pbmobi]
7/21/11: I so totally want one of the iSketchPads pictured in It's Coming: A True Replacement for Paper, Christopher Mims, 07/20/2011 (See, I even gave it a cool, if obvious, name. Now the hard part's done -- Steve, get [back] to work.)
7/23/11: Keys to Drawing with Imagination: Strategies and Exercises for Gaining Confidence and Enhancing Your Creativity, Bert Dodson [via Peter i.e., @pbmobi]
and Sunni Brown's sketchnotes from STORY Seminar with Robert McKee [via Peter i.e., @pbmobi]
7/15/11 Also Interesting
And if you're interested in supporting women in tech:
Here are some of the women in software architecture and Agile in the Twitter-verse:
I reread some posts from October 2009, starting with the System Architecture post and then following Serendipity as she drew my attention to some of the other posts. So that's why I journal! That, and to keep track of good stuff like this:
Software documentation is like sex: when it is good, it is very, very good; and when it is bad, it is better than nothing. (Anonymous.)
There are two ways of constructing a software design: one way is to make it so simple that there are obviously no deﬁciencies; the other way is to make it so complicated that there are no obvious deﬁciencies. (C.A.R. Hoare)
A computer language is not just a way of getting a computer to perform operations but rather that it is a novel formal medium for expressing ideas about methodology. Thus, programs must be written for people to read, and only incidentally for machines to execute. (The Structure and Interpretation of Computer Programs, H. Abelson, G. Sussman and J. Sussman, 1985.)
If you try to make something beautiful, it is often ugly. If you try to make something useful, it is often beautiful. (Oscar Wilde)
-- GATE: General Architecture for Text Engineering, Chapter 1
Every software architecture document should begin like that! (With as much panache and insight, not necessarily all trotting the same quotes, though these are superb and worth duplicating, as I've just modeled for you.)
7/16/11 Tweet Roundup
They're coming around! So, I guess our next open enrollment Enterprise Architecture Workshop will fill nicely.
We'd better get the Visual Architecting book out while the tide is coming back in for architecture, huh? Whatever else Lean Architecture does, it leaves a lot of room on the table for what it does not do. :-)
I realize that this Trace is a perplexing investment from any point of view, but if I pick a month (say last October) and read across it, I am glad I did it. It participates in the grand experiment of the Internet -- this point we're at where we are discovering what it means to have our personal self extended into a global digital world is a tremendous state-shift for humanity, and I have been doing something here that is well, unkindly and kindly, beyond all!
Sara is wearing a Nelson Mandela t-shirt today. I suggested she take it off and wear it tomorrow. But it is as well to celebrate the life Madiba has lived, as it is to celebrate his birth. Besides, she says she'll wear it tomorrow too. Guess I'll be doing laundry tonight. (That's for you Doug. Grin.)
Mandela Day is July 18th, and it is a day to celebrate the great man's birth, life and to follow his inspiring example: "This day is a call to action for people everywhere to take responsibility for making the world a better place, one small step at a time, just as Madiba did. Madiba spent more than 67 years serving his community, his country and the world at large. On this day, people are encouraged to devote just 67 minutes of their time to changing the world for the better, in a small gesture of solidarity with humanity, and in a small step towards a continuous, global movement for good."
7/18/11: Wow, Google messed up big time -- there isn't even a tribute to Mandela on the google.co.za page! We need to celebrate people while they are alive! Too. So often we wait 'til they're dead or dying to notice that they had an impact on us that we value and want to acknowledge and celebrate.
7/25/11: Well, even if Google forgot, South Africa didn't:
"Monday morning Dad left for work a little later than usual. The reason being that it was Nelson Mandela’s birthday, and all over the country, at 8:05 people were going to sing Happy Birthday to him. It was estimated that 12 million school pupils joined the ranks of countless adults at companies doing the same thing. A special birthday song was even written for Madiba’s 93rd birthday, and we all spent some time familiarising ourselves with it. So, just before 8:05 we switched the TV from Disney Jr. to one of the local news channels and sang our hearts out." -- Norwin Lederer, Walk The Talk, 24 July, 2011
7/17/11 Great List
When I read Dave Gray's Connected Company posts (part of the book he's working on), I didn't notice his Readings page. It is a great list of readings, and his video and books lists are great too! See also Dave's collection of connection patterns.
7/17/11 Advising All Leaders
Partnering has become something of a meme in business/leadership circles -- like it isn't something spouses and domestic partners have been doing for the longest time! Not to mention many small businesses... Anyway, great partnerships like that of Bill and Melinda Gates has brought partnering into the limelight. And even if we have pop-meme antibodies, there are important lessons in the buzz-cloud a meme creates -- like this from a blog post by one of my long-time muses:
"Reason #3: You can't confront evil alone. Okay, evil is a strong word, but in the case of corruption or injustice, it fits. The same principles holds for more benign barriers, such as recalcitrant establishments, stubborn bosses, or reluctant associates. There truly is strength in numbers. Lining up others who feel the same way about a barrier or an obstacle makes change more likely. Coalitions lend credibility. They provide cover for controversial decisions, so that you're not a target for retaliation, or at least not alone — a principle used in the air cover around Libya. In Nigeria recently I was asked what small and mid-sized businesses can do to behave responsibly in the face of government corruption. The answer: Companies should get together and meet with officials as a group. A multinational has decided to take the lead."
-- Rosabeth Moss Kanter, Three Reasons Everything Goes Better with Partners, July 5, 2011
7/17/11 An Intelligent Take on Intelligence
What Is Intelligence, Anyway? by Isaac Asimov
And Robots Get Kinect's 'Eyes and Ears' -- this is major!!
And more outsourcing to silicon and steel.
It is my position that our greatest work, the work that enables all the work we do in the world, is the work we do on our own mind (and spirit). Sara scoffs at my protestations and claim to being a writer, but I have simply chosen to apply the development of my mind and the focus of my craft to this field -- one she doesn't recognize for its great writing, fiction and non-. All I do "in the real world" raises questions and stimulates my heart-felt concern, but also informs where I quest and what I see. It is very much that "what I see defines me" that I wrote in my by-line -- where I look and what I find is peculiar to me, what I see gets built into me and in that sense defines me. So it is a wonderfully geeky recursion, but poetic. I majored in Math and English Literature but my engineer friends' fascination with computers intrigued me into taking Computer Science classes (before I entered university I'd never seen a computer, much less thought of a career in software). That was a good thing because the combination made me a perfect fit for programming embedded systems in assembler (worship me; wink) and Forth (my introduction to creating DSLs, really). I became that other kind of writer, the kind that writes the fiction we invent in our heads into reality, into instructions that make machines do things in the world.
For as long as we've had language, we could through words create wondrous fantastical magical worlds that we transmitted into the imaginations of listeners or readers. We could use words to affect man's actions -- for good as Mandela has done or for tremendous evil, as Hitler. But with software something new, something subtly yet powerfully different is possible. Now, for example, we have pirates of the digital seas taking the keys to personal identities, wallets and secrets, crumbling the fidelity of intelligence agencies, companies and individuals and creating shock-waves of shaken trust in the digital shields we thought protected arenas of commerce and private life. For swashbuckling sport mostly -- for the power of turning words, instructions, into digital actions that humble the establishment "security" elite, demonstrating the power of thoughts. The tremendous power of words compiled into compute actions hooked up to the digital substrate that has embedded itself, woven an enormous and intricate web through our pockets and purses and homes and cars and businesses and governments. Words in languages only a privileged class -- an elite order undercover as nerds -- understands, and computers too. Words that, riding digital seas, can take short cuts; words from my mind to yours unmediated by the publishing gatekeepers. And take short cuts to digital and digital-mechanical action. Make things happen in far off places in the world. For good. And for evil.
And so with those words I demonstrate how with words we mediate our and other's experience of the world. We can use words to expose a "truth" others will lean to, if it is cunning and seductive enough. And with words you will either cast me an idiot who has lost her mind, or you will sit up in wonder at what my words can do. How words can be spun up, shimmer as if by magic some keen insight into view. And it will shimmer and dance and fade, if you let it, or take on life and get fleshed out if you take that shimmering image into your mind and build on it. Create your own views that blend my words with your own. Creating new generations of thought children you'll loose into the world.
And so it goes. You can choose how you see my Trace, box it up in your mind labeling it as intimidating for its very muchness. Or you can, with considerable generosity, apply a word like "enchanting" to it. Or useful. Different. A dose of remedial diversity reading for a field with very few women in it. Whatever.
I think I just used words to allow myself to fall again to the gravity of this digital web of mind and machine. ;-) That, despite the danger that those who would use words to cut my playful spirit down to size, could read here. Seeds of ideas and torpedoes of ill-intent both being allowed passage on these digital seas.
The first stop on the people problems list being me myself and I. Human fallibility being what it is. Decision biases. Cognitive biases. Ego. Defensiveness. You name it!
If my mind is just too busy, and you'd rather have something else to feed into your "diverse perspectives" pipe when it is too hot outside to make proper use of the weekend, here's a great list of podcasts.
And wow, as words go, the comments here remind me why I so love the software field! Funny, insightful, smart. Stimulating.
Tangential aside: Images of listening devices before radar and the inter[mind]net and the internet-of-things.
7/19/11: Posts like this one remind me that my internal Dobby, while active, may not be active enough. I've deleted it, and in protest replaced it. My daughter remarked that novels are not about real people because most people's real lives aren't as interesting as the fantastical lives of stories. Stories where protagonists take flight (on broomsticks even) and face the massing of evil in grotesque and ghoulish forms that threaten all that is beautiful and good. I've long protested that view. If we have lively inner lives, and active lives in the physical and social world, mediating what we encounter in the real world with a leaning to being awe-struck and kind, but also seeking, in that bliss-following way, truth and understanding, then our lives are as wonderful as any in a story. As full of surprise, and challenge. And yes, it will mean we sometimes dance, sometimes stumble even to the edge of moral hazard and fight that battle within ourselves that is mirrored and magnified in the greatest battles between good and evil in the world. So, in part, I insist, at least in my own experience, on a renaissance of self-perception. A liberation of full self into our work worlds, where creativity and innovation was never more demanded! Compartmentalization has served business well. But it boxes us in. Labels us. Divides us. Within and among ourselves. We need to find a new kind of harmony where we allow that we are human and that can be complicated. But wonderful too. The kind of wonderful where every person is the amazing protagonist of an wonderfully interesting, rich story! A story, which in its incredible interweaving with that of others, is an epic story full of themes of major social, technological, environmental and spiritual impact.
Of course, we're living at a time when things look so much the same, seem so much the same, as last year, as five years ago. This is because our techno-social worlds are evolving -- at a more rapid rate than in the age of cave paintings surely, and arguably more rapidly than any age before ours. But evolving all the same. So it is hard to remember that we didn't have the likes of Google or Facebook nor all that they give us access to just decades ago. There are tremendous shifts, many seeming small in isolation but they are so diffuse, and penetrate into every corner of our experience, expanding understanding and wherewithal, generating still more evolution. So there is a lot for us to figure out in terms of how to be in this ever more connected, interrelated world -- where the opportunities to hook up minds into the currents of human thought across centuries and geographies have expanded at overwhelming pace and at the same time we're galumphing into a world of much greater intertwining between human and technology -- so intertwined we can well ask whether we extend technology or the other way around. And we're doing it on-the-fly. Which, by the way, is why I think we have way too much going on to slow down and have another recession! We need philosophers to figure out the meaning of all this, we need historians to give us context, social workers to heal our social fractures in all this world-reshaping upheaval, ...., scientists to show us where we're messing up and how things work, and technologists to reinvent a more sustainable world. And storytellers who help us see and understand what is going on. Storytellers who often work in advance of theory, shedding keen insight on the world as we encounter it -- sometimes directly, and sometimes with imagery and metaphor. Helping us see what we want to be, and what we are that we must leave behind.
This from a Cisco infogaphic on the Internet of Things:
I think we need to let our imagination go to work, don't you?
First, we opened up learning and we've done much to, though sadly not completely, globalize access to learning. That democratized work of the mind, so that it was no longer an exclusive province accessible only to the wealthy and hence educated. Now we need another revolution -- one where we free ourselves from our societal conspiracy to censure the imaginative. To relegate creativity to a fringe of "artists."
7/21/11: America's Imagination Summit streaming live this evening and tomorrow!
7/25/11: I found this article by Google's resident philosopher interesting:
In it, Horowitz writes:
"There is an industrywide shift toward more "product thinking" in leadership—leaders who understand the social and cultural contexts in which our technologies are deployed.
Products must appeal to human beings, and a rigorously cultivated humanistic sensibility is a valued asset for this challenge. That is perhaps why a technology leader of the highest status—Steve Jobs—recently credited an appreciation for the liberal arts as key to his company's tremendous success with their various i-gadgets."
I'm not sure that what is required is a Ph.D. in the humanities, but I am sure that an inquisitive questing into more than the current sizzling technology (that too, but more) pays off in more worthwhile and insightful conceptions of products, their capabilities and the abstractions from which they are constructed.
7/26/11: For me, this post connected with some of the points Peter Merhold
makes as he is exploring and investigating themes of the
Connected Age. In
particular, I'm suggesting that the heightened connections between us, demand
that we be more connected within ourselves. The heightened connections with
machines, demand that we be fully human -- at work even. Not just to be distinct
from machines so we have jobs to do. But also to be the best of what makes us
human. Compassionate, connected, caring, vibrant people who do good in the
world. Even as we make a living. Who do the remarkable in the world, even of our
work. Connected within ourselves, connected to each other, and connected to this
wonderful digital substrate that extends what we are capable of intellectually
and together, in circles of friendship.and caring that extend now throughout the
globe as never before.
Out kayaking in the cooler part of the morning today, Dana and I came upon an absolutely astonishing beach of fossil crinoids on Lake Monroe. Perhaps the tremendous flooding in the Spring unearthed them. There is something amazing about being in the presence of creatures that lived millions of years ago! And seeing the structure, sometimes split so the inner ridges are revealed. And sometimes finding the negative space a creature left imprinted in a rock. Tonight I read First Impressions, about early cave art and Chauvet. I haven't watched it, but I heard the title of a documentary the other day and it is wonderful -- The Cave of Forgotten Dreams. I gather it takes up the story told in First Impressions.
I'm so glad that Grady Booch urged the Computer History Museum to preserve code, too. But I'd like to see the "cave drawings" and "sketchnotes" preserved along with the code. And since we have language, we could also, you know, preserve some of the stories that went along with the code that reveals treasured facets and mechanisms the architect/designer/developers were proud of because they wrestled with the gnarly problem of how to accomplish something and succeeded.
7/28/11: The CHM is interviewing and preserving "oral histories" of many of our field's luminaries and shapers. But I'd like to see designs and design fragments and discussions of gnarly/interesting/ground breaking pieces of the code preserved, and these are more contextual and "zoom in" deeper than the oral histories tend to do. (At least, my comment is based on those I've read -- I tend to read the transcript rather than listen because while it is nice to hear the person's voice, to fit more in to my day, I sometimes just need to go at read speed.)
7/19/11 Python Camp
We're assembling our "Python camp" environment. If there are Py-dev tools you
like (including and beyond PyDev), let me know what and why. (Yes, I know about
Pyro. And QT. Etc. But it's a big world and you're my trusted guides! So guide!)
"Conceptual Architecture" is (a conflation of the terms) the conceptual view of the architecture. This view includes diagrams and text which identify, explicate, rationalize and contextualize the key structures and mechanisms of the system, and the relationships among them.
We can think of systems as wholes interacting within their context. And we can think of a system in terms of the parts (which could be systems, sub-systems or other elements, components/parts) that comprise it, and their interactions. System design is a matter of designing the whole, and designing the parts and their interactions or the mechanisms by which the system sustains itself.
Software architecture is often defined in terms of architectural elements and their relationships, and Conceptual Architecture allows for the conceptual representation of these parts and relationships to support system design and evolution.
Conceptual Architecture expresses, at a conceptual level, the key architectural elements and their relationships that give "shape" to the system. Each of these architectural elements, or abstractions, play a critical role in the system, according to their responsibility assignments. They collaborate and interact, and these relationships are a significant design consideration.
The Conceptual Architecture Diagram renders the formative architectural abstractions and their interrelationships. For complex systems, there may be a set of such diagrams, exploring facets of more complex, architecturally interesting architectural elements.
The Conceptual Architecture also identifies the responsibilities of each of the architectural elements. We advocate doing this on Component-Responsibility-Collaborator-Rationale Descriptions for each (conceptual architecture) component, so that the reasoning behind component design choices is captured along with the responsibilities, and dependencies and other relationships are also recorded.
The architectural elements of software systems (that is, elements significant enough to the system to draw out and deal with in architectural design) are constructs of inventive human thought. They are not built from natural elements, nor organic materials. We do, of course, build from elemental building blocks that are programmatic and mathematical concepts.
We leverage analogies to draw experience and knowledge from other domains, including mechanisms of nature (e.g., swarms), man-made (e.g., mechanical mechanisms like hubs), and social constructs (e.g., social and economic mechanisms like brokers).
During early system conceptualization, we start to envision the form or shape of the system, its boundaries and interactions, its primary elements and their interactions. The sketchy shapes of these elements, expressed mainly in terms of their responsibilities, analogies, and drawing on patterns and experience, take a rough form, allowing us to investigate the system concept. Then, as the system concept (identity, value propositions and capabilities) takes shape, so too does the Conceptual Architecture.
In designing complex systems, separation of concerns is a key strategy for gaining intellectual traction and enabling ourselves to reason about important design decisions while considering emergent properties that are part and parcel of the decomposition-composition dual of systems design and development. The concerns focused on at this point are related to giving shape to the system paying attention to user-facing capabilities and developer-facing properties of the system.
As system capabilities are defined, the associated responsibilities are allocated to "chunks" or components of the system, and these responsibilities are an important thinking tool for architectural designers who need to make software abstractions cognitively malleable and communicable. With something so simple as a list of responsibilities, it becomes possible to assess
clarity of responsibilities
whether responsibilities are missing
components for cohesion of responsibilities
the system shape for balanced distribution of responsibilities
The conceptualization of a product isn't limited to the conceptualization of its use, but the product in use. We're designing what the product is capable of in conjunction with what the product is and how it is built. A car isn't a faster carriage, and although early cars had aspects of a carriage, the engine was quite different than a horse, and this translated to mechanisms to make the wheels move (from passive to active). In other words, when we conceptualize a new kind of product we iterate across designing capabilities and the mechanisms and structures that deliver those capabilities, and the product changes the ecosystem so to make the product successful we also redesign key aspects of the ecosystem or larger system of systems into which our system fits, interacts and in important ways shapes and is shaped by.
Working note: I
realize I need to massage this piece, and will illustrate it. Anyway, as
it is in creative suspension, your ideas and suggestions would be most helpful.
If you agree that we need to get a better sense of what design aesthetics means when it comes to software (the guts or developer experience, not just the user experience), you might like to read this thought-provoking essay on beauty:
It's like this, see:
Image source: funniest code comments you ever saw! StackOverflow
"... the musicologist Leonard Meyer, in his classic book Emotion and Meaning in Music (1956), analyzed the 5th movement of Beethoven’s String Quartet in C-sharp minor, Op. 131. Meyer wanted to show how music is defined by its flirtation with – but not submission to – our expectations of order. To prove his point, Meyer dissected fifty measures of Beethoven’s masterpiece, showing how Beethoven begins with the clear statement of a rhythmic and harmonic pattern and then, in an intricate tonal dance, carefully avoids repeating it. What Beethoven does instead is suggest variations of the pattern. He is its evasive shadow. If E major is the tonic, Beethoven will play incomplete versions of the E major chord, always careful to avoid its straight expression. He wants to preserve an element of uncertainty in his music, making our brains exceedingly curious for the one chord he refuses to give us. Beethoven saves that chord for the end.
Like curiosity, beauty is a motivational force, an emotional reaction not to the perfect or the complete, but to the imperfect and incomplete. We know just enough to know that we want to know more; there is something here, we just don’t what. That’s why we call it beautiful. "
-- Jonah Lehrer, Why Does Beauty Exist?, July 18, 2011
I really like but find myself not fully satisfied with Jonah's analysis. I want to know, to figure out more. ;-) My curiosity is piqued.
Actually, I've been toying with that "imperfection in beauty" thing for a goodly while. There are lots of places to go with it, not just in an evolutionary and neuroscience directions. Spiritual questions. Cultural. And more.
Here's another interesting take:
Math for Art's Sake, Dec 14, 2010
7/20/11 Hmmm, Carrots
This, about the Bredemeyer site (the other site I write) in a mailing list sign-up today:
"Your site continues to be a resource beyond compare."
7/20/11 And Taking a Dunking
Applying this to me: there's a difference between being a leader and just writing more than anyone else. ;-)
7/20/11 To Lead is to See, to Frame, ...
Here's the link: Vint Cerf: Essential Ingredients for Innovating in Government, Vint Cerf , July 14, 2011
Still not interested in Part II huh? Oh. That put me in my place! ;-)
Part II. Hm. As EA rushes to don the stylish BA robe and leave IT to the CIO, I'm wondering if Part I is worth a little attention?
7/20/11 Um, ...
"I drove manual transmission for years before I knew what a clutch actually did. It was just a lever I pushed with my foot that left me go faster." -- Scott Berkun, Nothing is Obvious to Everyone, July 20, 2011
I wonder what he used when he wanted to shift (change gears)?
Image source: funniest code comments you ever saw! StackOverflow
Couldn't resist. (In good part because I so know how that goes. One can mean one thing and write another. I'm the one that wrote 9/18/11 when I meant 7/18/11... No-one asked me if I really thought I was writing from the future. Says a lot about what my Trace readers think of me, huh? Oh, right. They didn't read that. Phew!)
7/20/11 A Peek at Watson
'When IBM began its work on Watson in 2007, it built a state-of-the-art Q&A system. However, the system could only answer "Jeopardy!" questions 50 percent of the time, well below what human champions were capable of.
IBM committed a huge amount of time and resources to overcoming this challenge, McQueeney said. The company assembled a cross-disciplinary team of several dozen experts and had them focus on the project for four years. This was a considerable gamble and a commitment of resources for the company, he said.'
-- IBM's Watson breaks new ground in artificial intelligence, Henry Kenyon, Jul 20, 2011
Grady Booch characterizes himself as a software archeologist, earning him the Indiana Jones of Software designation. I think that hat fits, don't you?
Along the lines of using the internet to extend our memory, here's a related aside I want to keep track of:
Todd Biske invites discussion of "implications" in Architecture Principles. You no doubt remember my post on Architecture Principles (grin), and our simple statement of the implications field:
But let's back up and look at another brief, but earth-shaking (well, it shook my ground), post, simply titled Principles. In retrospect it seems obvious, but its not what people talk about, when they talk about principles, is it? I'll repeat the insight here:
The point is that a principle is worth stating as an Architectural Principle if it makes a difference to decisions/behaviors that have strategic impact. If we need to change behavior [take a different action or (to more consistently) make a different choice or decision than would be the case absent the principle], there must be something that is driving or motivating the alternative behavior -- that's a counter-force. Then we have the principle name (catchy!) and its statement (to align decisions/behaviors), its rationale (connecting the dots to strategic intent), and counter-forces (what pressures may form against it or what drove the status quo) and implications (what do we need to put in place, to live by the principle).
That is, a principle is worth stating as an Architectural Principle if it causes something importantly different to happen than what would generally happen anyway, without the principle -- if, for example, in a situation of contention, it illuminates which hard choices should be made.
That has tremendous implications for implications, doesn't it?
Our brief guidance on the implications field has two parts:
You might ask if those two parts map to Todd's behavior and architecture/design implications, but there are behavioral implications that are a result of the principle -- things we have to do in accordance with the principle, as well as things we need to enable so we do those things. Perhaps your eyes are lighting up just now, thinking "I see a 2x2 matrix here", but there are often implications for budget and schedule, competency requirements, relationship implications, tooling implications, and more that might come up prompted by exploring down either channel.
Ok, so the exploration is one thing, and what we formally document doesn't have to be entirely the same thing -- which is to say we need to be careful to document only what really signifies. Document too much, and it will be ignored. [That harsh and perpetual lesson of my Trace, no? :-)]
So I agree with Todd that there are sub-parts to implications, but I'd caution that adding sub-fields is a slippery slope. Rather add guidance so any strong implications are identified so they can be resourced and attention paid and that sort of thing.
Instead, if you really want to add a field (to the usual template), add the counterforces/counterarguments field. [impish irony-recognizing smile.] That will help direct attention at an avenue of implications that need to be considered. Behavioral implications, yes. [Heartfelt bravo.] But economic implications, technical implications, etc. too. And focused at enabling us to shift attention and resources so the principle isn't an uphill battle for those concerned.
7/21/11: Mike Burke's post further illuminates this discussion.
7/22/11: For example, when we're thinking through what we will do as a result of the principle, we're assessing tradeoffs we'll make (differently) because the principle is in place. Yes, this relates to the rationale, and may show up there. Good enough. It relates to counter-forces and may show up there -- for example, from a local perspective of a person or stakeholder group, does the principle compromise something they're responsible for? They'll resist that. So what do we (organizationally and technically) need to do, so that we ease the path. [An image of sweepers in curling comes to mind.]
The big idea is that principles should be a small and highly significant, strategic set. To test for significance, we ask "how does this help our strategy?" but we equally need to ask "who will this hurt, and how?" If it doesn't cost, if it is the easy choice, that's a flag we'll look at when we're deciding on our small set of Principles. But if it hurts, what can we do to remove obstacles and enable choices consistent with the principle? We can't think of everything in advance nor document all implications, but spending some time thinking the impact of the principle through, is a responsible orientation.
7/23/11: Here's another take: An extended principles template, Jörgen Dahlberg, 2011-06-27
"Make yourself useful. Your life as an enterprise architect is not about being served. Roll up your sleeves, get in the trenches and serve others. Your list of principles isn’t as useful as you would think." -- David Bleeker, Do Enterprise Architects have it backwards?, 7/24/11
You could think of implications as directing attention at the cost (of attention, effort, resources, changes, etc.) side of an impact analysis, but it will also highlight positive downstream consequences that elaborate on the benefits in the rationale. The bottom line, though, is sift out and eliminate principles that don't make a highly strategic contribution. And for those that do (and you've tested with stakeholders around the organization to make sure you're not just going on a flawed hunch), think through the big obstacles and figure out what needs to be put in place, what conversations to have to build understanding and buy-in among key influencers, etc. You may not put all this in your principles document, but it is important to think this through enough. Just enough.
Aside: The points David Bleeker makes in his lessons learned are very much in accord with our approach. We really emphasize commitment to the success of our stakeholders (that is a defining characteristic of the "consultant" band of the architect competency model) and discourage "filling the process shopping cart" with lots of templates and models to fill out. Where we do use models and templates, it is in that "drawing people in" way that has a strong bent towards working in small groups "modeling out loud" and drawing on and creating circles of diverse influence to spark ideas and to improve designs. The rest is "what should I be thinking about at this incredible moment" stuff, and how best do I support that information seeking, problem defining, solution resolving in a way that is participative when and where it needs to be, but also values individual cogitate and steep time, priming and extending the reasoning that needs to be done in quiet undistracted time.
7/26/11: All this is predicated on the minimalist architecture guiding principle. If we are going to articulate principles and spend the organization's attention -- a scare resource -- on sharing and agreeing on and understanding and following, even governing, principles, then we want them to be a manageably small set. So we want them to matter. To make a difference. A strategic difference. Where by strategic we mean they impact our identity (including values and aspirations or defining mission and meaning) or differentiating value (what makes us sustainable economically and environmentally, in social, biological and technical terms).
As for values, here is a template we find helpful for it grounds the values statement in shared expectations:
valueName -- to statement of the value. Living this value, we: list of behaviors
-- adapted from two of the CAEAP EA Doctrine value statements
We may use a principle to interpret and provide decision guidance in accord with a value, or some other element of our strategy. As for values statement, same principle applies: what is the minimal set of values that underscore the identity of the group in ways that are important to draw out into explicit discussion and conscious choices. The organization has values. We know this most surely when the stated values are at odds with values implicit in actions/choices/decisions. Which values the strategic team draws out explicitly, depends much on shifts in identity and values and distinguishing the values of this organization, contributing to unique identity.
7/27/11: Culture works through our actions and stories and words, but if we overload culture with a truckload of statements about values and principles we lose the impetus of that medium for leadership because it simply becomes too much of a drain on cognitive and other resources. We draw attention to values that are core to our identity, that are or could be in contention in situations of tension or pressure, that involve hard choices, changes in what and how we do things, because they distinguish how we want to be, or they underscore a new strategic direction. If the strategy places innovation at the top of the strategic agenda, are there principles that will help us steer differently, to ensure that innovation happens. Principles around changing how we explore, experiment and interpret "failure" (or creating market, technology, value chain learning opportunities) with implications for eliminating layers of idea gatekeepers with vested interests in putting their stamp on ideas, for example.
One approach to identifying principles that make a difference, is to consider "a day in the life of the principle" and identify the dilemmas and choices where the principle would apply, and putting ourselves in the horn of that dilemma and seeing what impact the principle has, but also what it costs and how it hurts. We might argue that we value simplicity and so a "keep it simple" principle is not just motherhood and apple pie, but a deep value we need to cast formally in front of us so it factors in our decisions (explicitly) and designs and actions (with all their implicit decisions). Then we have to ask, when push comes to shove in little and big ways, do we live this value or do we just treat it as a nice idea that has another time and place but not here and now? If we have really decided that the way we're going to compete and differentiate is to simplify everything -- our user interface, our feature set, our product line, our distribution system -- then we need to set our course accordingly, and we are going to constantly make, and hold ourselves to making, tough judgment calls that pull in the direction of ever simplifying. "Keep it simple" is then not just a guideline. It is a high bar we're setting, and it means we're going to have to say "no" a lot and upset people. And they're going to have to say "no" a lot and upset people, and they won't like that either. So what are we going to put in place to make this happen? What stories are we going to tell and what messages are we going to reinforce over and over so the principle becomes part of the mindset? Remembering that these stories will convey what simplicity means to us and how it will differentiate us but also, importantly, what kinds of challenges and sacrifices we'll face and what battles we'll encounter so that our stories are full of the drama and tensions that make them vibrant and real, worth pursuing but not unrealistically easy. And we'll consider what resources we are going to make available -- for example, what training will we provide in design techniques targeted at simplicity? Yes, much of this is outside the bailiwick of the architects -- we don't set budgets. But we influence, and if the strategic thrust is to differentiate through simplicity and this is to be vested so deep in the organization as to become part of its identity (values are part of identity), we play a role not just in changing our talk, but in surfacing what others in the organization can/should do to enable the principle to be followed.
Values statements and principles are not the only way to express, communicate and reinforce values. If we want them to be powerful, we shouldn't try to push too much into those buckets. Yes, we want to articulate our defining values, the values that give us distinct identity. So we don't need, for example, to add values around preserving human life unless that is what makes us distinct. We hold that value. We assume that we all do, and it is not in daily contention. Unless that is a business we're in, or the environment changes tragically!
It occurred to me that it might be worth reducing the discussion above into a set of "litmus tests" for principles. This is our process, is it not? We explore and expand the space, gathering insights, and then we need to cull (horrid image I used there! lets rather say prune back to increase the fruit bearing capacity) to increase the powerfulness of what we're doing or advocating. It's that diverge and converge, expand and winnow, explore and decide kind of pulsing process that is very much the architect's process.
See also: Why Being Certain Means Being Wrong, Ted Cadsby, July 25, 2011 (the post is wonderful and Richard Melrose's comment has useful pointers)
A point worth bearing in mind though, is that uncertainty is hard -- really hard -- for many people and is going to draw hostility and even outright attempts to undermine and overthrow a person who is comfortable with ambiguity and uncertainty, replacing them with one who is more full of that comforting hubris and action-makes-the-world-certain attitude. A leader will need to adapt his or her style at different points in the lifecycle of the system, sometimes embracing divergent thinking and insight-seeking, idea-finding, connection-making, and sometimes drawing out clear lines of certainty that we will work within to build and deliver something. That something may be built incrementally and tested (in real use) and adapted somewhat during this phase, but there is confidence that the direction is worth pursuing.
See also: Simplicity: The Next Big Thing, Rosabeth Moss-Kanter, February 12, 2009
Ok, that's a playful post title, but the work referenced above is provocative (in the best way) and exciting!
7/21/11 Leaps and Sputters... and Leaps Again
"In a flywheel system, energy from the wheels is used to spin a flywheel at high speeds. The flywheel continues spinning, storing energy until that motion can be transferred back to the wheels via a transmission. The idea isn't new, but it's hard to make flywheels efficient—a lot of energy can be lost to friction. In 1982, for example, GM engineered a flywheel system that was intended for its 1985 vehicles, but they canceled the project after discovering that the fuel efficiency improvements were less than half of what they'd expected. Advances in the technology now have automakers taking a second look. "Industry has gone from being skeptical to thinking it can be done, but there are enormous challenges," says Derek Crabb, vice president of powertrain engineering for Volvo."
-- Kevin Bullis, Automakers Give Flywheels a Spin An old technology could make hybrid cars much cheaper. July 13, 2011
I remember when wind energy was the laughing stock of educated turbomachinery experts/mechanical engineers... Now you want to carbon date me? It wasn't that long ago, really!
One of the things I really liked about the way Shepler and Booch presented Watson is that they pay tribute to the software profession and all that went before, and hence into, the 4 years of Watson development. Watson is really a crowning achievement of our epoch, and a foreshadowing of more to come. These "shoulders and ashes" we stand upon are a great platform for burgeoning innovation. Again, this is no time for sliding back into recessionary caution and fear but rather it is a time for bold steps into an exciting, though not altogether without foreboding, future. As I see it, that foreboding only means there's more work that needs to be done! By philosophers and ethicists and systems thinkers and environmentalists and ... and ... and technologists.
7/21/11 A Measure of Our Values
If it works, it's good, no?
(David caught the perspective/prospective glitch; I simply scanned over that because my fingers sometimes misconstrue (wink) my brain too. I just loved the bigger point it makes about how entrenched our "working code is what matters" value-set is.)
I added yesterday's xkcd (on standards) to the Software Visualization Resources page. :-)
Google axed Wave. Now Google Health. This isn't just about Google being willing to bite the hard bullet of cancelling projects and losing star developer/architects. This is about Google being willing to throw our investment, our social capital, to the wind too. Of course, that means we have a shared responsibility to make Google+ work, so Google won't abandon all those who boarded this next in the line of Google's social ships. I just can't do my part if I'm not invited to play too.
7/21/11 This Just Happened?
Jest aside, it is a good thing that Uncle Bob is using his standing to draw attention back to architecture and discipline.
We're all learning on this software road, and the reason I do this is because I, more than anyone, have a LOT to learn!7/21/11 Decisions... or Tough Cases
I enjoyed this article: Insight: A Conversation with Gary Klein, Daniel Kahneman, 7/2/11
It makes something that has been jiggling and itching at me more clear. We tend to say that architecture is a set of decisions. This isn't just Grady Booch and Philippe Kruchten and so forth. This includes the likes of we Bredemeyer folk -- the Architecture Decision Model that provides the structure which enables the Visual Architecting Process to be highly iterative and exploratory with much forth and back multi-spiraling, dates back to the 90's. But many of the decisions are implicit in the design we craft, captured, at least in some form, in the models and views of the Decision Model. The foreground activity is creating the designs. As we do so, we're making a series of decisions many of which we don't even realize we are making.
"A simple situation. But, in fact, he had encountered a decision. Do you do an interior attack, or an exterior attack? There was a decision point. He didn't experience it as a decision point because for him it was obvious what to do. That's what 20 years of experience buys you. You build up all these patterns, you quickly size up situations, and you know what to do, and that's why it doesn't feel like you're making any conscious decisions because he's not setting up a matrix. But that doesn't mean that he's not making real decisions because the decision I would have made, he thought was a bad choice in this particular situation.
That became part of our model -- the question of how people with experience build up a repertoire of patterns so that they can immediately identify, classify, and categorize situations, and have a rapid impulse about what to do. Not just what to do, but they're framing the situation, and their frame is telling them what are the important cues. That's why they're always looking, or usually looking, in the right place. They know what to ignore, and what they have to watch carefully."
When we're designing systems, it is much like this, isn't it? Now it is true that a good part of what we do with our architecture documentation is (try to) make those intuitive judgments explicit so that we can share our judgment and reasoning with others. This helps us improve the architecture by involving more minds in testing early and evolving design ideas. It also helps us to: share expertise, effectively coaching by sharing the reasoning process; foster integrity of the design; help new team members understand the overall design and key mechanisms; etc. So we (try to) draw the tacit knowledge out of our minds by focusing in on what we need to convey -- for example, identifying forces on the architecture and how our design addresses those forces. But, again, our foreground activity is designing the system and many "decisions" are going to be intuitive judgment calls that show up in how we shape and form the system, what mechanisms we conceive and how we (architects and other developer-designers, but under the ultimate design authority of the architect who is responsible for system outcomes) design them. As documenting goes, there's figuring out what to draw out so that it is explicit and discussable in the team and beyond, and what to leave implicit.
Oh sure, some decisions are going to be explicit and have an obvious analysis trail. A significant technology choice with ramifications for system lifecycle cost / cost of ownership/operation implications is probably going to be obvious enough to garner attention -- even be a matter of contention. [While others will just slip in on the back of other choices that are made.] Probably. Of course, when we "ship the prototype," all kinds of implicit decisions not intended for prime-time get shipped.
Tangential aside: Speaking of analysis trail, I read the VizTrails chapter of the AOSA book. They use the word "provenance" and I suspect that as we get more into MDD, provenance (or exploration histories) will become a big deal. What do I see? Well, if we can reduce the cost of development, even within well-bounded cases/domains, to being largely a matter of model development, we can take a much more trial and error tack. Following "if you give a mouse a cookie" logic, we quickly get to "we'll want to know what we did two trials ago" -- involving not just model versioning but also hypothesis and results associating/tracking.
Further aside: there is a thrust in Agile that eschews leadership. Apparently because it is thought that empowered self-organizing teams shouldn't and can't have leaders. Leaders, in the chaordic (Dee Hock) sense, empower their teams to lead -- to lead their leader, while their leader leaders his or her peers and seniors. In complex organizational settings, this makes a lot of sense.
7/21/11 Tracing Bugs!"In 1876, Thomas Edison coined a term we use to this day to describe those pesky glitches, malfunctions and other deviations from the intended paths of technology:
It has been just so in all my inventions. The first step is an intuition — and come with a burst, then difficulties arise. This thing gives out and then that — ‘Bugs’ — as such little faults and difficulties are called.” ~ Thomas Edison
So opens Dean Buonomano’s excellent new book, Brain Bugs: How the Brain’s Flaws Shape Our Lives"
-- Brain Bugs: The Glorious Imperfections of Our Brains, Maria Popova, 7/20/11
So that's where it comes from... at least, that's presumably the first reference in print. Edison seems to have been saying that the term was already in use...
7/21/11 Landing Zones
I really like the concept of landing zones in Rebecca Wirfs-Brock's latest blog post. We likewise point out that we should view quality targets as being on a slider, characterizing a permissible range (around the target value) to work within rather than an absolute number, and the vividness of the aircraft carrier metaphor is appealing!
We have long leveraged what Tom Gilb and also the SEI team have done in the area of "quality attributes," and they have sensitized our field to the importance of characterizing quality attributes, etc. That said, given what I see teams struggling with, I'm working on/testing some different ideas. (For as much as I babble on here, I am very careful about keeping VAP simple and sparse.)
That reminds me, yesterday I read through Scott Sehlhorst's slidedeck on Kano Analysis. I like his adjustments to the model, recognizing that "more is better" is not linear, and that not all customers are the same.
7/27/11: Defining quality, Seth Godin, 07/2011
Our heartfelt sympathies go out to everyone who lost loved ones and who were directly and indirectly traumatized by the attack on Utoya Island, and to the people of Oslo.
The last time we were on the lake, there was a line of yellow roses floating
on the water and it was lovely and so sad. Lovely that a person lived and was so
loved, and painfully beautiful that the person is so missed that on the
anniversaries of birth or death yellow roses are left in their wake. My brother
lost a 16 year old son, and a close friend lost his 19 year old son. In both
cases it struck me that teenagers are so exuberantly present in the world that
losing them is extraordinarily huge and painful to such a wide circle!
The link? Dinosaur comics, of course!
7/22/11 Context Maps
Context Maps in software development methods often focus on the technology context. These are important. But what I am going to explore now is the broader ecosystem context into which the system fits. We have used Competitive Landscape Maps to explore business context (described in Getting Past ‘But’), and the model that underlies the Map is our strategy model. Dana suggested we go back to Context Maps and I agree. The reason is that we need to look at the value network and entities that add value, rather than simply looking at customers (and market segments) values, concerns, frustrations, etc.; and competitors and how they serve the value system; and our value propositions, capabilities and challenges.
We've long used an adapted variant of The Grove's Context Maps, and recommend their strategic visioning group graphical facilitation workshops and graphic guides.
Other styles of Context Map
Or, for fun, you could create a treasure map. Complete with "there be dragons" locations, what prevailing winds mean, where x might mark the spot, etc. ;-)
Context maps can also be used to understand the context of decisions made in the past. Here's a neat historical example: Context for treaty making in the NW.
As maps go, are these associated other than coincidentally:
Context figures. It is good to map it, to picture -- to BIG picture -- it.
Why? Oh, you know, It Draws People In. Something IT hasn't been so good at, but
needs to be. We've gone beyond the language of "alignment" (where the "elephant"
in the room is the unspoken assumption that this means that IT needs to kowtow
better to the business) to the language of partnership. We need more seamless
ways to translate that language into action.
Back in 2007, Bill Branson asked me some interesting questions. His attention shifted to other things he needed to do and he never used my responses. I think they cover some good points about the value of graphical facilitation a la Dave Sibbet (author of Visual Meetings and the forthcoming Visual Teams) and The Grove. So here they are:
Bill: How did the two of you get exposed to graphic facilitation? (I seem to remember it going back to HP.)
Yes, it dates back to our days at HP. The Software Initiative (SWI), the internally focused software consulting group that both Dana and I worked in at Hewlett-Packard, really invested heavily in developing its people. At the time that I was part of the group, it was headed by Ron Crough, and he and the team at SWI were always scouting out leaders pushing frontiers that would stretch us as software consultants. By the time I joined SWI, it was standard for SWI consultants to take graphic facilitation workshops from The Grove. Dana also attended a Strategic Visioning Workshop facilitated by Dave Sibbet at The Grove.
In addition to the Grove influence, we also did modeling work with Charles Faulkner, among others. SWI’s own Rand Barbano and Ron Crough were also strong facilitation coaches and role models. Rand and Bill Crandall set the bar for “teaching through metaphors.” It was a group who took seriously our role in working with innovators across HP. If we were to have “goodies” in our toolset that would make us effective at helping some of the best product development engineers and architects, technical managers and executives, in the world, then we had to keep learning and stretching too.
Bill: It is very main stream to your workshop delivery and teaching. What attracts you to it?
All the things you’ve said about the power of graphic facilitation! It is effective for actively engaging people, and we use graphic facilitation to get important work done in a training workshop. Using graphic facilitation for early large-group exercises, we demonstrate a technique we believe (and see in practice) is vital in the architect’s personal effectiveness toolkit, and at the same time we build group understanding that feeds into the architecture work. In the workshop we have just 4 days to accomplish a momentous amount, often moving from little knowledge of the case study problem to a draft of all the various facets of the architecture, including vision, scope and architecturally significant requirements, and architecture strategy and conceptual architecture, and we’ve explored key pieces of the logical and execution architecture. All this, and training too! So we have to use “better than Pareto” tools to move our architects through the upfront vision/strategy pieces really pretty quickly, building shared understanding and buy-in, and the graphic facilitation tools are central here. If you think how long this fuzzy front-end usually takes, how hard it is to get real traction, then you start to realize that 4 days to do just that wouldn’t be long at all! But we don’t have that kind of time, and graphic facilitation buys us time. It creates a high-leverage group mindspace, one where everyone works quickly and effectively together to build a shared picture.
I find that even when I’m facilitating (I have the pens), I’m not part of the foreground. The graphic space is the foreground that the group is paying attention to. It becomes the shared view, the shared picture. For example, if we’re trying to create a shared sense of our business context, the best way I’ve experienced to do that is to get the right people in the room to create a big picture (literally) of the competitive landscape. Everyone gets to share their perspective, and see everyone’s perspective being integrated. The shared picture that emerges is vivid, vibrant with images, concerns, values that have been injected by all the different people in the room. It builds a sense that this view we’re building, this context, is shared, and more than that, it has its comfort and its motivating force. I think it’s a really neat thing that the facilitator isn’t in the forefront, trying to persuade and otherwise bludgeon (figuratively) people into alignment with his/her ideas. Alignment is a natural outcome of using social tools like this. And yes, by being careful about inviting key individuals with various important perspectives, we play a shaping role in creating a valid shared context view. Out of this context view, we see opportunities that we can go after to build differentiation, we get a shared sense of where the competitive heat is being turned up so we don’t get complacent, and so forth. Shared context provides the motivating ground for a vision, as well as being a source of exciting possibility.
Complex software is a collective work product, so constructing software systems necessarily has a social dimension, complicated by the fact that the work is innovative and non-routine. As architects, we ignore that social dimension at our peril. So, throughout architecting, we are trying to create the best technical solution to the most strategically positioned problem while building understanding and buy-in. We use graphic facilitation to draw in participation, build alignment and get better outcomes because we got richer, broader input in a visible and shared forum. It blends active listening with building a vivid shared picture, rich with imagery drawn from minds of our key stakeholders, so they see themselves in the picture.
Bill: What kind of results or feedback do you see or hear from using it all these years?
We have been doing architecture workshops integrating graphic facilitation and visual modeling for some ten years now, so we have seen architects grow through a significant span of their careers. We hear from architects who have seen multiple architectures to successful deployment, and we hear from their managers and their peers and reports. Generally the feedback isn’t specifically related to graphic facilitation, but I know that the results are very much tied to graphic facilitation and to visual modeling, for these are at the heart of, and the integrating, driving force of visual architecting. But sometimes we specifically hear about use of the Grove style graphic facilitation approach, and one group has just spent a few weeks developing context models. I’ll ask the architect who led this work to check in on your blog, and comment directly.
Bill: Any top three suggestions for getting familiar with this mode of teaching and facilitating?
Bill: Any advice about what to avoid?
It seems like the biggest trap is the “it’s all very well for you, but it couldn’t possibly work for me” trap! I don’t claim to be at all good at this. Dave Sibbet is a true master. Dana is masterful. And I just know you, Bill, are masterful, though I haven’t had the privilege of seeing you in action. But me, well, I certainly don’t have the charisma base covered! And my writing is horrible. So I think I have all the weak spots pretty well covered! Yet, in spite of me, we always get really great work done using graphic facilitation tools. In workshop after workshop we use Context Maps to get everyone’s head into the game, and with intact teams we also do Graphical History, Architecture Roadmaps, and Vision. Bottom line is, if I can be comfortable using graphical facilitation, and if we can get great work done in spite of me, this is a technique that depends very much on the participants and their attitude. So just do it!
Of course, this is graphically facilitated group brainstorming, so all the classic pitfalls apply. For example, if one person is sounding off negatively and trying to derail the group at every turn, you have to deal with that. If the group is stuck in some rut, you have to notice that, and move them to a new space on the picture to free them again. As the facilitator, you can’t get stuck on your pet peeve, or use all the time selling your agenda, and so forth.
It would be interesting to see how many of the architects we’ve pointed to The
Grove have become CIO’s, CTO’s, chief architects, and so forth. I have heard
from a good number, and I suspect there are more out there. These tools really
are effective in leading people. They help you draw in and use input from a
diverse and large group. Hub-and-spoke mechanisms (like one-on-one interviews
and personal conversations) have a place, but these group tools have huge
leverage! You can get entrenched, stuck groups to gain momentum, and you can get
excited, passionate but unfocused groups onto a shared page, a roadmap to a
vision they care about building together. Architects have to deal with this;
graphic facilitation lets us do so effectively.
Additional comments, 7/23/11:
Of course, we don't by any means stop at The Grove-styled graphic facilitation tools. We also use rich pictures and stakeholder profiles and we use empathy maps and force-field analysis from Gamestorming, among others, and of course, we sketch key structure, behavior and interaction diagrams using UML (and sysML) -- emphasizing the value of "model out loud" in team forums. We use ad hoc diagrams to sketch mechanisms. We use textual templates too, of course. We fluidly move with the key concerns and decisions that we determine are important to consider "at this extraordinary moment."
We have introduced thousands of architects to graphical facilitation, and I know that many have followed our recommendation to follow up on our pointers to The Grove resources, and to go learn from The Grove source -- and since its publication,David Sibbet's book Visual Meetings. At the same time, I know that many have not. There is a hurdle because architects very often feel painted into the "design the guts" corner, and there are various concerns including not wanting to step on someone else's toes (like the product manager's, and stepping there can provoke strong defenses; product managers being, like the rest of us, human) when it comes to looking beyond the internal construction of the system and the technical infrastructure context. One thing architects in that situation can do is encourage their managers to work with us, to help clarify how to set up roles and foster practices that enable design conceptualization to be fast-tracked in a highly agile, lean, iterative and incremental, highly visual, highly participative and empowering approach. The key, whether we help make the case for it or not, is to partner with business analysts, marketing, product manager and so forth, so you aren't "taking charge" of early conceptualization work (at the outset, and at significant pivot points in the evolution of the system and its derivatives) that others feel they own, but rather engaging in a dynamic sort of dance where you enter into a rich collaboration and help create a reframing where it is recognized that these different roles come together in a core system concept team that shares ownership for leading to good, right, and successful! Leading, in a highly collaborative, highly empowered sense that is not at all the same thing as commanding and controlling! But also is decisive and resilient where it needs to be. It is simply important to have someone paying attention to the system, from concept to code but in system terms, not in this feature or that, this algorithm or that, etc., terms.
There is an interaction between the system as a set of collaborating, interacting parts and capabilities (the "guts") and the system as a whole and the capabilities it presents to the world. We need to understand that this interaction is designed in ad hoc entirely accidental ways, or in more intentional, attentional ways that bring experience to bear but also a keen eye for what we don't know and need to experiment with to sort out. Ideally the architect has that whole-system charter. Doesn't do it all. Couldn't! But is a pivotal connection between the different design frames of system and its capabilities (working with the product manager and business analysts or requirements analysts and marketing), the internal design and all the capabilities, mechanisms and elements within the system that together deliver the system capabilities, as well as system deployment and operations, the selling cycle, system evolution and product derivatives, strategic decision making about system or product direction, and more. If the architect doesn't have that whole-system charter, then the architect needs to partner all the more with those who have the charter for these other design frames, otherwise system design -- design across all these boundaries -- is not coherent, and the lack of system integrity will be telling.
[Sorry, my nested sentences are meant to be parsed iteratively... First, ignore the bracketed clauses. Then think of an objection to what I just said. Then read the sentence with the bracketed clauses to see if I addressed your objection. ;-) ]
7/23/11 A Woman's Place
This is interesting:
A Woman’s Place: Can Sheryl Sandberg upend Silicon Valley’s male-dominated culture?, Ken Auletta, The New Yorker, July 11, 2011
as is this:
Android alert, The Economist, Jul 20th 2011
An Asian woman going head-to-head with Steve Jobs!
"The company has continued to produce a string of subtle but clever features: phones that ring louder when placed in a handbag; ones that stop ringing when flipped over in a meeting; ones designed to work smoothly with Facebook and other social-networking sites, and so on. It has made progress in building a brand that reflects innovation and trust, allowing it to escape from the low-cost treadmill on which some of its peers remain stuck." -- Android alert, The Economist, Jul 20th 2011
At some point, groups like this will work harder not to be so glaringly all white male, not because they have a diversity box to check off but because they value what happens when women are part of the conversation.
Here's some interesting perspective:
7/25/11: I included this to underscore the point that it is not just that we need gender diversity, but color and cultural diversity too.
India's Leading Export: CEOs, Carla Power, Time Magazine, August 1, 2011 [Yes, Time writes in the future! ]
7/25/11 Excellent post, awesome point:
"The language of competition not only doesn’t appeal to many women, it actually puts them off."
-- Leah Hanson quoted by Anna Lewis in Girls Go Geek… Again!, July 26th, 2011
Wow. I sure hope that Joel reads that.
7/23/11 Google says "NO"
This sounds great:
"Making plastic from sugar can be just as cheap as making it from petroleum, says Dow Chemical. The company plans to build a plant in Brazil that it says will be the world's largest facility for making polymers from plants."
-- Kevin Bullis, Cheap Plastic Made from Sugarcane, July 25, 2011
Replace fossil fuels with renewable sources. Except. Will this encourage more rainforests to be cut down to make room to grow sugarcane? That would be a loss to the planet on the order of the loss of fossil fuel, so what is being done?
This makes the value of context maps clear, doesn't it? To find opportunity, and to assess our approach to taking advantage of that opportunity, we need to step back and look at the bigger picture, the context into which our value contribution fits.
When our local co-op grocery store installed a new system for checkout they put up a sign saying "please be patient; our new system is getting used to us." A nice little twist of humor there. We have to adapt to our systems; sure, they may try to accommodate us, as best the designer-developers can conceive how, but they still fit into bigger (eco-econo-)socio-technical systems. And bigger system will reshape itself around the system, accommodating to it. (Well, if the new system is too poor a fit, asking too much adaptation from the context, the context will disgorge the new system.)
Organizations change the ecosystems into which they fit. They explicitly, but also in untold ways with little considered and many unintended consequences, impact the direct value network, and the broader ecosystem. Some of this we couldn't foresee. There's too much uncertainty, too many interrelationships, too much that is hidden by all that complexity. Nonetheless, as we increasingly recognize that economic sustainability is linked to technical sustainability and social and environmental sustainability, and that these are interconnected and impact not just long term results but short terms results too, we realize that we need better tools to help us contend with, to contemplate, to reflect on, to observe, to make new connections. Better tools, but not exhaustive -- and exhausting of attention and resources -- tools.
We want lightweight approaches that do "good enough" in the face of huge uncertainty and complexity that means we can't know or deal with it all. And in the face of needing to get better at surfacing what is important in that make-or-break sense -- when we think about the broader issues of sustainability. Sustainable -- exciting -- value propositions for customers and shareholders and others in the value network including employees and value partners (in the supply chain directly, but also, for example, advocates in the social networking space). But also environmentally sustainable choices that consider the huge debt we are building to Mother Nature.
We find that Context Maps bring structure to the conversation that helps a diverse set of stakeholders build a shared picture of key facets of the ecosystem.
7/26/11 Experimentation and Discovery
What is the common thread in these three pieces:
i. The Montessori Mafia, Peter Sims, April 5, 2011
ii. Tim Hartford's TED talk "Trial, error and the God complex":
iii. Frank Gehry as a young rebel from TED 1990:
Yes! Experimentation. Trail and error. Discovery. Often accidental discovery. Serendipitous discoveries that cause the path to break, pivot, diverge, lead to something unexpectedly better! False starts. Spirals of learning that "go nowhere" ending as "failures" -- but really are the very fits and starts, bursts and disappointments, implosion or setbacks and renewal that are the path of innovation. Interesting how Gehry's sculptural furniture, including his rebellious protests at his success in furniture, seeded something in him that showed up in his innovations in sculptural buildings.
'Mr. Bezos often compares Amazon’s strategy of developing ideas in new markets to “planting seeds” or “going down blind alleys.” Amazon’s executives learn and uncover opportunities as they go. Many efforts turn out to be dead ends, Mr. Bezos has said, “But every once in a while, you go down an alley and it opens up into this huge, broad avenue.” ' -- Peter Sims, The Montessori Mafia, April 5, 2011
Now, in business we try to curtail this messiness and put a front -- a facade -- of manageability on it. If we deny too much, if we put on a bluff of certainty, our cost of failure goes up, and up. Instead, we want many mechanisms that allow contained experimentation, so we don't bring the whole starship enterprise down with us when a tack we take turns out to be a dead-end. We contain the cost of early experimentation by using the cheapest, fastest ways to explore a big value space, scanning for big opportunities and promising "alleys." We invest just enough, with a few good men and women leading the early explorations, moving quickly, working with messy, cheap hacks to prove and disprove ideas. And improve the sense of which avenue is worth investing in, worth opening up. And as we get clearer what it is we are doing, we bring other architectural principles and practices to bear. Separation of concerns and decomposition, to isolate areas of great uncertainty and experimentation, and spawn off highly experimental, investigative initiatives to resolve that uncertainty. Our broad brush investigation of opportunities to create differentiating value shifts to delivering increments of value, testing out our concept of what value is and how to deliver it, how to delight users (and customers, if users don't manage the purse strings, or if customers are a subset, as in the freemium model).
This is the messy highly iterative process of VAP, where we embrace messiness and seek it out -- bringing diverse expertise, perspectives and styles into the process early, for example. Again, this is messy! Cognitive distance that can sound like cognitive dissonance. But we can take a chaordic approach, allowing that there are realms of chaos and uncertainty and exploration, and ways to bring these to some order. We have an organizing model to keep track of and map where we are, we have a way to make sense of and to tell the story of what we're about and where we're at. Thus, even though our path has lots of forth and back, lots of little investigations that may lead nowhere, lots of backtracking, even direction resets, we have a way to structure and frame our discoveries so we can build confidence and demonstrate where clarity and certainty has been built and where big make-or-break uncertainties remain and how we're going to approach those and (roughly) when.
I read Nick Malik's post Finding Common Ground: in response to a BPTrends article on Process and Capability (26 Jul 2011) with interest. We were early to the "capabilities as architectural element" game because when process was designed without taking technology (existing systems but also possibilities in the technology space) and people into account, things faltered not always but in ways that could be avoided, and likewise if technology was the driver. Capabilities are multifaceted. They provide a view of the enterprise that overarches these facets, so that we can explore what capabilities our business has in place and needs to adapt or build to support its strategy. Enterprise architecture is so very important because it uniquely takes an enterprise vantage point and does not make process or technology or people a subservient design consideration a priori. If within our innovation capability we have "customer insight" identified as a capability we need to prioritize and build up (adapting and leveraging what we have in place), we have facets of business intelligence, process, and people/relationships interacting together to create that insight generating capability. We recognize that more informal, ad hoc, spontaneous, emergent kinds of insight generating approaches rely more on relationships than explicit managed processes but they ride on a substrate to relationship platforms that have security implications and so we're back into the space of technology. And so we see that capabilities allow for this richness and interaction, designing across facets and perspectives and zones of expertise and organizational control.
Capabilities are also fractal, and so we can end up with these hierarchies and we do need to watch the manageability just as we have to watch any top-down kind of design. We can use capabilities to set context, and empower those who impact and build the capabilities considerable design freedom within that context.
We know that what the future holds is that more and more of the "systems integrating" and "intelligence" functions of humans is going to be shifted to robots and other automated assemblages of metal and electronics with that invisible cloth of software binding it all together and giving it more and more semblance of "life" -- of sensory and motor capabilities, the ability to learn and adapt, the ability to sustain itself. A self-driving car is a wonderful avenue for experimenting with all this, because it has applications we embrace enthusiastically, like making blind people more self-sufficient. Not to mention taxiing my kids to all their lessons. So it is PR goodness. Just like Watson and Jeopardy! And Watson's first foray into the commercial world being into medicine.
I'm just kidding! I don't think there's a sinister plan. I totally believe that the Watson team means only to give us a better tool to serve humanity. Wink. Wink.
I'm just kidding! But I do think that the Google self-driving car will lead not only to business in the auto industry, but to other kinds of applications. That kind of sensory response system has so many applications.
Google is moving apace to carve out a chunk of the social networking space that was, in effect, declared "done, taken, move on" by Facebook. I for one am wondering what they'll do with this car project as they shut down Labs... Not sure if it was even run within Labs...
7/27/11 Other Interesting Tidbits
Two weeks ago today, I received an email full of the mixed messages this Trace has duly earned, with the subject line "Your blog is really .. intimidating." 5 1/2 years of sharing this exploration has been met, with few marked exceptions, with silence. One email out of the blue, and that's it! That's the reaction. I didn't want to be rash, so I let some time pass. Well, the conclusion I drew then hasn't changed: Time to apply the "less is more" principle to what I do here.
We regularly get feedback like these about the Bredemeyer site:
"Thanks guys, I find your work inspiring." -- Martin
"Thanks for your software architecture site. Wow, what a resource." -- Douglass
The contrast between these responses and "intimidating" bears consideration. My gut reaction is that if my Trace creates impetus more to hurt than to be kind, I shouldn't do it! But my reasoned reaction is that I've long had concerns about how this "open brain experiment" is received and perceived, and one architect taking on the responsibility to speak the unpleasant truth he sees is only the proverbial straw.
So, push came to shove and I'm working on a different format for the public front to my scratchings. I'm not sure when I'll feel ready to make the new format accessible. We'll see. In the meantime, I'll continue to post some of the links I find interesting or useful to draw into the current debates on what architects (of various scope) do and how they do it.
Anyway, it is a service to stop [potentially] contributing to the glut of distractions -- see
It's time to take back your attention, Tony Schwartz, 28 Jul 2011. (The
alternative point of view is that given the information glut, trusted scouts are
valuable. Trusted then being a significant word.)
7/27/11 Useful Posts on Systems and Failure
"A pure "robust" system is highly process-based, engineered to return to the status quo regardless of what gets thrown at it. The problem is that a robust system (by definition) is designed to operate the same way in the face of change. Any breakdown of a robust system is likely to be a catastrophic failure.
By contrast, a pure "resilient" system is flexible enough to change to deal with almost any unexpected internal or external pressures. It is not formless but by being so sensitive to pressure it's state at any given moment is very hard to predict."
-- Resilience vs Robustness, Stephen Bounds, 4/16/2011
Context maps are used variously in strategy and system development. In developing business or product strategy, context maps focus on the broader environment into which the business or product fits. This is for example, true of The Grove's Context Maps, and Bredemeyer Consulting's Competitive Landscape Maps (a derivative of The Grove's Context Maps adapted to our strategy model). We also use rich pictures to focus on the system in the context of other systems, organizations, and other significant entities, including individuals or roles. We will explore system context maps and rich pictures separately. Here we are "zoomed out" further, if you will, to take in the forces and concerns in the broader environment. We are doing as broad a scan as makes sense given our charter.
7/28/11 The Architect's Role in Context Mapping
Architects generally understand the importance of context. We say "it
depends" a lot, and what it depends on isn't just "context" but specific
contextual factors that aren't only technical. Our choices are context
dependent, meaning we choose one option over another based on desired outcomes,
starting state, resources available, and so forth. Well, context is a big
matter, and a lot of it lies between the lines of stated objectives and official
positions documented in "the strategy" and "the requirements." So one of the
reasons to include architects in strategy formulation activities like context
mapping is that they hear the conversations that create shared context from
which the strategy emerges and which make the strategy make sense, and they take
this context into the strategic architectural design decisions, and become the
conduits for transmitting and translating context for more narrowly scoped
design decisions. That is already important, but architects bring something
important to creating the shared view of the context too.
7/28/11 Highly Recommended!
Mentioning his Dr Dobbs article, Discovering Hidden Design (July 26, 2011), Michael Feathers expressed some reservations:
@KentBeck Thanks. I'm a bit embarrassed by the `ignore semantics' part of my argument, but I think it's right in many cases.
In response, Kent Beck tweeted:
@mfeathers i agree completely. i often urge partners to "stop thinking" and follow the morphology for a while.
I'm guessing that Kent means "stop over-thinking, and go with the structure that is emerging from intuition and tacit knowledge and blink judgment for a while."
(Side note: morphology is the study of the form and structure of organisms and their specific structural features.)
This is from a presentation I put together for Dana last year (Dana has that awesome American male voice, and he gets invited/requested to do more of the roadshows than I):
Image sources: Feynman: Gleick's Genius; Feynman at 10: No Ordinary Genius: The Illustrated Richard Feynman; Louis Agassiz: wikipedia, fish: Hćmulon elegans drawing by H. L. Todd; Chuang Tzu: not ascribed when used here, so I need to find the source to ascribe it properly.
If you haven't read Surely You're Joking Mr. Feynman, I recommend it highly! The "fixing radios by thinking" story is told in that book. The boy Feynman thinks about what could be wrong with the radio, what the options are to try, and then he tries the likely candidate and it works, impressing his (skeptical) customer who then becomes his most vocal advocate.
I mentioned Agassiz* in a journal post in early 2010, and there is a set of delightful stories told by Agassiz's students of their first weeks and months studying under Agassiz, for the most part following the same pattern -- close, attentive observation of a preserved fish to "discover the relation between form or structure and function or essential effect." The last pair of story slides in that sequence focuses on the Taoist story of the master butcher. The "Wha...?" is responding to the anticipated question "What does this have to do with software?" Well, these all build a picture of the development path of a master -- for example, a master software developer.
I drew the sequence into the introduction in the Software Architecture Workshop, and here are the "teaching students to observe morphology" slides:
We observe -- notice -- what we and others have done and develop a sense of when there is, and how to create, fit of form to function and outcome. In pair programming and apprenticeship models we hear the verbalization of reasoning, but even studying another's code to change it presents these opportunities to observe closely what others, and our past selves, have done. We develop not just "blink" intuition but an aesthetic, a "point of view" or style. Sure, each system brings new challenges and we still need to experiment and discover, but we build mastery as much in shaping these experiments as the code itself.
Warning: Agassiz's racist theories have earned him quite some disrepute. The point here has nothing to do with his theories of race or evolution, only the system of observation that highly influenced his students and many in diverse fields, even the study of literature, since. Structure being of interest to so many of us.
7/27/11 Anatomy and Morphology
I was interested in the difference between anatomy and morphology, and this came up on a quick googling:
"Both a layman and a professional seeing a beetle firstly pay attention to its appearance and features of its external structure. In coleopterology, and in zoology in general, there is a special discipline, morphology, studying body structure. There are two points of view concerning the scope of this discipline. Some zoologists believe that morphology (a study of forms) concerns both external and internal structure of an animal. Thus, they distinguish external morphology (eidonomy) and internal morphology (anatomy). This, for instance, was the viewpoint pursued by the eminent entomologist G.Ya. Bei-Bienko in his manual on general entomology (G.Ya. Bei-Bienko, 1980). Other specialists regard morphology as a study of only external body structure, and anatomy as a study of internal structure, separate from morphology (e.g. Zakhvatkin, 1986). We prefer the first point of view and, therefore, will consider morphology and anatomy as a single discipline." -- Morphology and Anatomy
And this would go down well in ontology circles, methinks:
7/28/11 Link Round Up
Software Architecture Workshop:
Enterprise Architecture Workshop:
Role of the Architect Workshop:
#jobcreationNice. Solar Energy Is the Fastest Growing Industry In the US http://bit.ly/r2awx1
We have this mindset that constructive criticism is a good thing, giving us sanction to lash at, demolish what another has built, patting ourselves on the back for setting a Phoenix free... if the destruction leaves will enough for that. Wouldn't it be better to recognize what is good, so that good is built upon it?
7/29/11 We're Losing Our Capacity (and Appetite) for Listening
"We don't want oratory any more, we want sound bites." -- Julian Treasure: 5 ways to listen better, TED talk, Jul 2011
See also: The Power of Listening, Geoffrey Webb, July 27, 2011
7/29/11 A Positive Voice
"take some personal responsibility for some things you think are outside your bounds. You'd be surprised at how welcoming your colleagues will be at your engagement." -- Angela Yochem on the State of Enterprise Architecture
7/29/11 Collaborative Approach to Conflicts
"Defining conflicting interests as a mutual problem to be solved by collaborative effort facilitates recognizing the legitimacy of each other’s interests and the necessity to search for a solution responsive to the needs of all. It tends to limit rather than expand the scope of conflicting interests. Attempts to influence the other tend to be confined to processes of persuasion." -- Morton Deutch, Cooperation and Conflict
This is a really important paper for architects -- architecting across means developing cooperation and turning conflicts into creative resolutions to create desired system outcomes without undermining local interests.
Also related to collaboration:
"There is an old saying that a
problem well put is half solved. This much is obvious. What is not so obvious is
how to put a problem well." -- Churchman, Ackoff et al. 1957
7/30/11 Value Creation and Strategy
Reading Implementing a Stakeholder Strategy, I'm reminded that our strategy model has always considered the creation of value for all stakeholders in the value network. Pulling value propositions to employees and others in the value network explicitly into the value set was a key point of differentiation from Kaplan and Norton strategy maps.
7/30/11 Fractal and Emergent
"... have you demonstrated team impact, or do you want to talk about yourself? Because at the end of the day this is what really makes a difference — you will be put into a situation where you have to convince people, you have to build trust, and you have to rally people around you. So you must be able to foster very solid and robust discussions to understand what the options are, the reasons for different decisions, and you must be able to draw this out of the team."
-- Peter Löscher, president and C.E.O. of Siemens A.G., The Trust That Makes a Team Click, New York Times, 7/30/11