"It is an architects’ job to be a systems thinker, a bridge builder, a diplomat, and a change agent.  One interesting thing is that many hiring managers don’t quite understand this.  They see an architect as a senior technical leader of a small development team.  As a result, the architect can easily find him/herself in a position of little leverage.  Archimedes said, “Give me a lever long enough and a fulcrum on which to place it, and I shall move the world.”  There’s a third, unstated phrase in this quote – the position of the fulcrum.  If it is too far from the object and too close to your hands, you can’t use the lever to move anything."

Charlie Alfred, personal email, 2/13/08

The lever image is such a helpful one! We have to help our managers understand where that fulcrum needs to be placed!

2/13/08 Books and Green Things

Daniel Stroe recommended that I read Invisible Engines: How Software Platforms Drive Innovation and Transform Industries. And I will, I will. In the meantime, I've returned to "Education of a Reluctant Businessman" and I'm really enjoying Chouinard's stories, design perspective, and call to ecological consciousness.

I lift my head from the sand, and discover the world became a more frightening place while I was burying myself in the distractions of work. Madi Hirschland, of microfinance fame, is working to raise awareness locally. Indiana, you see, is fighting for top spot in the nation's worst offenders list when it comes to global warming. On the Forbes list of greenest states, Indiana was 49th, followed only by West Virginia! And Indiana is ranked 6th in per capita carbon dioxide emissions. It's not that we consumers are so much worse than others in the nation, it's more that our electricity is produced by the biggest pollution offenders—coal plants. And, apparently, we export electricity—to our greener neighbors to the west??? And we (Indiana) have no state incentives/subsidy for installing solar panels! Still, the US is second only to China in CO2 emissions. And one could argue that China's pollution is fueled by our consumerism; we didn't just outsource our manufacturing to China, we outsourced our pollution—but the impact is global. The New York Times says: "Every week to 10 days, another coal-fired power plant opens somewhere in China that is big enough to serve all the households in Dallas or San Diego." 

This whole global warming thing is a mess. Our neighbor has been researching global warming for years, from an academically serious environmental physics angle. His concern has been growing, and seeking to do the right thing, he bought a hybrid electric car. I don't know where a hybrid falls on the pollution scales, but in Indiana where coal-fired power plants are spewing carbon dioxide it makes for a conundrum. While it is hard to tease out the bottom line impact of our "carbon footprint," it is clear that we need a small car for day-to-day errands. And that's just for starters. er... I think I need to do some more work now. The "real world" is much too depressing! Ah, elephants.

Mohandas Gandhi said "Be the change that you want to see in the world." First, we have to see the world. It looks much like it did yesterday. Unless you're a polar bear. Or you just read that 1 in 8 women get breast cancer and it is the leading cause of death among women aged 35-54. And your kid has to go to Alaska to eat the fish he catches. Life's simple pleasures, and life at all, are at stake. More or less. And so forth.

This will impact us. More than just the ("do no evil") Googles and Amazons will have to look at their power hungry ways (the concern even made Dilbert—this link goes bad on 3/11). And we have to look at personal and business consumption. Right at a time when recession threatens and pulling the plug on spending could send the world into a tailspin. Conundrums a plenty!

Anyway, Chouinard and Patagonia set out to provide an example to the world, and they're not letting conundrums hold them back. Hard choices can be made, if you have principles to guide you. But take heed; the world is changing fast. I was looking at CO2 emission rankings for 2002 and China was below the bar in terms of offenders, and with rampant industrialization it is now top, and growing. And our greedy ways keep us in an embarrassing position. Obama says: 'And I will send once more a message to those yearning faces beyond our shores that says, "You matter to us. Your future is our future. And our moment is now.” '  Those are not yearning faces any more. Those are pleading faces, and we need to heed them. And Hillary has a plan to do that. Whatever your political leaning, the world will be better off if you lean on clean.

For counterpoint, see Michael Crichton's essay on Complexity Theory and Environmental Management.

2/14/08 Books and paper things

Paper (digging into architecture archeology) I speed-read yesterday (don't quiz me on it):

• Quantitative Observation and Theoretical Construction in Software Architecture, by Grady Booch

Here's a little piece I did that alludes to my approach to architecture archeology.

Books I've returned to reading this week:

• SOA Principles of Service Design, by Thomas Erl

• Let My People Go Surfing: The Education of a Reluctant Businessman, by Yvon Chouinard

Books I'm looking at buying:

• Distributed Event-Based Systems, by Gero Mühl, Ludger Fiege, Peter Pietzuch

• Event-Based Programming: Taking Events to the Limit, by Ted Faison

• Enterprise Architectures, by Michael, M Gorman

• SOA Design Patterns, by Thomas Erl (due out in May, 2008)

• SOA in Practice: The Art of Distributed System Design, by Nicolai M. Josuttis
   

• Catalyst Code: The Strategies Behind the World's Most Dynamic Companies, by David S. Evans, Richard Schmalensee

• Doing What Matters: How to Get Results That Make a Difference - The Revolutionary Old-School Approach, by James M. Kilts, John F. Manfredi, Robert Lorber

• Leadership and Self-Deception: Getting Out of the Box, Arbinger Institute

• Richard Watson’s new book Future Files: A History of the Next 50 Years (due out in April 2008)

I'm interested in system health and business intelligence, so want to ramp up more on event-based systems. Do let me know what you've found helpful in the event-driven architecture space. My interest in SOA and EA needs no comment. Catalyst Code is the follow-on to Invisible Engines. Doing What Matters is by Kraft/Nabisco/Gilette CEO Kilts. Not relevant to IT/architecture as strategic enabler? What about this:

"During my tenure at Kraft, we developed something called IMI - Integrated Marketing Intelligence - which gave us state-of-the-art insights into our brands, their categories, and consumer dynamics; how responsive they were to different forms of advertising and promotion; how much spending was optimal; and much more that will be explored later [in this book]. When we spent marketing dollars at Kraft, we were truly investing in brand building." (Page 94, Doing What Matters)

How you put together information to create business intelligence is strategically interesting. It is taking information systems to the next level; not just replacing route work by digitizing it, but using information and events to support competitive intelligence.

Books in my cart:

• Invisible Engines: How Software Platforms Drive Innovation and Transform Industries, by David S. Evans, Andrei Hagiu, and Richard Schmalensee

What am I missing???

2/14/08 Happy Valentines Day!

"The sheer wonder, the joy, of working with architects, is that I get to interact with people who shine; smart, creative, investigative, multi-dimensional people. People who lead me." moi, 10/5/06

Happy Valentines Day to you!

2/16/08 Taken Literally

Someone pointed out that there was no scaffolding in my "cartoon" of the Roman "architect" standing under the arch as the capstone is lowered in. I had to smile! Any guesses?

2/16/08 "3 Circles" Innovation Model

We developed a "3 C's" (capability, commitment and culture) model for architecture readiness assessment. Now we have another "3 circles" model (knowledge of customers, knowledge of capabilities, and knowledge of the industry) for innovation. It could have been 3 C's, but using competition instead of industry is strained and misplaces attention. The idea is to beat competitors, but not in an incremental nose-to-nose feature race for a market position, but rather by understanding value networks and flows, and opportunities to truly differentiate by delivering unique and compelling value to customers and users.

Technology-driven innovation is technical capability in search of a use. Like Segways finding a niche in airport and mall security. Market-driven approaches seek to understand the day-in-the-life of the customer, and find ways to add something the customer will value.

Each approach has worked. But if we want to make innovation more the way we do business, we need to look at putting insight into what is possible together with insight into what is desired, passion for what technology brings to the table together with passion for making a difference that customers care about.

We need to add another circle if we want to go beyond simply finding opportunities, to sifting for those that are differentiating—they make our products or services, and our business, stand out in the minds and experience of our customers.

So, who holds insight into customer/user needs? Who holds insight into our current capabilities and what new technologies make possible? And who has insight into our business positioning and direction relative to competitors and the value network? Yes, the center of expertise for each is different: marketing and business units, architects, and strategy planners and strategic management. Innovation happens when you put these together. Yet in most of our organizations, job specialization and role turf is pushing these circles apart.

What happens organically in the small, as in startups and incubators, becomes harder and harder to achieve in the large, where functions are separated and gatekeepers are placed, and place themselves, at the organizational divides that form between IT and "the business," or "marketing" and "development." The importance of cross-functional teams comes up over and over again. Difference in perspective, diversity, can have it's rough edges, but it fuels creativity.

Aside: I went looking for a Tom Peters quote on creativity (sometimes... I like ... some of... what he says... like... when he is... quoting others...) and I found this "The Birth of the IPM"—I looked for the date to see if it was an April Fool's Day post... Grin. Well, it sure puts me in context! I'm a staid product of the engineering world, after all. Creativity is all very well, as long as it serves getting real work done!

2/18/08 Don't Go Big Bang!

When we talk about creating a compelling vision, we are also careful to talk about creating a path to the vision that does not overstretch the elasticity of the organization. To illustrate incremental change and delivering value before resistance sets anchor, we have long been telling the story of Jaime Lerner's creation of the down-town pedestrian mall in Curitiba**, Brazil. You can read our version of the story in the EA as Strategic Differentiator paper (pages 20-21) that I mentioned earlier this month.

Find the value. And find a way to quickly demonstrate that you can make real on it. This is just as important to making a personal shift, like "getting past but," as it is to project visions. This is about incremental change, but it is also about being careful to take steps that deliver early value.

** If you don't have and don't want a NY Times account, google "jaime lerner new york times" and you can get to the article without signing in... Or not; the command and control thing I reserve for Ryan's dog. I did see a woman wearing a t-shirt with the following on the back "Sit! Stay!" and I wanted to ask her where she got it! There are so many ways I could use that one... ;)

2/21/08 Grady Booch's Memoir (or, should I say, Mine)

Grady Booch posted his "memoir" on the 5th anniversary of becoming very rich, or something like that. :)  It is an interesting read. Of course, he didn't mention that Fusion was the first integration of OOAD methods, including Booch and Rumbaugh's OMT but also Schlaer-Mellor, Cunningham and Wirfs-Brock. Oh yes, he was writing a personal perspective on Rational's history, not the history of OOAD.

It is an interesting history; one I'm so glad to read; thank you Mr. Booch. That said, I just wouldn't want the industry to forget that other factors influenced Rational's move to integrate and then lead the push to standardize OOAD notation—namely the proliferation of OOAD methods and their idiosyncratic syntaxes (hard for a tool vendor to support such an explosion, and the explosion fractured the market). I have to say (with great respect) it was a strategically masterful move, one that should be written about in strategy texts dealing with market leadership through standards leadership. We were creating Team Fusion, the next generation of Fusion that was to do two main things: address architecture and pull in Todd Cotton's work (inspired by Tom Gilb's Evo) on Evo-Fusion or evolutionary development (a precursor of agile, and as I understand it, an influencer of Scrum). We decided to throw our hat in with the UML standardization effort, and Martin Griss and Reed Letsinger were HP's key men on that one. Martin started to collaborate with Ivar Jacobson, and that led to their book on reuse. I continued to focus on architecture, but no longer exclusively architecture for object-oriented systems. Anyway, Booch and Rational's leadership was a clear catalyst to the UML effort. Still, the community got behind it, and it is good to remember that community support, including the collaboration of other leaders in the OOAD space, was also critical to UML being developed and embraced as a standard. 

I do want to say that I have had very strong respect for Grady Booch from those days working on Team Fusion. I gave presentations on Fusion at conferences and at OO shindigs, and had to explain both what made Booch's contribution great, and what made Fusion an advance on what had been done by Booch, Rumbaugh and others. One has to pay careful attention to do that; from Booch I learned so much, and that positioned me well to learn still more from Fusion. It set out to be a true method supporting the decision lifecycle from analysis through design. For example, in Fusion, designing behavior is integral to designing structure. We have learned a lot since Fusion. But Fusion, as far as I'm concerned, was a big milestone on the OOAD path, and I would not want it to be glossed over by the tellers of OOAD history. Of course, my opinion here is not unsullied by vested interest. I owe so much to Derek Coleman, the "father of Fusion," and respect him greatly. He was a demanding task master, and brilliant. Being British, compliments make Derek excruciatingly uncomfortable—even when they're being paid to someone else!!! :) 

Well, I'm happy Booch used the opportunity to tell the story, for it helps reveal more of his role and contributions to Rational as well as OOAD. And it gave me an opportunity to say I think Derek Coleman and Fusion should also hold a watershed place in OOAD history. This doesn't at all diminish the role of the pioneers who launched OOAD, notably Grady Booch.

Moreover, I have been privately lamenting that Grady's early blog years left a much richer trace of his thinking and exploration, and I miss that. Grady figures among the most important leaders in our field for good reason, and for all that, he remains a nice guy with a sense of humor.

2/21/08 Architecture Documentation: Educate by Providing Value (lighthearted for good health)

I mentioned I've been reading Let my people go surfing (which I like to refer to by its subtitle, because for most of us engineers-turned-architects it is the subtitle to our personal journeys: The education of a reluctant businessman). Well, many things have struck me which I still need to note in this journal for preservation of my thought. But this is the part that pertains to my present train of thought: Early on, Patagonia established a practice of educating it's customer community with essays in their catalog. So, they educated customers on the damage that pitons were doing to rock faces. Yes, they went after their own core business, deliberately cannibalized it, and educated their customers on their disruptive innovation, all via a 15 page essay in their catalog. Now, if you think software engineers don't read, try mountain climbers! Yes, like any community, there are some who read and some who read... let's just say... less. But an essay in a mail-order product catalog. Hmmm. They did this again with layers. They invented an undershirt that wicks away moisture (modeling diapers), and had to educate their customers on the value proposition of layering. They treated their customers like smart people who can and do read when it is of personal value to them. Hey, what a concept!! Shall I say that again: let's treat our customers like smart people who can and do read when it is of personal value to them! Of course, our customers are software engineers, so, glory be, they are smart people (and many are climbers, at that; extreme sports and extreme programming seem to go hand in glove)! The heart of the matter isn't getting them to read. The heart of the matter is writing what will get them to read.

Now, I've been reading a client's (proprietary) architecture document. It is to their company what Booch's Handbook will be to the world, and more. Very impressive!! Still, is it for every business person in that company? Good gracious no! Is it for every software engineer? No again; at least not in absolute terms. But parts of it are. Is it for every architect in that company (actually, it's a division of a company, but a fairly big division). Absolutely, in absolute terms. Every architect should know at that level every other architecture in the division. This is not frequently practiced, because too often architectures are implicit in the code and the architects can't afford to go spelunking in more than one codified architecture. But imagine if all the architectures in your division were documented and in one spot (where the owner architects have to keep it up-to-date); wouldn't that be empowering? Enable synergies and reuse, facilitate alignment and consistency, leverage learning across projects, and so on.

Different views need to be crafted for different stakeholder communities, and for strategically influential stakeholders, views should be tailor-made for them. And there's a standard for that; that would be IEEE 1471, the "standard" for architectural description. It essentially says architectures have various stakeholders who have concerns and architectural views should be created to address stakeholder concerns. 

We have to remember that we engage differently with the written word than with the spoken word, and this makes the written word very powerful. Once we have chosen to read words, they run in our heads like our thoughts. We can challenge them in an internal dialog, pass them through the test of our own experience and reasoning, without losing our place. And if they wax too flowery, we can disengage. So, enough of that.

Bottom line—write about your architecture, but make it interesting and valuable to the intended readers of the document you are writing. If you explain some crucial aspect of your architecture to the engineers it will impact, adding to the mix the considerations you took into account, the driving forces, your insights and experience that made your choices make sense, odds are they'll find it interesting and valuable. But, I'm betting on your experience and your ability to communicate it in diagrams and written words.

And, because unintended use may be made of your document, be explicit what the intended use is. Otherwise unanticipated "quality requirements" will be levied against your document, and it may come under fire only because it is being used in a way you didn't scope for.

Now, to be clear, this journal is first and foremost for me. I am my own harshest critic, and I don't wish to be challenged from that rank. And then this journal is for a small and very select set of architects who are extraordinarily smart, who have an intelligent sense of humor that catches the many layers of satire, irony and wit that permeate my writing, and who are so comfortable with their own rare and outstanding qualities that they can stomach my occasional indulgence in self-congratulation. Ok, quickie quiz: just what part of this is satire and irony and what part wit?

Ok, that's how not to do your document scoping! This we call leading by counter-example™.

Uh, I think I'd better go now.  Bills to pay. Life to lead. That's me I'm talking about, though clearly you're procrastinating too.

I was joking with the trademark thing (me, joke? no!), and then I thought I'd better check in case I was infringing, and phew, faith restored in humanity, leading by counter-example™ is not trademarked! But (sinister laugh) now I have dibs on it... ;)  Bills! That must be it. I thought it was Scott Andersen's bad influence. Really Scott, humor in the middle of the day?  At work at that! Tsk!  Tsk!   Then again, thanks to Scott, Dana is in less danger of a heart attach today. Hopefully, I've done my part for you. But, only if you're in the small and select set... you know, awesomely smart, great sense of humor, and ... you have a certain amount of tolerance for my quirkiness...

2/22/08 A Note on Minimalism and Documentation

Ryan is the master of minimalist or parsimonious writing. Of course, in this journal I'm on the other extreme and I recognize that parsimony is hard work. Ryan labors at his strictly elemental reports, with not a single superfluous word. He can summarize an entire novel in half a page, including every important element of the plot and character development. I takes him twice as long to write his book reports as it does for the others in his class who write 10 pages or more. And then, are they interesting? Well, only as an intellectual exercise in assessing accuracy and completeness...

There is a difference between minimalist architecture (keeping the decision set minimal, and the blueprint elemental) and doing what it takes to breathe the architecture to life in the minds of the development community, and to keep its sponsors fully on board as reality throws fake feeds at the scrum. The management team may need sound bites with a liberal sprinkling of "innovation," "excellence in strategy execution,"  "business alignment" and "attract the best talent." (No, I'm not leaking your competitive strategy, I'm leaking everyone's competitive strategy!!!  Hmmmm! Oh yes, context matters!) But the point is, if our system is strategic so that successfully getting hearts and minds engaged is crucial, we need to think about our audience and deliver our architecture to them through different routes. Participation is one. And interesting reading that delivers value to the reader is another. See the Outcomes and Deliverables section (starts on page 5) of our Meta-Architecture chapter.

Minimalist architecture papers:

• Scott Andersen, "Difference Architectures," 2006

• Ruth Malan and Dana Bredemeyer, "Less is More with Minimalist Architecture," IEEE's IT Professional, September/October 2002

2/22/08 Architect Jobs

I just added two more architect job listings to the Bredemeyer Resources for Architects site (so in the last week I've added listings for a software architect, senior data architect, senior enterprise systems architect and enterprise architect). I think this is the tip of the iceberg, as we keep hearing about internal development and hiring plans that reflect a growing demand for architects, and a growing recognition that architects play a role that is key to value delivery and risk management.

In areas where capital equipment is invested in, there is a notion that these investments have a lifespan, or alternately put, the equipment becomes out-of-date and even if well maintained, has to be replaced. Our software systems, too, have to be modernized. Some approach this modernization by carving up old systems and giving the chunks a new coat of integration paint. This is all very well for systems that are not central to our differentiation strategy. And it may even be just fine as a migration strategy for core systems, where any change is risk. But we do have to think very carefully about where our strategic systems need to be 5 years out, and have an investment plan and technology roadmap to get us there. Craig Jordan talks about the need to envisage the mosaic, when you're creating the bits of tile—the A in SOA, that must come first, if we want architecture not accident.

Yes, a growing demand for architects, so we can chart new systems and bring old ones into the future. A wave of expertise will be leaving our industry as those who pioneered it take their bows. At the same, we stretch our complexity boundaries in ever more ways, raising the demand for innovators and complexity handlers. There will be more opportunities for technical specialists or "structural engineers," and there will be more need for the architects who create the "big picture," the design that allows all the pieces to work seamlessly together to deliver differentiating value in all its dimensions, from function to quality to life-cycle cost.

2/22/08 Einstein versus Zen Master

Einstein said:

"Perfection of means and confusion of ends seem to characterize our age.”

 

Yvon Chouinard said: "I've been a student of Zen philosophy for most of my life. In the practice of Zen archery, you forget about trying to achieve the goal—which is hitting the bull's-eye—and instead focus on all the different movements of shooting an arrow. You practice your stance, reaching back and smoothly pulling an arrow out of the quiver, notching it on the string, controlling your breathing, and letting the arrow release itself. If you've perfected all the elements, you can't help but hit the center of the target. The same philosophy is true for climbing mountains. If you focus on the process of climbing, you'll end up on the summit. As it turns out, the perfect place I've found to apply this Zen philosophy is the business world." p74-75, Yvon Chouinard, Let my people go surfing, 2007

Nature's Zen Art, photo credit: Dana Bredemeyer

That sounds like perfection of means to me. But it is perfection of means without confusion of ends.

When an architect we'd worked with took on a chief architect position within the same company, but a new business area for him, he said he was comfortable because he knew VAP would carry him through. He'd used VAP on previous projects, so was confident that if he went through the VAP movements he'd practiced before, he'd nail the target. And he did. Because he let the process lead him, he got the target clear, and then he let the process line him up to hit the target.

It is pretty neat, being more than a decade into this, because we hear more and more of these stories from architects who feel they owe us a nod. Even so,

'I have to agree with David Christiansen: "I’m not convinced there is such a thing as a methodology or process that can produce good architecture." People create great architecture, just like it is people that create great software systems.'      moi, 8/12/07

That is to say, VAP didn't create great architects, but great architects are saying that VAP helped them reach their targets: good, right and successful architectures. A good solution: technically sound. The right solution (product, application, system, etc.): it meets stakeholder needs, where the voice of the customer, the voice of the business, and the voice of the development team, are taken into account. And it is successful, in that it is used and is delivering business value. We often get so heads-down-focused on doing the product right (or at all), that we forget to question whether it is the right product. We can't birth or evolve software solutions without writing (lots of) code and we get so focused on code as the holy grail, that we overlook the huge, huge waste in our industry because we forget that getting the right solution concept takes work; needs talent and time to incubate.

We can argue back and forth—for example, one might argue that we need to fail sometimes to create the kinds of gains that we see coming from software. Or we might argue that what we count as failure, isn't really. It was simply failure to estimate correctly. Or failure to judge the technology gestation rate. Or some such failure to establish the right success criteria. But to me, where our failure costs most sorely is in human terms. In terms of the software developers who throw their hearts, brains and serious years of their lives into making something, only to have it be cancelled before it stands the market survival test.  

Put me on a rock face and all the process in the world won't get me to the summit. A hiking trail, yes. A vertical drop? Never! I leave adrenalin sports to my brothers. I got the cautious genes, but more than that, I don't want to hurt the people I'd leave behind. But hey, it makes me really good at risk management, for it is not risk that I don't like, only what would be for me unconscionable risk—the measures for which are context dependent, and often personal.

2/24/08 Update on Failed Projects

In May 2006, I collected together links on software failures as well as some links on learning from failure (see Project Wipe-out).

Now, if I was doing research on failed projects, I would want to separate those that are cancelled early from those that are cancelled late, from those that are a market failure. Those would be interesting and relevant statistics, for it would better help us track the kind of failures we want to invest in, from the kind of failures that create waste. We want to fail fast and cheap; if we're innovating we need to stretch and have a tolerance for exploration and course resets. We don't want to fail fast and cheap all the time; but enough to find the solutions that will win out in the market.

I would also want to track not just how many projects are late and over-budget, but how well they do, once they hit the market. I've been close to enough of these "late" projects to know that the market is the ultimate judge of a project's success. Being late hits team morale like an avalanche that is hard to dig out from, and may require digging deeper into the investors pocket book than they may be willing or able to go. But if they make it past these challenges, how often do they meet with market success? Does project lateness correlate with failure in the market?  

But, I'm not running those research projects. Here's an update to what's "out there:"

One remarkable thing is that there are many out there still using the 1994 Chaos Report statistics, in which it was reported that some 31% of software projects are cancelled, as opposed to the 2005 Standish Group report which apparently found that 23% of projects were cancelled (I don't have access to the 2005 report, but these stats are reported in Why Software Projects Tend to Fail by Cohen Oren, 2008). Did we get more tolerant and give projects more leeway on schedule and budget, did we get better at not starting doomed projects, or did we get better at establishing and hitting the target?

2/24/08 Requirements as Invention

Anyway, the big take-away is that Einstein is still right. Software projects too often fail because there is a confusion of ends! And our pursuit of perfection of means too often detracts from establishing the ends. A perfectly constructed system is only a perfect system if it delivers sufficient value to its stakeholders.

Einstein also said:

"If I had 20 days to solve a problem, I would take 19 days to define it."

No, I don't see that as an argument for waterfall! I see it as an argument for applying agile principles to figuring out what the problem is, but I don't believe we need to use (only) code to do that. Building architects use drawings, 3D models, and simulations to convey the design to the stakeholders so the stakeholders can perform thought experiments to see if the design meets their needs. We need to do more of that.

Requirements don't exist out there in some extant state that just has to be captured—like cattle that just have to be corralled. Requirements are an invention that come from an interaction between current need, future (invented) capability, and future need that springs from the interplay between that capability and the new world it creates. So, yes, we need to involve the architect in coming up with models and talking stakeholders through the thought experiments that will help them figure out what their problem will be, when the system is in place; that is, what their real requirements are for the fielded system! (If I think of replacing a paper-based forms system with an online forms systems, and do not think about the immediate next things my customers will want the minute online forms are in place, I have done them and my development team the disservice of leaving everyone less than satisfied. After all, we know, the moment we think about it, the next thing they'll expect is integration with some other system to turn that data into business intelligence. And what else?)

To make it more clear why our status quo approaches to requirements and architecture lead to failures, in early "architecture appreciation" courses for managers, I developed these two slides:

Ok, so that's making a point by over-emphasis; still, even our best used-patterns salesman uses that tactic. :)  But, yes, too little requirements and too little architecture leads to a baroque concoction of best guesses and interpolation.

Links to material that didn't make my earlier list:

• Rick Hower maintains a list of computer system failures due to bugs

• The sorry state of our software industry, Mitch Barnett, July 27, 2005

• Why Do So Many Software Projects Fail so Dreadfully?, July 2004

• Major causes of software project failures, by Lorin May, CrossTalk, 1998

• Software Runaways: Lessons Learned from Massive Software Project Failures, by Robert L. Glass, 1998

2/24/08 Kiva Widget

Something along the lines of HelpMatch, but here and helping people in need today:

This is similar to the Heifer widget that you can plop on your blog, and is really neat! Kiva is a microfinance organization that connects lenders of small loans to individuals, to help people help themselves out of poverty. Now, shouldn't we be doing this for informal, person-to-person help networks that enable goods (new and used) and services to get from people willing to help to where they are needed?

And I hope you'll be inspired to join the Kiva loan network—do look at the impact ticker that flashes on the left column! See, for example, the story of Danson Ng'ang'a—read all the way to the bottom, for an update in this time of political crisis in Kenya. Good and evil still duke it out in this world, even while we bury our heads in work. But it is heart-warming to see how rapidly innovation and "technology" (online social networks) is being applied to helping the needy.

Another site that is doing something along the lines of HelpMatch-y ideas is Give2Network. It makes an important contribution, though it still focuses on giving help to organizations:

"The team at Give2Network wants to enable people to contribute to their communities. We want to do our part in giving back by giving away free and easy-to-use tools that schools and non-profits normally can’t afford. We recognize that much volunteering revolves around fundraising (selling and buying needless products) and time-consuming tasks such as sending emails, soliciting sign-ups, and communicating with supporters. Give2Network aims to simplify those tasks." Give2Network

It is a good place to look for analogies. DonorsChoose.org is similar to Give2network and Kiva. It is focused on helping public school projects. There are more and more sources of inspiration and ideas.

Ready to help build HelpMatch? Money is much easier to move than goods, but helping with stuff still seems to be on the table.

2/25/08 Architects and Innovation

In the discussion around whether or not architects still write code, the point is often made that architects need to investigate new technologies, and that is where they still keep their hand in the coding game. Bill Barr makes salient points, but I would want to caution that innovation is not just about better means; that is, not just about developer productivity. Naturally, we couldn't be creating the complex systems we create today using Assembler! One might argue that without innovative development technologies, the iTouch/iPhone couldn't have delivered such innovative capabilities. But innovation in software development isn't just about retooling or modernizing software development, deciding whether to hibernate until spring, or make some other framework or technology selection. It is first and foremost about innovating to create products, applications or services that delight customers. But it is also about keeping talented, passionate developers in high gear, and (carefully selected) new technologies generally speaking help boost productivity in of themselves (after the learning curve is scaled), but they also help attract and/or keep tech-star developers.   

That said, those in the architect role, especially architects with broad scope of influence and accountability, may find themselves squeezed on time to code. Chris Kempster, in the comments to Bill's post, goes out on a limb that would come down in flames if Bill's blog was as hot as TheServerSide:

"The idea of IT architects coding is a joke, it's akin to building architects laying bricks! Rather than spending time talking with clients and understanding business issues, laying a foundation with your technical staff to collaborate, share ideas and capture enthusiasm... You don't need to code to show leadership and span the business and technology divide." Chris Kempster, comment.

I would agree with Bill Barr, that for project architects and technical leads, being credible comes from being able to dive in and explain by example—with an example, and setting an example. But I also agree with Bill and Chris, that creating the right product requires the architect's bandwidth, and this competes for attention with creating the product right. Naturally, we don't want to go wrong on either—we want right products with right capabilities and properties, and right properties is a function of both creating the product right (structurally sound, correct, and reliable) and having the right target (SMART property requirements).

And since "right" products or applications don't usually simply exist out there in users minds, just waiting to be captured and coded up, we need architects to be working actively to understand what technology makes possible for stakeholders. For developers and how they build and assemble systems, yes, that too. But not only that. Architects need to actively, creatively, explore how technology capabilities could be brought to bear so that, for our customers, our products:

"...make their lives the way they wish they were."

Yes, this is the quote Joachimsthaler (Hidden in Plain Sight) uses from the inside front cover of the J. Peterman catalog. I never knew that the way I wished things were had anything to do with an iTouch, but now that it's here, I wish my interactions with my iPAQ were that way! How we want our lives, and the things and experiences in it, to be, is constantly being reshaped, and it takes innovative thinking not just in the present, but in the future world that our application or product will exist in, to really find our opportunity to differentiate—to be "right."

If we ask the architect to be the design authority, and if we really mean design authority, we need to give the architect the charter and the bandwidth to be doing the things that allow innovative ideas to spark. Remember, "Eureka's" that may, in a flash, gel a sudden innovative idea, take hard work, exploration, and ... many showers, in my case; bath's in Archimedes case.

So, what things should we be doing to allow innovative ideas to spark?

• Yes, what Chris Kempster was saying—get out there and talk with customers (clients, users, etc.) and understand their business issues and their dreams, hopes, aspirations, passions, sources of joy and delight, and concerns, fears, aversions, frustrations or hassles—focusing on goals, not just actions and activities. For ideally, many of their actions and activities will be obsolete, not just automated, if we do our part right.

Booz Allen found that the companies that get the [highest return on their R&D investments] are the ones that integrate customers and users deeply in every stage of the innovation process.  These successful innovators involve customers in idea generation, in selecting the best ideas, and in refining and elaborating ideas.

Keith Sawyer, "How Customers Drive Innovation," Creativity and Innovation blog, December 14, 2007

• Look to our innovative customers to find what they're already doing to "...make their lives the way they wish they were." This anecdote makes the key point that Patricia Seybold (Outside Innovation) takes a book to make:

• Understand the demand ecosystem (every time I use "ecosystem," I think of David Ing's post), à la Hidden in Plain Sight. Remember that we bring a technologist's perspective to bear on understanding the ecosystem. This is not to say we're running around techno-pontificating, arguing that MySQL is just the thing for the backend, or whatever our current wicket is. Rather, we see where technology fits in the daily experience of our customers; what it does make possible and what it could make possible.

• Create a Competitive Landscape which gives a big-picture view of business context, including the demand ecosystem. It is a good place to start bringing the perspectives of the multidisciplinary product concept (or at a higher level, business strategy) team together on the same page. It provides for structured brainstorming, and when the participants are selected for diversity of perspective, can really stimulate the creative foment that produces new ideas.

• Use multi-functional teams, with diversity of style and perspective. Innovation is not, should not be, the exclusive province of marketing, or business analysts, or architects, or any one discipline. Ten Faces of Innovation makes this point. And we've been making it with the circles of innovation model. Though of course, like the circles that ripple out from a pebble thrown into a pond, each of these circles speaks to other circles—customers and those who understand them, technologists and those who have foresight into technology ripples well before others do, and value partners and competitors, and those who understand forces shaping the marketplace.

• Read for other ideas on how to get ideas, and how to get ideas to pay off—for example, here's a good one: 7 Reasons Why No One Likes Your Ideas.

 

CIOs, CTOs, COOs, even manufacturing VP's are getting with the idea that technology is a key ingredient in innovation-driven business success:

"Management's most crucial investments for factories of the future won't be the hardware or the software, but in the understanding of how new technologies provide new options for powering business success."

John Teresko, Finding the Next Big Thing, IndustryWeek, March 1, 2008

Yes, apparently John is writing this from the future!

2/25/08 Innovation Blogs

• Sanjay Dalal, The Innovation Index

• Andrew Hargadon, Harga-Blog: a conversation about technology, design, and creativity

• Roger von Oech, Creative Think blog

• Endless Innovation blog

2/25/08 Coupling

Here's a neat story about tight coupling (not software, but I expect you'll be able to translate):