A Trace in the Sand

by Ruth Malan


Architects Architecting Architecture  

November 2015


Design Visualization: Smoke and Mirrors

These are the slides from my Design Visualization: Smoke and Mirrors (90 minute) talk at the Software Architect Conference in London on October 14, 2015.

I am working on the notes ( Part I is complete, along with the first sections of Part II, and some notes in the middle and end, to illustrate what's ahead). The notes convey the spirit of the talk, but while rehearsed, I didn't follow a script. (I didn't stand where I could see my speaker's notes, because that would have put me behind an obstruction and way out of contact with the audience; tradeoffs.) Anyway, the talking points are, for the most part, close enough to what I said to convey the spirit of the thing. Well, I did take some artistic license here and there, expanding some of the points. Just a smidge. ;-)

11/29/15: Well, the slides are unchanged (except for a minor fix), but it's been long enough that the notes are more along the lines of "what do I imagine I could possibly have wanted to say about that slide?" I mean, it was like 6 weeks ago...

This page may take a while to load (due to the slide images), but it's totally worth it, obviously. To make it a bit easier to navigate:

I experimented with moving Part I into a (Duarte-styled) SlideDoc, so we can see if that's useful:

Thanks to everyone who has retweeted!

Part I: Introduction and Software Architecture


These are the slides from my Design Visualization: Smoke and Mirrors (90 minute) talk at the Software Architect Conference in London on October 14, 2015.

What I tackled, was a rethinking of software architecture, its place in software design, and how we approach it. And I courageously went out there and did something few (in software) do -- I went to bat for wisdom!





  • Introduction
  • Theme of talk
  • Outline of talk
Design Visualization 1

This is the part where I tell you "This flight is going to Indiana by way of South Africa [addressing the question of my accent obliquely]. If you're on the wrong flight, this would be a good time to attend one of the awesome talks down the hall."

My name is Ruth Malan -- I don't know how to pronounce it either, so any way you say it is fine. 

Well, as long as you're staying, let me tell you that Dana Bredemeyer says "There is a silver bullet in software engineering, and it is relationships of goodwill and a commitment to objectivity." There's a catch though. Those relationships take effort and attention to build and to sustain and grow.

So here I am, relying on you to extend a little goodwill on credit to me, and as we proceed, I hope I'll earn a little more, as we talk about some ways we can work towards objectivity. [Aside: we never reach objectivity; too human, too complex; but we're making tradeoffs, and we can strive to make them more objective/balanced/less self-interested/etc.]


The M.C. Escher Company holds and protects their copyrights to M.C. Escher's work quite tenaciously. Not being able to reproduce Day and Night for you here, presents a little opportunity to demonstrate a point at the heart of this talk.

Day and Night is one of Escher's more famous works, so you have likely seen it and I'd like you to call it again to mind. Do you see it in your mind's eye? It is the one where there is a town in the light, and a mirror image in the dark of night. A river runs by, in the light, and the night. There are fields that become abstractions that become duck (or geese), flying <gestures a mesh with interwoven fingers of both hands> from the light, others in mirror image out of the night.

When you called it to mind, perhaps at first you were thinking of another Escher, or perhaps it struck you differently and you remembered different features. And that's the point, really. We don't know if what I call to mind, and see in my mind's eye is quite what you see, even if the original was the same image. Our software systems are so large, we can never see them in their entirety. Our mental models are central. But idiosyncratic and fallible.

Another point of salience in this context, is that it conveys a movement, a fluid interplay, between the concrete and abstract, between what is and what is mirrored.

And that is where we are headed -- to talk about design as intention, and reflection of design as realized, and how we express and envision our designs. But first I would like to revisit how we think of software architecture, and the role visual expression plays in design.


Software Architecture section outline:

  • Intention and Reflection
  • Software Architecture
  • Decisions
  • Design
  • Significance
  • Technical Wisdom
  • Structurally and Strategically Significant
  • Minimalist
  • Intentional and Emergent

I playfully subtitled the talk "Smoke and Mirrors" -- for it can seem like we practice magic and sleights of hand when we don't expressly communicate our design intention, or reflect on the design as realized, in order to iterate on and improve our understanding of what we have, and intend.


Here we have the definitions of intention and reflection from Ambrose Bierce's Devil's Dictionary. If you're not familiar with the Devil's Dictionary, it gives definitions a satirically cynical side-eye, helping us understand something better by looking at the (negative) space left by contact of the concept with reality, and what that defines. Intention, then, is what we get, despite our intention -- an involuntary act. It is a big wink at our bias and foible and self delusions and intellectual aggrandizements.

As for reflection: "An action of the mind whereby we obtain a clearer view of our relation to the things of yesterday and are able to avoid the perils that we shall not again encounter." That so well captures the weltanschauung of our day and field -- we forget, of course, that it was Heraclitis who said "Change is the only constant."

Now, when it comes to definitions, you might recall that Richard Feynman said: 'We can't define anything precisely. If we attempt to, we get into that paralysis of thought that comes to philosophers… one saying to the other: “you don't know what you are talking about!”. The second one says: “what do you mean by talking? What do you mean by you? What do you mean by know?”'

Undeterred, we're going to take a look at the definition of software architecture.

This is the definition from the touchstone of our era, wikipedia.

When I see a definition, especially in a context like this, I tend to see it as words words words.. words words. So I helped a little to lead attention from software architecture to system.

What do we mean by system?


Dana Meadows characterized a system as "an interconnected set of elements that is coherently organized in a way that achieves something."


Returning to Paul Clements and colleagues definition on wikipedia, we note "high level structures" -- comprising elements and relations, along with properties. High level?

Grady Booch memorably observed "All architecture is design, but not all design is architecture. Architecture represents the significant design decisions"


We know what to do with decisions! We'll name them, describe them, identify what problem the decision addresses and the forces we're weighing to harmonize, resolve and balance. We'll keep track of assumptions we're making, explicitly identifying what in the context we presume to be stable and what to watch. We'll connect the dots to the business rationale and technical goals we're trying to achieve, or the hard-(l)earned scars of experience we're seeking to avoid this time round. We'll outline alternatives we considered but ruled out, so we don't have to revisit those arguments again and again. And we'll note implications so we -- our teams and those we collaborate with -- can be prepared. Then we'll consider what further requirements are implied by, or derive from this decision we're making and documenting. And identify related decisions we need to make next...

Decisions are central, and it is a great template, but you can just hear the captain in the cockpit yelling "pull up, pull up" -- we'll run into a veritable forest of decision trees if we speed too far too fast down that runway just now.

Which decisions? Design decisions. Design!
Herbert Simon, according to Jabe Bloom, is "ground zero in design discourse." And this is quintessential Herb: "Everyone designs who devises courses of action aimed at changing existing situations into preferred ones."
A variation on the theme: "The engineer, and more generally the designer, is concerned with how things ought to be - how they ought to be in order to attain goals, and to function."
And that is just one of those so precious moments when the world is full of simple wonder again -- we design to get more what we want.
Returning to the eminent Mr. Booch: "Architecture represents the significant design decisions"

"What decisions does the software architect make?"

The architecturally significant ones.

"What is architecturally significant?"

The architect decides!

The architect decides. That sounds like a tautology, but it really is the crux of the matter.

Is this the part where we get to talk about technical wisdom?

No, this is the part where we get to say "Awww"

Just kidding.

Technical wisdom factors. Factors in and factors out. What is shapingly crucial?

Ok, so if architecture is a set of decisions, but not all decisions, and we're asking which belong in the architecture, we're looking for those that shape, that

  • give form to the system
  • set direction
  • constrain
  • bring integrity and consistency

And those that have the highest cost of change. Decisions which would incur substantial

  • attemtion and time, and other resources
  • operational downtime
  • deferral of value
  • emotions and otherr personal impact, and politics

to change or revert and rework, are architecturally significant.

Also, if a decision meaningfully reduces the cost of making changes, enabling our business to be more fleet, adapting and extending its services or product set as the market shifts, it is architecturally significant. Hence, for example, reversibility of decisions and continuous deployment is part of the consideration space for architecturally significant decisions.

Those are important insights, but I would like to add: architecture decisions are structurally significant. They deal with the organizing structure of the system and the design of architecturally significant mechanisms to yield desired system outcomes, including system properties, while addressing inherent challenges. Structurally significant. You know, make or break.

And strategically significant. "Software is eating the world" (Marc Andreessen). Software enables. And more, across industries, software is increasingly a source, if not the source, of differentiation. It is game shaping and game changing.


Software systems are in place -- for 3 months, 3 years, 10 years. 20. Because they enable something our business depends on. But, increasingly, and more as systems age and the architecture erodes under the weight of accommodations and agglomerations, these systems constrain the business, impede agility and adaptability and responsiveness to an ever shifting context.

And architecture, the critical decisions that hold the system up and tie it down, in turn enables and constrains the code (and those who write it).

Architecture decisions create the context for further decisions, reducing -- cleaving -- the decision space. This is good. It reduces the overload of overwhelming ambiguity and uncertainty, creating "ground under the feet" (Dana Bredemeyer) that we can move forward on. Critical decisions take time to make attentively and can be fraught with downstream consequences if made inattentively and without foresight. And this is bad, if it takes decisions away, reduces empowerment or degrees of freedom and motivation, where it matters. So we seek to keep our architecture decisions, in Dana Bredemeyer's urging, to a minimal set.

To reach a helpful notion of what, then, we consider architecturally significant, Dana points to Daniel Day-Lew--er(r) Abraham Lincoln:

"The legitimate object of government is to do for a community of people whatever they need to have done, but cannot do at all, or cannot so well do, for themselves, in their separate and individual capacities. In all that the people can individually do as well for themselves, government ought not to interfere."

Architecture decisions are those that impact system outcomes. That is, outcomes that can't be ascribed to locally-scoped parts of the system. Intentional architecture decisions are those, from a standpoint of experience, we deem must be made from a system perspective, to get more what we want from the system. More than we would get, if we left the decision to be made at a more narrow scope with only local information about the forces that impinge upon, and outcomes that are contingent on, the decision.

That is, architecture decisions are those that need to be made across boundaries -- the system boundary, and boundaries within the system, in order to achieve desired system outcomes -- to meet system goals with more the system properties we want. Properties like usability or performance that the user cares about. Properties like resilience and scalability that the ops team cares about. Properties like understandability and adaptability that the dev team cares about (or will, as these properties are compromised). Properties that emerge from interactions and relations among elements, rather than localized concerns.

Architects need to take a broader perspective -- across the system, across stakeholders, and a longer time horizon. But we do just enough, to achieve concert and coherence, structural and design integrity, to get more the outcomes we seek.

Of course, despite our best intentions, some implicit decisions will prove to be architecturally significant. The point is, do we want to leave matters of system integrity and strategic import to accident, or do we want to bring what we can to bear, to get more what we want?




Part II: Visual Design

Visual Design ToC

Visual Design section outline:

  • Sketching in Engineering: Leonardo da Vinci
  • Visualization in Other Fields
  • What’s Missing from Code
  • Why We Model
  • A Change in Perspective
  • Mechanisms
  • Wisdom and Fallibility
  • Extraordinary Moment Principle


When I googled to find an image of Grady for the previous slide, this was one of the results. It's on an IBM site related to a conference, and next to the image of Da Vinci and Booch there is the playful caption "separated at birth?" The relationship, at least in my view, has to do visual design, for Grady Booch is one of the fathers of visual design in software. While Leonardo da Vinci is one of the fathers of visual design in engineering.

Anyway, I thought the image was a neat serendipity because when I was collaborating with Grady Booch on software visualization several years ago, Grady introduced me to this book: Engineering and the Mind's Eye. Which in turn introduced me to Da Vinci as engineer. Of course, I was familiar with Da Vinci's work as artist -- even privileged to see his cartoon (as a full-size preparatory study for a painting is known) of the Virgin and Child (with St Anne and St John the Baptist) in the National Gallery. I'd seen his sketches of imaginative flying machines and was aware of his notebooks.

But I hadn't explored the extent, manner and contribution of his work in engineering.

Now this Renaissance man would be inspiring in any event, but additionally so in this context, for he used sketches to investigate, to find out, to create visual "demonstrations" that teach.

He puzzled things out, to astonishing effect. Da Vinci foreshadowed Copernicus by 40 years, declaring "IL SOLE NO SI MUOVE" ("The sun does not move.") And adding, "The earth is not in the center of the circle of the sun, nor in the center of the universe." Further, 200 years before Newton, he wrote "Every weight tends to fall towards the center by the shortest possible way." He was likewise prescient in other fields. 400 years before Darwin, he placed man in the same broad category as apes.[Gimpao Ni Ei Suuh]

We may know him as a painter and sculptor, but he was esteemed in his day not just for art—holding official positions as engineer including "painter and engineer to the Duke", and “First painter, architect, and engineer to the King.”

Leonardo was intrigued by problems like inertia, friction and resistance, and investigated mechanisms and mechanical elements such as screws and gears, pulleys and weights, with a piqued visual curiosity. Though his scientific notes are articulate and precise, for Leonardo, illustration took precedence—the drawing does not illustrate the text; rather, the text serves the illustration, only adding detail best expressed in words. To this end, he developed principles of graphic representation—stylization, patterns, and diagrams.

Indeed, he developed and practiced a rather modern form of cognition—studying what was known from masters current and past (his library was extensive), but extending that study by "knowing how to see" (saper vedere) with scientific inquiry augmented and abetted by technology—that of pen/pencil.

He and other engineers of the day would study each other's works.

And they copied designs from each other's notebooks (Engineering and the Mind's Eye).

Leonardo's notebooks capture and extend understanding of phenomenon of nature and machine, and include numerous inventions, some built in his time, others later -- such as the lens grinding machine pictured.

These inventions, we might note, were constructed as sketch-prototypes so compellingly drawn as to both persuade feasibility in many cases, and to inform construction. (The Archimedes steam cannon is a fun story).

Leonardo's notebooks stand testimony to sketching as a means to

  • observe more closely
  • study, think, reason
  • record, not just to persist, but to extend the range of cognition -- one's own, and also to communicate with and teach others
  • invent, by making novel connections
  • test ideas on paper
  • persuade

Leonardo da Vinci had studied human anatomy with an emphasis on "knowing how to see" -- especially with the aid of a pencil or pen.

This is from the Encyclopedia Britannica, which I reproduce here [but didn't include in the talk] because the language is telling:

"Leonardo combined anatomical with physiological research. From observing the static structure of the body, Leonardo proceeded to study the role of individual parts of the body in mechanical activity. This led him finally to the study of the internal organs; among them he probed most deeply into the brain, heart, and lungs as the “motors” of the senses and of life."

And from Martin Clayton's book on Da Vinci's anatomy work:

"First of all, as a sculptor, engineer, architect, he had an intuitive understanding of form — when he dissected a body, he could understand in a very fluid way how the different parts of the body fit together, worked together. And then, having made that understanding, as a supreme draftsman, he was able to record his observations and discoveries in drawings of such lucidity, he’s able to get across the form, the structure to the viewer in a way which had never been done before and, in many cases, has never been surpassed since."

The point that I want to draw out here, is that of sketching to observe and attend more closely. And sketching not only to see structure, but the relation of structure to function and behavior. The emphasis on understanding mechanisms by considering which parts work in concert to achieve some function or capability, and how they do so. Seeing to understand, looking for surprises, for contradictions that unseat our assumptions, letting us see from a new perspective.

Leonardo 's studies of the brain were on corpses. Today we can study the brain structure and function while the person is alive.

Peering inside the head, while the person is thinking! Gaining a better understanding of what parts of the brain are active, when we are doing tasks of different kinds.

We're learning more, through visualization, of such deep and dynamic structures as the brain's pathways.

Now. No talk on visual design would be complete without the obligatory reference to the amount of the brain devoted to visual processing.

It's not my field and I don't know what is considered the definitive reference, but one source mentioned 30 percent and another 50 percent of the cortex is devoted to processing visual information -- the discrepancies being due, as I understand it, to visual processing being used in conjunction with other systems, such as the motor system, so it is hard to separate out.

At any rate, we are geared to take in a lot of the information we do, through our visual system. Advantage seeing over hearing, touch, smell, taste. Though smell plays a strong enough role to be the referent metaphor when code goes bad, and we seek to identify and remove code smells.


Of course, before we concede victory to those whoop-whooping at the advantage this confers on visual design expression, we might note that code is visual.


In an essay titled What is Software Design (published in the C++ Journal in 1992), Jack Reeves argues that source code is software design.

I agree. Indeed, I'd argue code is the dominant medium of expression of software designs. We design in the medium of code (including tests) -- that is, we reason about, we shape, we express how we are creating system capabilities, assigning system responsibilities and responding to forces, in the medium of code.

Let's think about that, for a moment. A few years ago, this data point struck me -- the six million lines of code for an airplane, is equivalent to a 3-story high pile of books!

Sam Guckenheim said "The code is the truth" and Grady Booch added "but not the whole truth."

Three stories, and it's not even the whole truth?!

That's a lot of truth! It is hard to wrap our arms, or heads, around quite so much truth.

Dana Bredemeyer puts it differently: "Code is fact. Truth is emergent." From interactions within the code, and of the software with its context of operation and use. That's even more to wrap our heads around!

Individually, there's a lot to try to comprehend and shape, to intellectually gain traction on, the design -- so we get more what we want. But software systems are complex, and more than one mind can manage, so we're not just communicating the design to the compiler, but with others on the team, now and in the future.




Okay, so I was being a little dramatic there. The software for an airplane isn't one humongous monolith. Various systems contribute to the aforementioned volume of code. No-one has to understand all that truth, in its entirety. But that helps make my point. There are dimensions of design where code is not the medium that best supports reasoning and expression.

The code doesn't generally express, in any ready way, a "big picture" view of the system as a whole, and we have to build up a mental model of the elements and relationships. Zooming out further, the code doesn't express the system in the context of other systems (the use context, and the deployment context).

From the standpoint of human understanding, code is lossy. We have to mentally simulate behavior, conceptualizing interactions. Further, the following is generally missing from the source (though, especially at narrow/local scope, may be partially inferred from tests and assertions or comments):

  • assumptions (where? under what conditions? ...)
  • rationale (why? connecting the dots to intended outcomes/value and challenges)
  • theory of operation (how? explaining how key mechanisms are intended to deliver outcomes with an emphasis on qualities or cross-cutting concerns which by nature are non-local and emergent, identifying challenges faced and how they're addressed)
  • alternatives considered, but ruled out (what we're not doing and why not)

Jack Reeves' point that source code is software design was, alas, taken to mean "Jack says forget design and just start coding." So much so, that Jack had to write a follow-up article and later letter to the editor, explaining his position and remaking the case that "When I am coding, I am designing." Writing code (and tests, and doing debugging) is not just implicitly doing design, it is explicitly doing design. It is a medium for expressing and externalizing our design thinking (making things more the way we want them to be), and interacting with it, probing and testing it, improving it. But it is not the only means of expressing our software design. Code is the expression we must end up with, so we should do all we can in code. Still, Jack reminds us of a point he made in his first article:

"In software engineering, we desperately need good design at all levels. In particular, we need good top level design. The better the early design, the easier detailed design will be. Designers should use anything that helps. Structure charts, Booch diagrams, state tables, PDL, etc.—if it helps, then use it."

He adds:

"Today, I would phrase it differently. I would say we need good architectures (top level design), good abstractions (class design), and good implementations (low level design)."

So, code is a design language. But not the only one, and not good for every set of design challenges. In particular, when it comes to architecture -- to design decisions of structural and strategic import, with broad scope of impact -- we need other support for individual and group cognition. To support thinking, collaborating, communicating. Enter the pencil.

Our models may take the form of exploratory sketches, diagrams, mock-ups or paper prototypes, proof of concepts, specification-level detailed models and simulations, prototypes or working code.

Visual thinking expands our mental capacity, and sketching these images and models expands our processing power still more. The mind’s eye is resourceful, but it can’t readily see what it’s missing. Sketching creates external memory, so we can hold more information and we can see the relationships, the connections, and we can reason about causalities and consequences.

We draw informal sketches or more formal, precise models, to collaborate much more effectively with ourselves--with our moment ago, our yesterday, and our next month self. We draw to see, and record our thinking, and to expand and enhance our own thinking. To test and challenge and improve our designs.

And we use sketches and other models to collaborate with others. Now, and over time and distance. By collaborating on pictures, on design models, we create a shared thought-space. Now we have more minds actively engaged in coming up with alternatives, testing and challenging and improving the models--while they are just sketches and the thought experiments and reasoned arguments are quick to play out and the biggest cost of change is the cost of letting go, the cost to egos. 

“Language shapes the way we think and determines what we can think about” - Benjamin Whorf

Modeling abstracts away complexity inessential to the reasoning under the lens of the moment, enabling us to bring complex systems within the grasp of our bounded cognitive capacity. [You could apply caveats to my statement, but you'd be adding complexity.] As programmers, we're using abstraction, and abstractions, rather fluidly. We build our systems out of richer and richer, more and more powerful, abstractions.

A children's rhyme comes to mind:

Big fleas have little fleas,
Upon their backs to bite 'em,
And little fleas have lesser fleas,
and so, ad infinitum.
-- Augustus De Morgan.

But adapted to:

Big things are made of smaller things,
With interactions to unite 'em,
And smaller things have smaller things,
and so, ad infinitum.

"Programming is a process of designing a DSL for your own application" -- Dave Thomas

"The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise." -- Edsger W. Dijkstra

You're no doubt familiar with the well-trod conference joke:

"There are two hard problems in computer science: caching, naming, and off-by-one errors." -- source*?

Well, "places to put things" is the dual of the "naming things" bugbear in coding, no? What we are striving to do is to craft "crisp and resilient abstractions" (a Boochism). Granularity is a judgment call, and cohesion of responsibility can be a bit of a slippery notion. Sure, we have heuristics, but assumptions are a kind of dependency, if you like, on some external condition. I stray. You see? Cohesion. Slippery slope. Anyway... Crisp speaks to being non-leaky, non-tangly, having an internal cohesiveness with clear boundaries, and resilient speaks to adaptable under change without flaking out.

"what’s everybody’s responsibility is nobody’s responsibility"


* Source not clear -- Leon Bambrick? And Tantek Çelik? Either way, it's a riff on (reportedly) Phil Karlton


Complex systems can't be approached as merely (many,) many "parts flying in formation, trying to be an airplane" (Wim Roelandts)


"I strongly believe that there is a law of conservation of complexity in software. When we break up big things into small pieces we invariably push the complexity to their interaction." (Michael Feathers)

Words matter. Visual models do too.

"Conceptual illustrations do what words cannot." - Steven Heller



"The responsibility of architecture is the architecture of responsibility." -- Jan van Til

Fundamentals of design:

  • clear Separation of Concerns;
  • crisp Abstractions;
  • balanced Responsibilities;
  • Simplicity
    -- @Grady_Booch


Martin Fowler is the eminent "thing explainer" of our field.
To recap, then, we've established -- okay, we've argued -- that software architecture is that part of design that entails decisions that are structurally and strategically significant. That is, the integrity of the system depends on them. And more, they shape possibility for the business. Enable and constrain in make or break, business defining ways. At the level of scope of impact of the system in question, of course.

Benjamin Franklin's address in the closing debates of the 1787 Constitution Convention begin:

Mr. President

I confess that there are several parts of this constitution which I do not at present approve, but I am not sure I shall never approve them: For having lived long, I have experienced many instances of being obliged by better information, or fuller consideration, to change opinions even on important subjects, which I once thought right, but found to be otherwise. It is therefore that the older I grow, the more apt I am to doubt my own judgment, and to pay more respect to the judgment of others. Most men indeed as well as most sects in Religion, think themselves in possession of all truth, and that wherever others differ from them it is so far error. Steele a Protestant in a Dedication tells the Pope, that the only difference between our Churches in their opinions of the certainty of their doctrines is, the Church of Rome is infallible and the Church of England is never in the wrong. But though many private persons think almost as highly of their own infallibility as of that of their sect, few express it so naturally as a certain french lady, who in a dispute with her sister, said "I don't know how it happens, Sister but I meet with no body but myself, that's always in the right-Il n'y a que moi qui a toujours raison."

-- Benjamin Franklin, Madison Debates, September 17, 1787

[Footnote *]

It is a modern belief that the fall of rational man is a modern belief. Fallible we knew we were then. Fallible we know we are now. Perhaps we're more aware? And more generally aware? Let us not count the ways, for wikipedia already lists some 130 cognitive biases!

Regardless, it begs the question: technical wisdom, please oh please?!


"In the long run my observations have convinced me that some men, reasoning preposterously, first establish some conclusion in their minds which, either because of its being their own or because of their having received it from some person who has their entire confidence, impresses them so deeply that one finds it impossible ever to get it out of their heads." -- Galileo Galilei

Of course, any invocation of technical wisdom, conjures Gerald Weinberg.

In this context -- remember, we are talking about human fallibility -- Weinberg's Rule of Three is the rule to bring to bear:

"If you haven't thought of three possibilities, you haven't thought hard enough."

Three possibilities? For everything? That smacks of BDUF FUD (fear, uncertainty and doubt)?? Can't we just YAGNI that? Well, remember, these are make or break decisions. Game shapers and game changers.

Still. We have more wisdom to call into play, this time Bucky Fuller's, and what Dana Bredemeyer calls the "extraordinary moment principle":

"What, at this extraordinary moment, is the most important thing for me to be thinking about?" -- Buckminster Fuller

Architectural judgment factors.


Part III: Intention and Reflection

"Design is not just what it looks like and feels like. Design is how it works." — Steve Jobs

Recall "we design to get more what we want" (paraphrasing Herbert Simon). There's a lot compressed into that insight. Who is the "we" in "we want"? What do we want? How do we get it? More? That smacks of tradeoffs...

“We shape our tools and thereafter our tools shape us” -- Marshall McLuhan via John Culkin in "A schoolman’s guide to Marshall McLuhan"
Developing software is transforming a fiction in our heads into something that makes stuff happen in the real world. More complex fictions, more heads, more hands touching the fiction into a realization.


[Footnote: I'd like to point to Brian Foote's "Cobbler's children" post because it amplifies the point I'd been making (then and since).]




There is a relationship between the system architecture and the organization architecture:

Conway's Law says: organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations

That is, the organizational architecture (and hence, in effect, assignment of responsibilities to teams) dominates or significantly determines the architecture of the system (elements, responsibilities of system elements, and communication among elements).

Law of the Commons:

"what’s everybody’s responsibility is nobody’s responsibility"

It doesn't mean the tragedy of the commons is inevitable, but when cross team effects aren't readily visible, say, structural drift and erosion may be more likely. Making code-team relationships visible as they evolve, helps to explore system properties and adapt the team, in order to get more what we want from the system (for development teams, as well the system as it serves users and other stakeholders).

"Because the design which occurs first is almost never the best possible, the prevailing system concept may need to change. Therefore, flexibility of organization is important to effective design." -- Mel Conway



[H/T: "pro-tip: if your business-systems architecture resembles a Jackson Pollock, it's time for a big rethink & remediation #entarch" -- Brenda Michelson]

Viz code  
Code As Crime Scene  

When what we are looking to understand better is easy though tedious to count, there's real value in using code to do so -- it can show us things we hadn't noticed. And it's objective. Well, objective to the extent that we are objective when we set up what we are counting, and how we use the results. Alternatively put, these tools are important, but again, judgment factors!

We need to study our system, its topology and ecological shifts, as well as its mechanisms, with the same curiousity and systematic intent that da Vinci exemplified. We need to understand the relationships between the form that is taking shape, and the functions and properties of the system. We need to do this, and that other thing -- to discover how intent is reshaping around our system.

And, let us not forget, that we can still (also!) use sketches to reflect (and reflect on) what is in the code, or what is emergent from it.

The picture is from a study an artist did with children and their "sensory journeys" locating smells and noises and other sensory experience on their routes to school.

We can -- should! -- create the same sorts of maps, collecting developers' impressions of the code base and mapping the "here be dragons" and "sublime -- look and learn" and "send clean-up crew," etc., places... If our architectural intent was that do-once kind of thing, with some diagrams lying in a drawer somewhere (maybe), we can get component or service owners in the room, and map out the architecture as it is.

As we seek to understand and share our understanding of the system, written and spoken words are important. But pictures, sketches, diagrams, models informal and formal, add appreciably to gaining cognitive traction, understanding relationships -- across time, across the system.

This is a visual map of the Amazon report on the AWS outage of September 20, 2015. The map was created by Richard Cook (an expert in the system's resilience space all of us architects should be paying attention to). Visualizations help us think about and see relationships. Whether it is the code itself, or the system executing in its operating context, and we're studying the operating envelope as it slipped towards an outage.





This is the stuff of architecting. We design to get more what we want. Structural integrity of the code, despite change along various vectors. And design integrity -- fit to context, purpose and use. And to get more what we want, we need to understand, to reflect on what we have and how well that is meeting our sense of what our stakeholders want. We can use code to help with this -- creating visualizations of the structure and dynamics of our code and software systems, much as we use code to create visualizations for other industries from medicine and MRIs to cycle helmet design and fluid dynamics.

Because our systems are ever evolving, architecting is ongoing. Structurally and strategically significant decisions come up early and we need to set direction so we can create that ground under the feet to move forward on. But these decisions need to be revisited. Structural integrity needs to be an active and ongoing concern and it is not attained and preserved by localized, piecemeal approaches. New architectural mechanisms may needed to meet emerging challenges, for example as the system scales across threshholds that mean new approaches must be sought. Emergent properties, cross-cutting concerns, need architectural oversight -- understanding, discovery, study, and (re)design.

I wanted to close with thoughts from the audience on visualization in design, because the big thing I wanted to do was put visualization back in our design toolbelt. We created antibodies to visual modeling when UML jumped the shark, and never really wholeheartedly course-corrected to where we appreciate the power of visual modeling -- in just enough doses using good judgment about when and how much. Of course, that is a reference to our other prominent theme -- that of technical wisdom. But there's also the silver bullet we began with, which is a nod to the importance of organizational wisdom, too.


'The hand is the visible part of the brain' -- (attributed to) Immanuel Kant (Who knows?)

Part II: Visual Design -- halted in progress. Fully annotating the slides takes time (even though I've already prepared for and given the talk), which is hard to justify given the work we're doing, but I'll see what I can do. Some other rainy Saturday night. :-)

11/4/15: Hopefully, though, as I work through the slides adding notes, you're getting both more eager to see more of the notes and excited about the slideset as a whole.

11/5/15: Well? Aren't you just itching to see what I said... about Peter Naur? I said he drove right up to my house, and then went on by. I'm not kidding! Now you're interested? No? I'm stung! Okay, how about Simon Brown? And how did Richard Cook end up in there?

But first. Jack Reeves. There's a cliffhanger. Serialized non-fiction ftw? Of course, since I'm the only one checking in on progress.... Well, if I leave it long enough, I'll have forgotten all I did and intended to say... and it'll surprise me too. ;-)

I'm going to have to fork more alter egos so I have an audience! :-)

11/10/15: So, you have the teaser -- the beginning, the middle, and the end. Is it worthwhile to forge on with writing up the notes? Hey, let's gamify this -- which slide should I annotate next?

It's getting to be far enough in the past, that recreating the notes calls this to mind:

‘Day by day and almost minute by minute the past was brought up to date . . . . All history was a palimpsest, scraped clean and reinscribed as often as was necessary.’ -- George Orwell, 1984

Keep my sense of humor in mind when I say that, and adjust for self-defacing inflation. Also, with the exception of a few minor corrections, the slides are unaltered.

11/23/15: There's enough ick in the world, we can make a bigger contribution being kind than hurtful. Reality is harsh enough. No more harshness is needed. Unfortunately it is true that we can elevate ourselves at another's expense in our own eyes, and in social standing. But really, do we need to? So why not be considerate, and notice and applaud the good?

We are all in this together -- this advancing what we understand system design and architecture to be, and how to practice it so we create better systems, is something we are all part of. And it is better if we treat it as a collaborative venture. It is a big task, way bigger than any of us. And important -- we're creating systems that thread through our lives, and we need to get better at it for complexity compounds in massively and variously interconnected systems in dynamic contexts. The more we advance this field, the more there is for all of us to do. Competitiveness may be necessary when an ecosystem is over-exploited, but this one has so many dimensions it needs to grow in, that we can create healthy niches and avenues for ever so many of us to do good work, and make contributions that are valued.


Those among us who are unwilling to expose their ideas to the hazard of refutation do not take part in the game of science. — Karl Popper

The other side of refutation is support. But if we don't-won't unambiguously support, then criticism weighs without counter-balance. Now, in the case of my talk, someone's criticism was not refutation but of form ("lacking in content and structure"), but the point generalizes. If we don't support good work, any detractors have undue influence simply by virtue of our indifference, our silence. As much as I value statements like "good example of a slidedoc" (because they mean to be good to me, as well as good to Nancy Duarte), the very slantness of the praise undermines the strength of my work here. We damn with faint praise, or praise of something incidental to the work. We also damn with late praise -- like praise that comes on the heels of a hissy fit about seldom getting unambiguously positive feedback. ;-)

[* Footnote: Franklin goes on to say

"In these sentiments, Sir, I agree to this Constitution with all its faults, if they are such; because I think a general Government necessary for us, and there is no form of Government but what may be a blessing to the people if well administered, and believe farther that this is likely to be well administered for a course of years, and can only end in Despotism, as other forms have done before it, when the people shall become so corrupted as to need despotic Government, being incapable of any other. I doubt too whether any other Convention we can obtain, may be able to make a better Constitution. For when you assemble a number of men to have the advantage of their joint wisdom, you inevitably assemble with those men, all their prejudices, their passions, their errors of opinion, their local interests, and their selfish views. From such an assembly can a perfect production be expected? It therefore astonishes me, Sir, to find this system approaching so near to perfection as it does; and I think it will astonish our enemies, who are waiting with confidence to hear that our councils are confounded like those of the Builders of Babel; and that our States are on the point of separation, only to meet hereafter for the purpose of cutting one another's throats. Thus I consent, Sir, to this Constitution because I expect no better, and because I am not sure, that it is not the best. The opinions I have had of its errors, I sacrifice to the public good. I have never whispered a syllable of them abroad. Within these walls they were born, and here they shall die. If every one of us in returning to our Constituents were to report the objections he has had to it, and endeavor to gain partizans in support of them, we might prevent its being generally received, and thereby lose all the salutary effects & great advantages resulting naturally in our favor among foreign Nations as well as among ourselves, from our real or apparent unanimity. Much of the strength & efficiency of any Government in procuring and securing happiness to the people, depends, on opinion, on the general opinion of the goodness of the Government, as well as well as of the wisdom and integrity of its Governors. I hope therefore that for our own sakes as a part of the people, and for the sake of posterity, we shall act heartily and unanimously in recommending this Constitution (if approved by Congress & confirmed by the Conventions) wherever our influence may extend, and turn our future thoughts & endeavors to the means of having it well administred."

but there was more in my own talk to cover, so I cut him short. With apologies. For they are important points about architecture and architects, and matters of leading well by following well, among other things.



clue bucket


Clue Bucket

Here's the genesis of the clue bucket idea:

‘when people just have no resources to deal with a tricksy situation, their “clue bucket” is empty.’

We fill our clue bucket as we move through life; we study, we gain experience. But we can, for instance, look through other clue buckets to help fill our bucket with useful clues. Books are a traditional formalized collection of clues, one might say. There's also looking at (other people's and reflecting on our own) code, looking for principles, heuristics, useful analogies we can hoist from another domain to serve ours,... Filling the bucket, sometimes intentionally, often through happenstance as we feed our curiosity, so we can “get a clue” when it’s needed.





Leonardo da Vinci apprenticed himself to Andrea del Verrocchio to learn anatomy. He studied geometry (starting in his 40's?) -- for a time with Luca Pacioli, and illustrated Pacioli's Divina proportione.



Architects: More relevant than ever!

Richard Veryard's Agile and Wilful Blindness post came up again. Which reminds me: I didn't think of myself as challenging Simon Wardley -- I was agreeing with him, but nuancing the point. That said, I liked where Richard Veryard went, namely:

"One important function of architecture is to pay attention to the things that other people (such as developers) may miss - perhaps as a result of different scope or perspective or time horizon."

We try to pry the architect's contribution horizon open still further, to include adding strategic value. An important aspect of strategy setting is situational awareness. This isn't just about risk, but about opportunities to create and extract value.

Also, I would put it more strongly:

"One important function of architecture is to pay attention to the things that others will miss as a result of different scope or perspective or time horizon."

Because. What I said:

What we're paying attention to shapes what we perceive and pay attention to

It is a seriously shaping factor. Remember -- gorilla!

Oh pfft. Yeah. The architect will miss things too. The point though, is that the architect has to take the system perspective (or system of systems, if that is the architectural scope), when the architect judges it to be an architectural moment. We can decide that "architect" is a role/hat/responsibility set that is shared... But when we do that, we need to be aware that we have moved responsibility from someone to the team, to tell architectural moments from more local (narrowly scoped) and impending (like this sprint) pressures. It's a nice idea to spread architectural responsibility around, but when responsibilities get mixed in like that, delivering value now takes precedence over working out what it will take to deliver and deliver and deliver. Architect, as a role, becomes a locus of responsibility for the system, and having a leader who is good at listening and influencing, in that position, means that technical considerations have representation in broader direction-setting decisions. Further, for complex systems, developing architecture expertise/acuity and responsibility for identifying architectural moments or decisions of strategic and architectural consequence and responding appropriately to them, is a matter of substance.



Be Principled about Principles

My response to this:

"Personally, I think the use of architecture principles is responsible for many, many IT disasters."

is: "shooting from the hip considered harmful." Consider that an architectural principle. (wink) In very exceptional, rare instances, if you don't have time to aim, you may be better off just shooting as you draw. Unless you're like... oh wait. That's a myth. Too.

Architecture principles are about setting technical direction. If we set direction, we need to stay on top of situational awareness. We need to be aware, and adapt/respond to sustain fit to the shifting context. This is why strategy is not a do once thing. In any arena of strategy setting. Including technical strategy (which enables and constrains business strategy).

Principles are a medium for strategic expression. Part of what leading means is enabling. And if we are going to use architecture principles, we need to enable them to be used effectively. Guidance like "explain or comply" doesn't have to be "toxic" (shudder); it can be simply a shorthand for a richer organizational understanding -- one where we expect to follow the principle judiciously and intelligently (it's just a principle, so it sets direction, meaning some of the decision space is eliminated, but the decision terrain that remains is still tricksy and needs to be approached with smarts fully engaged). And if we find we need to explore alternatives that would be inconsistent with the principle, we'll "explain" -- or more wordily put, keep the architects abreast of our needs and the forces we're responding to, so they can judge if we're missing something, or they are.

Uh. Ooops. A bit reactive/quick on the draw there, was I? The post challenges us to think about principles. I think it overstates its case, but that is a challenge to (re)think and restate. It is incredibly hard to make words mean what we mean. I mean it! On both sides of the words (sending and receiving), we have to enter into it in the spirit of moving understanding forward. Relationships of goodwill, and all that. Yes, yes. The words are important. The stance, the desire to understand, the constant re-orienting when people gunk gets in the way of understanding, all that is as, or more, important. Words do a lot of heavy lifting. But they are impotent in the face of determination to misunderstand, to obfuscate on the receiving end. Hopefully I didn't let my defense of principles cloud my judgment too noxiously. Bottom line: if we use words like "explain or comply" and if they take on a life of their own in a bad way, we do need to respond. It's dynamic, and we need to flex with it. But at the same time, if we can't use handles for meaning, we're going to be stuck with having to use a truckload more words for everything, and that is dysfunctional (shudder) too.

Btw. Can we make "counterfunctional" a word we use instead of dysfunctional or toxic? At least until it absorbs all the bad stuff and is too icky and we have to find another? Wink. (By "wink," I mean we have to hold ourselves in a playful, tender crucible -- we're fallible, foible-ridden creatures, but amazing too.)

But. Systems/projects fail for complex (interrelated/entangled, ambiguity fraught, ...) reasons. So let's not flame and broil principles under that banner.



Would the people raging about the use of the engineering term in software puh-lease get off my lawn? Yes? Awesome. Thank you so much.

That was too easy. I didn't get a chance to use my argument. Sheesh. I mean really, take a look for example, at this, from the Intel site:


That was just the first engineer job description I looked at, having searched for job postings for electrical engineers. Take a look at it. See any requirement for engineering certification?

In product development, we often see different job requirements than in other areas of industry. Maybe we will come to require certification for safety critical systems. Maybe we will come to require some evidence of serious training in ethics for engineers, including software, in robotics and autonomous anything design.

Maybe. Or maybe we will continue to have the sense in a highly dynamic field to stay away from certification. That is the kiss of inertia.

"An engineer is a professional practitioner of engineering, concerned with applying scientific knowledge, mathematics, and ingenuity to develop solutions for technical, societal and commercial problems." -- wikipedia

It goes on:

"Engineers design materials, structures, and systems while considering the limitations imposed by practicality, regulation, safety, and cost."

Oh wait:

"The work of engineers forms the link between scientific discoveries and their subsequent applications to human needs and quality of life."

That eliminates us. Not!

But seriously. Read the description of engineering. Speak to engineers in other fields about what they do. Some who have certifications (or rings) will be scornful (or worse) of others who don't; we want to make the club we belong to exclusive and significant. The posturing is socially important, but tangential to the point of whether we do engineering.

So yeah. We are engineers. Many of us, anyway. There are just engineers of different kinds. A chemical engineer is very different in substance than a mechanical engineer. And within an engineering discipline, the range of problems addressed, and how, is vast. Mechanical engineers designing on a Formula 1 team may reference craftmanship just as readily as many of us do. They have a range of application, and within them, diversity in approaches and styles, and team cultures. The same is true in software.

Using law to control the use of the term engineer is not minimalist, and we really do not need to decry our successes in avoiding an intrusive law (where it is not needed for some social good that can't be accomplished some other way).

Let's get better at being engineers, by recognizing we are engineers! So that where we can push frontiers and the application of mathematics, probability and stochastics, and scientific knowledge, we will. Then, while we will still use ingenuity a lot (if we're not innovating we need to worry about NIH and waste), we will have more to leverage and inform us, too.

TBH (seriously), I don't really care if we call ourselves engineers or not. I do like what it brings to bear as we think about what we need to develop in ourselves and in our field. I just had to play it out a bit, to surface some points I generally see being elided when we berate our use of the term in software. I just want us to be great engineers, no matter what we call it. (That's a mischievous wink. In case your mischief sensors are crudded over. ;-)

Well. At least my Trace never insulted its audience this way:


Indeed, I do rather suppose an incredibly sophisticated facility with nuance and... well.. ambiguity. Really. ;-)



Yes. That means someone pointed me to Gene Hughson's Tech Debt post, which had me looking at his tweet stream to see why a 2013 post was in the air again. And from there to his latest blog post.

I suppose I should just give in and reconstitute my @ruthmalan persona already? Right. Right. Your encouragement is deafening. ;-)

PS: Public encouragement looks like a positive pointer to something awesome I did. Avoid making me look foolish. I do that plenty enough for all of us, so no need for your help on that -- but thanks for being willing to contribute and all. ;-)



Collegial Camaraderie

Collegiality is underappreciated, or at least not sufficiently talked about in our field. Camaraderie creates a sense of social space, with conversations that illuminate, encourage, share humor and the moments that make us more human. If that's something you value, be sure to follow Eugene, Henry, Vish, Stuart, Andra and Gene. And perhaps we'll manage to convince Brenda that 100stickmen are architecturally significant... In the EA space there's Sally and Ernest. And Tom, who doesn't follow me, but you should still follow him. ;-)

I need to put this in my phoenix folder! In the game of life, my visibility setting is turned down so low, few people can actually see me -- so when this happens, it means a great deal to me:


Bestest 2

It's coming up for 10 years of tracing, but more than 10 years for this site, and more than 15 for the Bredemeyer site. I've been giving to the architect community for a long time. It is spirit nourishing when something like this comes back. It is rare that this happens, so there's no danger of overinflating my ego or anything like that.



We Design to Get More What We Want

A kind mention drew my attention back to the slide alongside.

Recall "we design to get more what we want" (paraphrasing Herbert Simon). There's a lot compressed into that insight.

Who is "we" (in "we want")? Stakeholders! Users? Yes, but when we're talking about design, it's not just users, important as they are. What do we want from the code (understandability, adaptability,...), the system in operation (scalability, monitorability, ...), the system in use (capabilities, usability, ...), the system in ecosystem (value, system-of-systems composability, cost of sales, ...). So developers, testing, management; operations (or hardware and manufacturing, etc. in product contexts); users and their business team (at some level); our business team, channel partners, and so forth.

11/15/15: Structural integrity? Imagination factors! Abuse cases are important too -- here's an example...


Thanksgiving Done Come Early!

Wow -- people were stunningly kind today! Thanks to Gene Hughson, Brenda Michelson ("Oooo"), Amitai Schlair (magical/"Keep checking"), Sally Bean ("Great use of @nancyduarte's slide doc method"), Stuart Boardman ("Looks good - almost as good as the content"), JP de Vooght, Eugene Barker, Russ Miller and Marc Burgauer. And Grady Booch!! Also Paul Harland who pointed out that this page is a lot to scroll down (in the nicest Paul way), which launched the whole slidedocs frenzy on my part today. :-) Also Cory Foy, because I was trying to reset my twitter habit when he was so kind. Anyway, your support and encouragement gives me resilience! It is always the same people who are kind to me, and as you can see, if it weren't for yawl (smile), I'd be one sad puddle of dejected Ruff. But, instead -- woohoohoooo :-)

I don't imagine anyone else is reading here, but if you are, them's the good folks to follow. Clearly! I mean. Smart. Patient. Kind. Endless list of great qualities. And not least of all -- tolerance for ambiguity. Which I work out excessively.

I do appreciate those who respond/converse and maybe favorite/heart. But mentions and retweets are important to get sometimes, or you begin to think you can't do anything right. :)

“If we take man as he really is, we make him worse. But if we overestimate him … we promote him to what he really can be.” -- Viktor Frankl




Misunderestimating Estimates

The (no)estimates discussions so often blast estimation on the basis of uncertainty. We can't estimate, because uncertainty. Uncertainty? The world is dynamic and complex. And yet at the same time, they assume away something fundamental to complexity, namely diversity. Diversity in contexts, demands, pressures, constraints, arenas of novelty, arenas of attempts to collaborate. Different diffferences. For example, some projects feed into complex webs of systems that have a fair amount of inertial intertwining with other systems where constancy over periods of time is (considered) a good thing. Yank a thread of their dependency web, and systems at a distance topple. So they push back strongly on continuous deployment and want some degree of predictability in delivery so they can prepare downstream systems for the new functions and their consequences at each rollout. What kinds of systems? There is considerable variety, but for example, systems in high risk environments, systems in highly regulated contexts that have to go through lots of hoops when something changes, etc. But also supply chains. And distribution and sales channels. Oh sure, we do need to keep unsettling our assumptions before they're unsettled for us -- ubered, or otherwise. Still, there is this dance between stability and change, and it's just as well to keep in mind.



Smoke and Mirrors

As things "smoke and mirrors" goes, Serendipity pushed this under my nose:


Source: Andrew Marvell's Liminal Lyrics: The Space Between by Joan Faust

Cool, yes? With tie-in to wisdom and everything!

And Chaung Tzu.


Fractal Design

For those giving poetry (e.g., the first Chaung Tzu link above) and architecture a "where's the concrete" side-eye -- I used the children's rhyme on slide 51:

Big fleas have little fleas,
Upon their backs to bite 'em,
And little fleas have lesser fleas,
and so, ad infinitum.
-- Augustus De Morgan (

because it was easy to turn into a little fractal design ditty:

Big things are made of smaller things,
With interactions to unite 'em,
And smaller things have smaller things,
and so, ad infinitum.

But the original Jonathan Swift (1733) is so much better at conveying the self-similarity, and a twist and winking side-eye of its own:

So, naturalists observe, a flea
Hath smaller fleas that on him prey;
And these have smaller still to bite 'em,
And so proceed ad infinitum.
Thus, every poet, in his kind,
Is bit by him that comes behind.

Then, of course, there is also Herbert Simon's The Architecture of Complexity (.pdf). Prose. But we'll forgive the Herbster, because pertinent:

"The four sections that follow discuss four aspects of complexity. The first offers some cominents on the frequency with which complexity takes the form of hierarchy-the complex system being composed of subsystems that, in turn, have their olvn subsystems, and so on. The second section theorizes about the relation between the structure of a complex system and the time required for it to emerge through evolutionary processes : specifically, it argues that hierarchic systems will evolve far more quickly than non-hierarchic systems of comparable size. The third section explores the dynamic properties of hierarchicallyorganized systems, and shows how they can be decon~posed into subsystems in order to analyze their behavior. The fourth section examines the relation between coillplex systems and their descriptions."

And Ray and Charles Eames' "Powers of Ten". The movie. (9 minutes--watch it!)

You can still like my ditty, though. It stole fire from Swift, as he foretold. As for concrete? First, we have to learn how to mix it? Where to use it? And when? And some other stuff...

It's a big world. There's room for some poetry. Some wisdom and... perspicacity. And some impish humor. Among architects, even!! ;-)


I also write at:

- Bredemeyer Resources for  Architects


Architects Architecting Architecture'



Architecture astronaut





Copyright © 2015 by Ruth Malan
Page Created:October 21, 2015
Last Modified: August 31, 2020