A Trace in the Sand
by Ruth Malan
My Trace is a playground for developing ideas, for exploring architecture and the role of architects. I don't know where these ideas are going to come from, so I explore, finding the dots to connect and sharing them, and the connections I make, playfully and thoughtfully, here.
It is interesting to consider how the changing relationship substrate (technology that supports communication, for example, but also cultural changes like increasing education level and expectations around social equity) is creating different mechanisms for the co-ordination of markets, enterprises, and complex value streams and transformations. The (business) ecosystem has different options than it did a century ago. And with the different options, different expectations and aspirations.
There is a co-evolution of demands for complexity (indirectly; of course the demand is for something that is complex, not complexity itself) and the ability to achieve it (and then do it better, and better -- hopefully).
Hierarchies were (and in many ways still are) very good at co-ordination of work and resources, and enabled very complex systems to come into being, and be run effectively and efficiently -- enough to sustain them... enough... to have some lifespan. To be sure, hierarchies have been modified with a criss-cross of informal networks that supplement the formal structure of control, resource and information flow. But as the substrate, the firmament, the infrastructure and social basis shifts, and as the apetite for complexity increases, and the ability to deal with it expands, new forms -- collaborative, co-creative, highly networked "podular" forms, for example -- can be tested by the trials of (a much sped up organizational and social and technological) evolution.
As fast as we can, we're moving responsibilities from people to machines, changing the work landscape. People become the interfaces and variability servers, the resilience buffers, and so forth, connecting points among increasingly automated systems-of-systems.
This places greater cognitive and creative demands on an ever thinning tier of humans in the loop. At the same time, there is greater amplication of those humans -- brain stimulation and cognitive skill building earlier (in childhood) and more richly throughout life, as well as the increasing capacity of "intelligence servers" at their command -- from Google to Watson's offshoots and Wolfram's computable knowledge. The discontent at hierarchy's efficient movement to, and concentration of resources in, the wealthy few is one matter; discontent at hierarchy's concentration of aspiration, prestige, and decision autonomy another. Discontent (not malcontent, but a desire for better) pushes at the fabric of organizations. As does the need to deal with ever more complexity and uncertainty, creative responsiveness and adaptability. Organizational adaptations like layering networks upon hierarchies go some distance. Enough?
The substrate is shifting and heaving, and society is reforming itself, dynamically trying to cope with the tumultuous changes in what is possible and desired.
We look around and much still seems the same. Except that the economic basis is shifting. Work is flowing to digitally enabled machines. Manual and cognitive. Co-ordinative. Fast. And ever faster? And relationships of organizations to people, and people to organizations are changing. When people are expendable, that is seismic. When it is you and I that are, we feel the quaking personally. Between IBM's Watson and Amazon's delivery drones, we're getting a glimpse of the near future, and it is putting people out of business as we know it.
We can't foresee where it all leads, but for now, we know that hierarchies will persist (albeit in hybridized and modified forms). For some time. And the cottage revolution that thrives on the infrastructural substrate provided by the likes of etsy together with the individuality aesthetic that gives the craftsman appeal will persist. For some time. Hierarchies. Networks overlaid on hierarchies. Hierarchies within networks. More and less loosely networked pods.
Moreover, as the relationship substrate is transformed, new organizational forms become possible or more viable. At the same time, people and social capacities are changing, augemented by technology, and the apetite or desire for new kinds of systems -- more complex (e.g., more software intensive with more elements of cognitive compute-enable adaptive responsiveness, etc.) systems -- grows. Putting pressure on organizational forms to adapt, to morph or mutate.
My point? These changes ripple through the ecosystems of organizations, upping the ante in complexity, with more interacting webs of interconnectedness within and among systems(-of systems) and the systems(-of-systems) that create and operate them. Some form of hierarchy may persist to provide some form of co-ordination to pods, in a fractal sort of organization. But we might also expect considerable diversity and experiment and evolution in organizational forms.
12/12/13: I was gently reminded that "complex" is a rather charged term in some circles. Well, if building a termite mound is described as complex, is it such a stretch to describe designing and building a Dreamliner as complex? It is useful, from within a framework, to use terms consistently, but I'm not using any framework as a reference point here.
12/14/13: My traces are exploratory and informal, but I hope you could grok the format and flow well enough to make sense of it. But, in case I did a poor job, here's how I see the flow of the (above) trace:
And here's a bit of clarification, for the sake of conversation, not combat (smile):
Relationship substrate: enabling flows of information, value, etc., or alternately put, lowering interaction costs, but also changing interaction dynamics and enabling whole new forms of value, and flows and transformations thereof.
Complexity: tentatively -- very, very tentatively -- by complexity, I mean systems where interconnectedness of components gives rise to capabilities and properties of systems, and the scale of the system and the interconnections is both cognitively challenging and vulnerable to unforeseen design and operational suprises and even failures. This is exascerbated in systems with ever more software in sense-and-respond/control roles, especially when these systems are interconnected, and the effects of some condition (failure or simply unanticipated operating condition, etc) can have ramifications at a distance. The design process is one of trying to move the system in the direction of being under greater control intellectually and operationally, but as the cognitive challenge across all the many components, subsystems and systems in these systems-of-systems grows, we rely ever more on simulation and production testing and feedback from operations and whatnot to improve the understandability and reliability of these systems, and even then reliability is a matter of performance envelopes and reliability curves, and so forth. So across the lifecycle, uncertainty and non-determinacy comes from a variety of sources. We try to damp and proactively stave off rolling impacts of failures and yadda, but it is still a messy set of wicked problems in design and as systems age or are operated in new contexts, or are added into new systems, or...
You get my point? I'm positing that when we characterize a system as complex, we are talking not about the built system under controlled operation within some controlled contexts under given assumptions about component wear, and so forth. We are bearing in mind all the complexity, all the wicked problems that had to be dealt with, to get the system to that point (during design of the system, and its manufacturing lines and supply chains) and we might also have in mind the undeterminable ways the system might fail when pushed outside that envelope in which it is understood to operate in some determined and determinable, anticipatable, predictable way. For example, cars have long had a dominant design, but because that design is ever being pushed, with more and more software in the loop, we keep pushing the frontier of cognitive control over the whole thing, so we have to keep pouring enormous resources into testing and warranties and so forth. Why? Because we have to learn what we don't know. When you have side-effects and "action at a distance" and knock-on effects, and yadda, from the unanticipated consequences of actions and interactions of various systems (such as tire pressure and power steering in my car, such that steering goes away if the tire pressure goes down!!), we're talking complexity, no? Moreover, as we move more of the "intelligence" -- the responsibility to sense and respond to environmental conditions, to set and adjust course, and so forth -- into systems, the more complex those systems become. That is, they become more able to respond to complexity, to the interaction of multiple forces and conditions -- unforeseen, even.
Anyway, don't take me to the mat on my characterization here. I am merely trying to move the conversation along by saying complexity is complex and fraught with emergent (mis)understanding. ;-)
So, I said complex systems. If you prefer to think a car or plane of today, etc., is complicated rather than complex, that's your choice. I think that if we let words adapt themselves to context, and we stay flexible in our interpretation (keeping our mind open in respectful expectation), we can generally make our way just fine.
12/18/13: My argument, as I think of it, is that we shouldn’t dismiss hierarchies out of hand – they have served us, and that they have, and that they have been adapted, demonstrates that social evolution has been at work in producing an organizational form that has been workable. It may not be ideal, and its problematic areas will be stressed as we move ever more “intelligence” (sensing and responding/controlling) into software-threaded systems(-of-systems), ratcheting up complexity by interconnecting everything*... But heck, consider all the ways that Nature has produced some pretty amazing design suboptimizations… For neat examples, take a look at WTF Evolution -- I like these for architecture classes, because… well, you know. Analogies. Evolution doesn't necessarily produce "best" -- indeed, we would not have evolution if "best" was instantiated in one go, to begin with. Evolution finds the flaws in fit to (shifting) contexts, and in better and poorer ways, shift-shift-shifts, ultimately finding workable, and for periods of time, even epochs, sustainable forms. Any one of which may be faulted -- consider humans. We don't seem particularly well-adapted to sustainability of the planet's ecosystems. The planet will survive us, but how much will we take down as we send waves of ?creative destruction through technologically-underpinned and natural ecosystems?
Anyway, I wasn’t advocating or defending hierarchies. Merely indicating that we humans have done some pretty amazing stuff with that dominant “architectural style” (see the Fractal and Emergent paper; I know you have read it, but it is rich and deserves to be recommended again so everyone gets a chance to read it ;-) To be sure, hierarchies have been adapted with inlays/overlays and “pod”-like cross-functional teams internetworked in part informally, though with the hierarchy doing the high-level resource allocations and such. Hierarchies have these webs of adaptations overlaid and threaded through them, to make them work, and at what point will the web (of more podular relationships, for example) dominate the hierarchy? In many cases, has it already?
And in another beautiful demonstration of the logical contortion we humans are capable of: 15 proofs that cats are liquid. ;-)
PS: Thanks to Stuart Boardman for taking me seriously enough to have conversations with me on this one. :-)
* Everything? Gulp; we've even hooked the climate up to our cars and greedy desire for stuff... (woeful smile)
2/2/14: A Company Without Job Titles Will Still Have Hierarchies, Harrison Monarth, January 28, 2014
2/8/14: Making Sense Of Zappos And Holacracy, Steve Denning, 1/15/14
Beautifully organic depiction of (1855!!) org. chart (does our visual metaphor shape how we view hierarchies?):
More examples in the vein of "what were you thinking, Evolution?!":
I find it useful to think how this translates to software intensive systems:
Thanks for the pointer, Peter Bakker.
The National Parks are getting friendly:
It was a spectacular photo...
Hm, this is why I need to trace, even if few will read any particular trace, and fewer will agree that it is significant:
I thought it was a useful insight, and shared it with a notable influencer who just didn't connect with it at all. Undaunted, I still think it is important and overlooked. (It threads into the waterfall-ish post last month.)
Here's another, related:
What a lovely phrase from Paul Harland today:
I also recursed on functional programming, The Sense of an Ending, I am a Strange Loop, mise en abyme, and the Droste effect.
Our brains are fun, aren't they?
Software. All about giving meaning to and making sense of abstractions? Inferring and testing out connections, and the emergent properties they give rise to? Okay, okay. I'm teasing. And yet.... do you hear that sinister music... ;-)
12/12/13: Dana pointed out that the root of stigmergy is stigma which means mark. Or goad. And ergon which means action. Or work.
12/18/13: Hm. It occurs to me that Margritte's The Philosopher's Lamp might be included with I am a Strange Loop... ;-) A prescient meditation on insight, no? The addiction. The... uh.... yeah. Okay then.
12/19/13: As self-referential goes, here's some xkcd:
(Why) Things Fail
They do. What? Fail. Consider these examples:
In the case of products, "reliability" testing is because:
I'm given to understand (Sarah Hrdy) that even pictures of babies stimulate our brains to release oxytocin, which "tends to dampen the parasympathetic reactions to stressful situations, making it easier to cope with stress." You need to know that if you read my Trace. ;-) You might also want to snip it and place it on your dash for stressful commutes. ;-)
Image: from Michael Feathers Joy of Coding 2013 Keynote.
Stability and Destruction
If you've followed along here for a while, you know that I find the ecology of organizations is incredibly interesting. My thoughts on this subject are tentative and act more as tripwires to my attention, sending me off on different paths of investigative reading. study, analysis. They also act as stimulants to conversations in my own head, and -- to the extent that people take me seriously enough to invest in dialog with me -- with others. With that caveat (pre)emptor sort of notice...
I've noted in earlier traces (2/12/12, 2/27/12, 9/5/11, 4/6/13) that organizations work to create ecosystem stability that allows mutually adaptive tuning, to put value into, and harvest value, from the value flows that weave through the ecosystem, and better thrive. So as new (sub)ecosystems are formed, organizations within them work to create moving regions of (dynamic) stability. But "waves of creative destruction" wash through ecosystems, and those that don't adapt are folded back into the environmental firmament, freeing resources for the new generations of organizational organisms and their value creating, value harvesting activities.
I have been writing about architects architecting (software, systems, and enterprise) architecture in this Trace for close to 8 years. A very small number of people advocate it; fewer still quote from or otherwise treat it like a work of merit that deserves collegial consideration and a handshake, now and then... With that as context:
Let me acknowledge with enormous gratitude those who have given this Trace a hat-tip in 2013, with mentions on Twitter (search ruthmalan.com on twitter if you feel the need to verify):
And in 2013, this Trace was quoted or mentioned in blogs by:
Of course, the most awesome thing that happened to my Trace this year, was Peter Bakker's storylines tubemap!
A few more people RTd (Tom Graves is especially supportive), or @directed, DMd, or emailed a comment to me about this Trace, and I value those forms of collegial responsiveness/inclusion too.
Aside: Since tweets have a very short shelf-life, I haven't worried about links to traces in journalCurrent that expire when the month turns over. If you ever wanted a trace link that will persist, just ask. :-)
System, System-in-Context and Context
Software Architecture Workshop
Take note: I'm running an open enrollment Software Architecture Workshop in Boulder/Louisville, CO on March 24-27, 2014
You should be there. ;-)
I'm not sure from the phrasing if this is due to Martin Thompson or Pieter Hintjens (who wrote the post), but I like it:
As depicted in this cartoon. :-) Anyway, it is a nice reference back to the ideas forming in the "morphing organizational forms" trace earlier this month -- we can expect continued evolution not just in the systems we create, but the organizational forms that create them. As we create more systems-of-(themselves complex)-systems, we can expect the organizational forms to have to evolve to accommodate the greater system-level challenges in the system and in the organization. They will co-evolve, because if they don't, Conway's Law warns us that the organization form will trump intended designs that go "cross-grain" to the organization warp.
It feels to me like Jérémie is giving guidelines that take Conway's Law into account, more than generalizing the Law. A useful set of ideas, and perhaps I'm just being an idiot. Won't be the last time, I promise. ;-)
Oh yeah. The very first programming class I taught, I had the teams compete to see who could find the most mistakes in what I said and coded. I was like 26 or something. It was hilarious. The CEO joined us at a break, to see why everyone was laughing so much. Some kind of gutsy, huh? Yep. Still going out on that wire, balancing humility and confidence. Not always very gracefully. ;-)
Paying close attention to someone, listening to them, attending to them, tending them, is taking attention from something else we might be doing. Attention, who we attend to, isn't just about generosity. It is also about the shape of the social landscape, that directs and creates conduits for attention. So there is not just the generosity of attending, but generosity in directing attention.
This is a wonderful example of generosity in directing attention:
People are so often flat in their pointers to another's work and it was generous of Grady to use his social cred and an enthusiastic framing to draw attention to Alex Bell and his work. Highly deserved, of course. Yet still generous to do -- generous to notice that he was delighted by what another person has done, and generous to share that delight, so others can too.
But I have an even better example. Smiles. Peter Bakker's storylines tubemap was extraordinarily generous both in how much attention he paid, and in directing attention. Peter not only read and distilled the essence of traces, but he took it to another level of finding the themes that were meaningful to him to draw out and connect up. Smiles. The map gave me a new way of seeing my own work, and that in of itself was wonderful! Generous too, for it communicates in a unique way that my work is worthwhile and that touches me deeply. It is also a very good demonstrator of the power of tubemaps to express/explore/reveal storylines.
Part of the realization here, is that it takes a generous inclining, or orienting in what we are paying attention to. We are giving time, but also positive expectation and moving in our own mind so that we better understand what the person is revealing to us of themselves, what they know, how they sparkle and spark insights to life in us through their work or confidences.
And we are putting aside any petty distractions of negative interpretations like self-interest that might niggle and try to claim attention away from our delighted and caring attending. We have to watch what stories we tell ourselves -- we can be remarkably inventive in how we let others come across to us. Categorizing and stuffing into Procrusteam frames that hack people down to the small conception we impose on them, instead of generously, with goodwill, finding where they are brilliant or hurting or... some other stuff... (it grows late; family chores require loving attention... reminding me of Jurgen coming to see Dame Lisa's domestic ministerings, albeit full of fussing and chiding, as a glorious poem -- near the closing of Jurgen: A Comedy of Justice.... my family gets no epic poems of domestic ministry from me... the daily drudgery is quite enough... though I did make biscotti at Ryan's request for the screening of the DeCycles 2013 movie the other night. That counts, no? ;-)
Of course, attention is far broader in scope than collegial attention. But collegial attention factors importantly in our lives, for our work time claims so much of our days... and nights... like now... oops ;-) But friendships within our professional circles are important too. They enrich us and buoy our spirits. A chance "incomparable value" from such a friend today, means untellably much to me. The conversations too, for they signify mutual respect and trust in wrestling understandings into fitter shape. But the authentic expressions of "you matter" or "your work is valuable" are the reflections back to us that we aren't just deluding ourselves about the merit of our work, and our spirits waste cycles yearning for such feedback, if it is never forthcoming. Or if it is so veiled and subtle as to be ambiguous. We need direct, authentic reflections to help us interpret those that are more subtle and indirect. Anyway, these things are what makes working in this field warm and rewarding. Attending and tending is important to our families but also our friends, for it is not exactly the common thing and yet so important to wholesome lives and glowing, thriving spirits.
[One of these days I will break down and use reading glasses and/or turn automatic spell checking on. But I'm in denial. ;-)]
Alex Bell? Here:
Bringing To Life
Oh hey, are you following Paul Harland? Interesting person and great conversationalist. It's so neat that twitter lets us twitter-stalk (the non-threatening kind) people like Paul and eavesdrop on their tweet-versations. :-)
In my privileged and hopelessly biased view, Visual Architecting is all about real agility or thriving in a changing world, and also sustainability (economic, technical, organizational, and environmental) and integrity (system integrity like resilience; conceptual integrity like coherence and simplicity, etc.).
The conceptual architecture write-up shares in a very pithy way the key considerations of what to document, why to do so, and how to approach architecture from a truly agile perspective. Pithy, concise,...compact? Indeed. It is a distillation from nearly 20 years of working in this architecture field, and each line expresses something really important. Consider, for example:
Doesn't that just give you goosebumps? Identify? Well, that's obvious. Explicate -- explain, make explicit, clarify, make plain/clear, spell out, untangle, interpret, translate, elucidate, expound, illuminate, throw light on. The "in detail" slant of explicate I can do without in this case, but in the previous sentence I was clear that we're doing this in broadbrush terms. (Logical architecture is where we design and document key interfaces, protocols, mechanisms in more actionable, precise -- though just enough -- detail.) Rationalize?!? Well, that does allow for my impishness because rationalize is one of those Janus words -- obviously in this context I mean clearly reason about, and provide rationale for, the design choices being made. Obviously? Well, that's a wink at cognitive biases and such. Contextualize? Ah yes. Tour de force. ;-) Key structures and mechanisms? Your lightbulbs going off in rapid fire there? No? Sheesh. Not just components and connections, but mechanisms? Structure and dynamics. So many definitions of architecture fixed with just that little dancing glance. Should I put so much into a single sentence? Well gracious me, think how much longer it would be, if each sentence didn't carry quite such a payload.
Anyhoo... Give it a read with an orientation open to discovery and delight, rather than orienting to find fault, because I worked hard on it, and Dana and I have worked hard to reach the understandings that it shares.
This was nice of Uwe Friedrichsen:
That is the most positive public thing said about that post! Come to think of it, I think that is the only positive broadcast... (thanks for RTing Gene and Oliver).
Advocacy for my work is so rare and important, I'm hugely grateful to Uwe. And. Just so you know, here's my fantasy version:
Smiles. I'm kidding. But it is important to frame my work in positive terms, or people won't get over the barriers to acquainting themselves with it, will they?
For those with tl;dr-itis, it might be worth noting that I broke it into 3 posts in my blog:
But those versions are cemented as blog posts from 2012, while the full version is revised from time to time. [Cute mistype -- I wrote 2102! But what an awesome freudian slip -- so far ahead, it is cemented in the future!!! ;-)]
Here's a couple of posts that floated by in the stream this week, on broadly the same topic:
The way that we ensure that we don't have Rosalind Franklins in our field, is to give open hearted, open minded support and respect to the women in it. Not saying I'm Rosalind Franklin, but I am indicating that we tend to back off on the gas on those rare occassions that we do talk about a woman's contributions. Work is hidden, unless people refer to it, talk about it -- enthusiastically even.
This trace would have lent much to the conversation, don't you think:
Oh, do note what was added at the end of the trace, on 1/31/13. ;-)
If You Celebrate Christmas
I hope it was full of joy!!
Of course, we did our part to bolster the economy and add to climate change. Sigh. So complex. But I totally won the mom badge with xkcd t-shirts for both teens. ;-)
With all the scurry yesterday, I didn't get the mail (all of 2 blocks to go get it, but... busy). So, we got it this afternoon. Why tell you? Well, as Serendpity serves, I got a lovely Christmas surprise -- my Buttercup Festival original from the Renaming of the Birds Kickstarter in effect arrived TODAY!! :-) Yay! I can't wait to frame and hang it.
Thank you, good friends who are kind enough to read here and interact with me, sharing camaraderie and your insight with me. You are the best!
[12/27. Woah. That real quick late night Christmas trace wasn't a good idea. Well. Fixed several typos. <Scolds fingers!>]
Searching for Sugar Man
If you've seen it, watch it again! If you haven't seen it yet, do! It is amazing! I smiled, sometimes through tears, through the whole thing! It is an amazing story, and it also makes it very clear that an audience is yin to the artist's yang.
In the end, I think we have this story of a revolution long overdue, a revolution we need to usher in. A revolution? Yes, you know. The one where we rise to our humanity, instead of sinking to it.
12/31/13 ... 1/1/14
Happy New Year!
I hope you have an awesome 2014, filled with dreams and realizations, joy and becoming.
I also write at:
- Bredemeyer Resources for Architects
Architects and Architecture
- Todd Hoff (highly recommended)
- Anna Liu
- JD Meier
Architect Professional Organizations
Agile and Lean
Agile and Testing
Other Software Thought Leaders
- CapGeminini's CTOblog
CTOs and CIOs
CEOs (Web 2.0)
- Don MacAskill (SmugMug)
- Wired's monkey_bites
Social Networking/Web 2.0+ Watch
- Dan Roam
- David Sibbet (The Grove)
Strategy, BI and Competitive Intelligence
- Freakonomics blog
Um... and these
- CNN Money Business of Green videos