What's This Then? A Trace? In the Sand?
This is where I keep a trace of thoughts on wanderings in the landscape of software and systems architecture and those that abut and have bearing on our field. And on February 3, I will have been doing this for 10 years (with a hiatus here and there, where I wondered if the risk of putting an open mind on display for potential public thrashing is a good idea, or just got discouraged because no-one was interested, so why take on the risk).
Okay so. My Trace is pretty awesome -- well, in the sense that if you like this sort of thing, you'll find this the sort of thing you like (after Lincoln, I believe -- at least, F. Scott Fitzgerald thought so too; but I repurposed it, with opposite spin, for in this case, obviously if you like this sort of thing, you're in good company... At least, we think so.)
What? Aghast at me saying it's awesome? Well, you know, it's been TEN years (I get to shout now? Just a little? It's been 10 years!)... And some of us need to place unequivocally positive words out there, to frame the thing in a way such that just the right sort of people notice it. It's a dirty job, so I pitched to help. I'm generous that way. :) I mean, we couldn't leave it all to Amitai Schlair and Stuart Boardman and Peter Bakker, now could we? (Also grateful for RTs: Gene Hughson, JP de Vooght, Paul Harland and Tom Graves -- to name the most recent. And to Michael Hill for "meaty" and "suppress yabbiting for a minute"). And hopefully if I'm just the right sort of rotten, those who would be ticked off at its playfully confident humility will move right along. :)
Having 10 years of traces behind me, is definitely great grist for the humility mill -- my inner critics take such glee in pointing out all the things past me got even a little bit wrong (in expression or understanding)!
To give some sense of the scope under the lens, here is a partial map of topics (and links to traces). For a taste, see for example,
Well, there are words here. And if you don't like to see words dance, move right along.
How Does It Work?
Michael Hill and Amitai Schlair each said something that slanted a new light on how wonderful people like them experience this Trace:
"so, looking at it, just the current page, one thing is clear, it needs to be soaked in. that is, you say a great deal in a surprisingly small number of words. it is "thick" -- Michael Hill
"It expects and requires much of the reader, and is no less than proportionally rewarding" -- Amitai Schlair
I suppose... that could sound intimidating.... like it's work to read, albeit with a payoff. Still, only worthwhile if really on a topic of direct importance? But then again, there's also this from Oliver Baier
"Don’t be fooled by her friendly and casual tone."
"As usual, Ruth’s material is incredibly dense, i.e. lots of information, thoughts and insights in a small space."
Anyway, my hope is that it is, at least to some people, worth reading! Not hard to read, but something that sets off sparkles and flashes of insight in the reader's mind. When I approach it as a reader, it works for me when it does that, and I feel like I failed myself when I read some traces where I was wrestling with some understanding and it was getting the better of me!
Once, upon first encounter with my Trace, one of our field's lovely, most lovable people (sincerely!), pointed me at a writing analysis tool so that I might use it to improve my writing, expostulating (I'm approximating from memory, as one does) "I don't know if that's what you think, or what I think!" Meaning, I presume, that I left some things to be worked out, to be reached, in the mind of the reader, rather than landed on him (in that case) fully formed, and unequivocal. And that says so much about my teaching/mentoring philosophy and my writing style. I think we have to wrestle with insights to see their multi-faceted natures, and how they shape-shift and morph in different contexts. If someone tries to distill understanding into a cookbook and lay it on us to imbibe and ingrain in the webs of our brain.... gonna work? I think not. I mean, I do place a lot of stock in education, in learning, in how to do more of that in a lifetime -- we're not neoliths with neolithic culture... not that you could tell, looking at 2 year olds and adults in our (best and worst) moments... :) So our schooling does leave some wake in our individual and collective minds and mindset. But I also think we learn by munching on, not just swallowing knowledge whole.
Ha. Did I rational argument you into believing my horridly ambiguous writing is entirely justified? You know what I say, because I've said it a few times (here and here) and i. you noticed and ii. you remembered. Hm.
I'm teasing! I mostly think that there are many ways to convey something. And we have much to learn, but it is not all uniform and approachable in a "one best way." Some knowledge can be quite distilled and directly, even assertively, pragmatic -- do this, do that. Other things we've learned, can be conveyed as the kind of checklist that we use as a completeness or attention and memory check -- did we notice this, did we remember to do that? But the topics I grapple with here, tend to be the kind of thing that is more multifaceted, and light needs to be shed from different angles, approached obliquely. Sometimes these are topics we tend to "take sides on," and become polarized around. But things aren't so simple or so easy, and reaching insights from a more holistic perspective may require a style of address that is more tempered, holding multiple simultaneous perspectives in creative suspension. Identifying paradoxes without (kicking in that self-protective lizard brain and) losing our cognitive resources in the face of the tension of holding opposing truths in mind at once. There's a place for those who can approach a divisive topic and not blow people's fuses with highly polarized winner-takes-all stance. Just as, apparently, there is a place for those who do! ;)
The other thing I did, was let my understanding evolve. A journal is never done. It maps a person's progress, and doesn't make any claims to completeness. I used this quote last month:
"As a writer, I love the irresponsibility of the short story...." --Tessa Hadley
It playfully captures that spirit. Traces are irresponsible in the sense of thrusting a reader directly into the thought fray without much easing into, like the usual bookends we expect on articles or essays (in blogs or more formal publications). I don't set the reader up with affordances like "this is what we will talk about" and "this is what you learned." We hit the ground running with an idea, with little warm up or warm down. So each trace has that spirit of being something in progress, part of something larger, something ongoing. Which is the state of an open mind.
Ten Years of Tracing
Thank you to those who were encouraging. It meant -- and means -- a lot! Several people, over the years, came back to look in on this Trace from time to time, and that was gratifying. WB Yeats wrote "and say my glory was I had such friends" and it is not nothing that wonderful people read here a little, now and then.
Now. Time to try a different angle of address. More... like essays, than a journal. Or something.
A Trace was its own thing. Anyway. Time to put it to rest, I think. Not to stop writing. But to try a different style -- force myself into writing more in a more "usual" format.
It is, in the end, just too depressing to write something that is set up to be rejected, because it is too idiosyncratic. Relies too much on a largeness of spirit and inclination to such a peculiar thing as a me, and the way I approach things.
Oh well. Better things to come then. At least, better by the standard of being interesting to, and recommendable by, more than just a (courageous? generous?) few.
People who have been around this Trace over the years, know I've resolved to dump it many a time. Rejection (and silence on a body of work is rejection -- of an insidious unspoken sort that causes a work to be ignored) is a hard thing to face. And hope is a hard quality to kill -- especially when a few people were kind. But, if you write in public view, you obviously hope that it will be valuable, or delight, or be some kind of positive at least to some people. How many people is enough to be sustainable? Well, for example, enough that even on this, the 10th "birthday" of this Trace, it wouldn't be left to ONLY Peter Bakker to rouse the generosity to say something celebratory or appreciative! Oh, I completely understand -- the few people who are responsive, have been responsive in the last month or six, and they are tapped out. But that's the point. If a work is useful and good, worthwhile, but also something special, unusual, above the bar, someone might say so. And someone else. Not always the same someones.
And no. Please don't mention this Trace to create a "sensation" -- let it slumber.
I should add --- I'm not just packing up my toys in a funk. ;) I'll probably use this space for drafts of more topic-focused essay things. Or to note when I've put slides or other material up somewhere else. And so forth. So stop by from time to time.
So less of more and more of less. Or something more or less like that. :)
Okay -- almost silence!
Thanks Eugene Barker and Peter Bakker!!
Bottom line: appreciate diversity. That doesn't mean just talk about it. It means be inclusive.
What''s a Trace? Well, It's like this, see:
On Technical Debt
I think one of the things we stumble on here, is trying to stay loyal to Ward's description, so keeping "[writing] code poorly with the intention of doing a good job later" outside the scope the metaphor. While it is wonderful in spirit to want to honor Ward's contribution to the letter, I think it disserves him and our field to lock down a term that wants to leave its parent and strike out on a life of its own. Gain its own nuance, maybe get a tattoo, ... uh... Or something... Language is alive and evolves. Sometimes, (at least from some perspectives) errantly.
"Technical debt" gained currency because it was expressive of something we evidently needed a handle for. Anyway, it's language. Understanding is contextually situated and active -- on assembly for transmission, and on making sense on receipt. But oh my, can we obfuscate with our cleverness. I don't know of any metaphor that hasn't met its match in the mind of its assailant. ;) A metaphor is a device for conceptual leverage that we use to aid our thinking and communicating. In the case of technical debt, it is largely rhetorical -- effectively so, for it is a relatable, intuitively grokkable way to make the case that code design is not free and if we defer attention to design of the code we will have to spend more (compoundingly more, the more we defer it) to bring the code back to a state of integrity...
That said, I think we need only the slightest tweak to Ward's cast, to make it work (as in, include its dominant colloquial use). In shaping how we think about technical debt, Ward was concerned not to advocate, or be interpretted as advocating, profligate debt, and he -- and/or we -- collapsed disciplined debt onto debt. But the notion of debt is independent of how the debt is taken on and paid back. We can wantonly take on (technical) debt, miring ourselves in pernicious couplings that impact our ability to earn revenue (by preventing us from adapting to market shifts and opportunities) and pay back debt (it's just too much mess and deferred design decision making/refactoring). Or we can take on debt with discipline -- taking on no more than we can afford and only as much as we need to take an opportunity we would not otherwise have, if we did not "borrow [code design and refactoring] time" so to speak, from the future. Then a kludgy ball of mud is profligate technical debt. Not to be recommended. Better to write the best code we can, given what we know now, recognizing that we learn as we go, but we're going to feed that learning back in after a bit, but we're borrowing time to get a chunk of value to market sooner -- the stuff of disciplined technical debt; cue "what Ward said."
@WardCunningham: "Dirty code is to technical debt as the pawn broker is to financial debt. Don't think you'll ever get your code back."
Do we still need architects?
Agile in various of its flavors, feeding into frequent, if not continuous deployments, can raise the profile/visibility of the team and sense of urgency -- we have, for example, "standups" (that make progress, and hold-ups, visible -- but visibility can be pressure; good and bad), "backlogs" and "release trains." We weight early and then frequent delivery to achieve shorter time to value, faster feedback loops. Good stuff. Still, there is a "but" to get past: it takes strong discipline to keep a handle on code quality; to keep the code from falling into (massing, miring) technical debt. Not because we aren't good at our craft, but because we are learning what our system needs to be and become, and we are learning, upon reflection -- if we pause reflect, how to do things better, now that we know more what the system is becoming and how better to enable that (technically). And what strains the system, threatening its integrity (structure of the code and qualities like adaptability and all that is bundled therein, as well as qualities of the software in operation and use including resilience, behavior under load and other issues of scaling, security and more).
Having strong and organizationally-valued technical leadership throughout the project becomes more, not less, important in Agile. The (structural and mechanism) design has to be continually advanced. Sure, the team is participating in the design, impacting the architecture, making architecture decisions – but if architecture is everyone’s responsibility, then architecture is no-ONE’s responsibility and it suffers the plight of the commons*. Someone (with feet to the fire accountability -- and by that mostly I mean they hold their own feet to the fire, managing their attention and responsibility) needs to be attending to the system as a system, to the structural integrity of the code when that competes with the constant-constant-constant get-it-out get-it-out release drum beat. Watching for architecturally impactful, structurally and strategically significant decisions, and making sure they get the attention, reflection, expertise they require. Even when the team periodically takes stock, and spends time reflecting learning back into the design/code, the architect’s role is to facilitate, to nurture, the design integrity of the system as a system – as something coherent. More so, when this coherence must be achieved across (microservice, or whatever, focused) teams. This takes technical wisdom, perspicacity, insight into not just consequences of design choices, but watching for design challenges ahead; for impacts across the system invisible to those whose perspective is shaped by working on a locally scoped piece of code (a feature, a service or microservice, anything bounded). Not just "laying down track in front of the train" but getting out ahead enough to scout out the lay of the land and know where we need to put in a bridge (or architectural mechanism) in readiness. (Which, by the way, is why we need to do some upfront architecting-- just enough, which is a judgment call, so hopefully a sufficiently experienced and proactively seasoned one.)
Can the team, in a self-organizing way, ensure that architecturally significant decisions get the attention they need? If so, great. The likely thing, though, is that dominant concerns of the day (this feature or microservice, now!) shape perception, which will mean playing catch-up on architectural concerns -- they'll be noticed when they bite hard. And architectural concerns are exactly those you want to anticipate, to the extent possible, given technical and organizational wisdom, experience, context awareness and imagination. (That is, they are, by nature, cross-cutting, have deep impact and high cost of change, are make or break structurally and/or strategically.) Further, we shouldn't underplay the importance of technical and organizational gravitas, credibility, ability to lead, to create concert and coherence, pulling towards system integrity, not pulling apart in turf protecting battles. (For example: Want to simplify? Whose pet idea is going to be culled or redesigned? How will that be managed? Different -- competing -- ideas about how to accomplish something? Who makes the call? How? Stuck, because uncertainty and ambiguity in the fog of confounding change? Who's going to create "ground under the feet" by deciding, despite the risk of being "wrong"? Hard things are hard. Harder when brain cells and emotions get tangled up in them. People are people, people!) Leadership is about doing the hard things we aren't doing, that wouldn't be done anyway, as a matter of course because it takes gathering organizational will, determination, concert of minds and hands, to do. It's bigger than a person, or just a couple or three. It competes for attention with other things that are more "obvious" or near term or de facto part of the organizational habit and routine. .
A team, and even more, team-of-teams, benefits much from a technical leader who deeply understands the technical ramifications and is advocating for the team and helping to shape the context (get resources, protect from political pressure, etc., but also do architectural investigations and design ahead of it becoming a trainwreck for the project, etc.) so the team can do great work, create a system with integrity -- structural and design integrity, but in the ethical sense too. Overly rigid dominance hierarchies that weren't a good fit for agility have left antibodies, but if we overcorrect in the direction of not valuing wisdom and foresight, experience, focus, context awareness, system-level problem finding-solving, ..., and leadership, other problems mount... But leadership in organizations is fractal, and fluidly, dynamically forming and reforming. Further, the architect (by title or simply by responsibility) leads where that is needed -- including leading by the example of following well or stepping out of the way.
What we are paying attention to, shapes what we perceive and pay attention to.
* Not sure who referred to the tragedy of the commons in software first... "Siri, who was first to apply commons ideas of Forster Lloyd/Hardin/Ostrom to software development?" Nothing. Watson? (I see I mentioned it in 2009, and it has a c2 wiki page (last dated 2010), so it was in the air before that. Probably goes back to the 70's, like so much else in software. :)
Conway’s Law: “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations” — Melvin Conway, via wikipedia
Conway’s Law paraphrased: if the architecture of the organization is at odds with the architecture of the system, the architecture of the organization wins.
Conway’s Law implication: managers are architects of the organization^^ and they need to partner closely with architects of the system to ensure mutually consistent, synergistic structures (homomorphic) [and mechansisms with associated structures, behaviors and properties, etc. -- the stuff of architecture that isn't just about structure]
Learning how to ask the question 'What does your code need?' -- Lynn Langit (pointing to Michael Feathers on Symbiosis)
** and as with other architects, there is a spectrum of possibilities in terms how this is approached; in shaping the organizational structure (at various -- potentially dynamically shifting -- fractal scopes) and dynamics; managers-as-organizational-architects can focus more on shaping the context that enables more emergent self-organizing, rather than determining team structure, individual responsibilities, and interaction paths, fait accompli.
How about this, titled "Strong Opinions"? ;)
Or, this from another musing on technical debt:
Developing software is transforming a fiction in our heads into something that makes stuff happen in the real world. More complex fictions, more heads, more hands touching the fiction into a realization. What could go wrong? (A tired meme, I grant, but only because it applies so liberally.)
In saying "I'm done with tracing," I demonstrated the value of what I do, just as surely as the silence on what I've done, proves that it has no value. The conundrum of being a me in this field! :)
“You live out the confusions until they become clear.” -- Anaïs Nin
Schmonz Appreciation Day
If you're internet-familiar, or better still, f2f acquainted with the Schmonz, you no doubt appreciate some or other facet of the awesome he brings to this world. Here are a few:
- his Agile in 3 Minutes podcast, which is a wonderful way to take Agile at its word, pushing 3 minute nuggets of high value to its audience on a regular heartbeat (and the book of essays from the podcast -- his voice is awesome, so listening as the podcasts come out is a real pleasure, but the book gives a mechanism for reference and speed reading back over a topic)
- his blog, which reveals both a lot of pragmatic mindfully-tried-and-practiced (I first said lovingly-worn hand-me-down -- because Schmonz has this wonderful quality of being susceptible to advice and wisdom, taking it on board, making it his own, then making it better and handing it on) guidance and insight, and a person who is real (life deals some stuff we don't seek, we seek some stuff life is intractably stubborn about, and so forth), and a heart-cheering life-affirming wonderful, talented, insightful and questing-seeking-understanding geek's geek (in the best of best ways)
- his music (he composes and plays) which you can introduce yourself to here
Raising a glass of virtual bubbly in celebration and honor to you, Amitai. So glad you're sharing this little ride on spaceship earth with us!!
Schmonz is ma homie, but I shielded him from actually meeting me for couple of years -- I'm best not met. At least, the world gives me plenty enough feedback that I'm better channeled through another interface than me straight up, shy, "female of a certain age," and nerd-awkward. ;) But it was a lovely coffee break that extended over 2 coffees. :)
My sentences are hard to read (hey, GeePawHill done tol' ya, stop yer yabbiting!)? So are people (hard to read because multi-faceted, dynamically shifting and adapting to varying and various contexts), people. We have to allow a little unfolding, no? Read me, and FP will seem trivially transparent. ;)
Also, I should use a spell checker to get my back on typos? You're so nice.
- Trace In the Sand Blog
- Bredemeyer Resources for Architects
- Journal Map
- storylines tubemap by Peter Bakker