A Trace in the Sand

by Ruth Malan

 

 

 

 

Architects Architecting Architecture  

February 2016

Table of Contents: 10th year anniversay month of traces!

About This Trace

Strategy

Technical

Organizational

Architects: What are they good for? What makes a good architect?

Community

2/1/16

Yeah!

 

What's This Then? A Trace? In the Sand?

This is where I keep a trace of thoughts on wanderings in the landscape of software and systems architecture and those that abut and have bearing on our field. And on February 3, I will have been doing this for 10 years (with a hiatus here and there, where I wondered if the risk of putting an open mind on display for potential public thrashing is a good idea, or just got discouraged because no-one was interested, so why take on the risk).

Okay so. My Trace is pretty awesome -- well, in the sense that if you like this sort of thing, you'll find this the sort of thing you like (after Lincoln, I believe -- at least, F. Scott Fitzgerald thought so too; but I repurposed it, with opposite spin, for in this case, obviously if you like this sort of thing, you're in good company... At least, we think so.)

What? Aghast at me saying it's awesome? Well, you know, it's been TEN years (I get to shout now? Just a little? It's been 10 years!)... And some of us need to place unequivocally positive words out there, to frame the thing in a way such that just the right sort of people notice it. It's a dirty job, so I pitched to help. I'm generous that way. :) I mean, we couldn't leave it all to Amitai Schlair and Stuart Boardman and Peter Bakker, now could we? (Also grateful for RTs: Gene Hughson, JP de Vooght, Paul Harland and Tom Graves -- to name the most recent. And to Michael Hill for "meaty" and "suppress yabbiting for a minute"). And hopefully if I'm just the right sort of rotten, those who would be ticked off at its playfully confident humility will move right along. :)

Having 10 years of traces behind me, is definitely great grist for the humility mill -- my inner critics take such glee in pointing out all the things past me got even a little bit wrong (in expression or understanding)!

To give some sense of the scope under the lens, here is a partial map of topics (and links to traces). For a taste, see for example,

Well, there are words here. And if you don't like to see words dance, move right along.

 

How Does It Work?

Michael Hill and Amitai Schlair each said something that slanted a new light on how wonderful people like them experience this Trace:

"so, looking at it, just the current page, one thing is clear, it needs to be soaked in. that is, you say a great deal in a surprisingly small number of words. it is "thick" -- Michael Hill

"It expects and requires much of the reader, and is no less than proportionally rewarding" -- Amitai Schlair

I suppose... if you were inclined to expect a negative, that could sound intimidating.... like it's work to read, albeit with a payoff. So, only worthwhile if really on a topic of direct importance? But then again, there's also this from Oliver Baier

"Don’t be fooled by her friendly and casual tone."

which follows:

"As usual, Ruth’s material is incredibly dense, i.e. lots of information, thoughts and insights in a small space."

I think -- hope! -- that what all three meant is not that it is hard to read, so much as that it reveals more, the more willing one is to engage and follow-up on the (get-a-)clue-trail it leaves. ;) I do hope that it is encountered as something that sets off sparkles and flashes of insight in the reader's mind. When I approach it as a reader, it works for me when it does that, and I feel like I failed myself when I read some traces where I was wrestling with some understanding and it was getting the better of me!

Once, upon first encounter with my Trace, one of our field's lovely, most lovable people (sincerely!), pointed me at a writing analysis tool so that I might use it to improve my writing, expostulating (I'm approximating from memory, as one does) "I don't know if that's what you think, or what I think!" Meaning, I presume, that I left some things to be worked out, to be reached, in the mind of the reader, rather than landed on him (in that case) fully formed, and unequivocal. And that says so much about my teaching/mentoring philosophy and my writing style. I think we have to wrestle with insights to see their multi-faceted natures, and how they shape-shift and morph in different contexts. If someone tries to distill understanding into a cookbook and lay it on us to imbibe and ingrain in the webs of our brain.... gonna work? I think not. I mean, I do place a lot of stock in education, in learning, in how to do more of that in a lifetime -- we're not neoliths with neolithic culture... not that you could tell, looking at 2 year olds and adults in our (best and worst) moments... :) So our schooling does leave some wake in our individual and collective minds and mindset. But I also think we learn by munching on, not just swallowing knowledge whole. And those things have to interest us enough, to cogitate and try things out -- in our mind's eye, and with our mind-hands working on it. So there is a place for ideas that are more suggestive, than pedantly put. That entice us to skip our own idea stones on the river of knowledge... uh... whoops. Little carried away on a thought stream there... The point is, what engages us? It's not all the same-same same.

Ha. Did I rational argument you into believing my horridly ambiguous writing is entirely justified? You know what I say, because I've said it a few times (here and here) and i. you noticed and ii. you remembered. Hm.

Rational

I'm teasing! I mostly think that there are many ways to convey something. And we have much to learn, but it is not all uniform and approachable in a "one best way." Some knowledge can be quite distilled and directly, even assertively, pragmatic -- do this, do that. Other things we've learned, can be conveyed as the kind of checklist that we use as a completeness or attention and memory check -- did we notice this, did we remember to do that? But the topics I grapple with here, tend to be the kind of thing that is more multifaceted, and light needs to be shed from different angles, approached obliquely. Sometimes these are topics we tend to "take sides on," and become polarized around. But things aren't so simple or so easy, and reaching insights from a more holistic perspective may require a style of address that is more tempered, holding multiple simultaneous perspectives in creative suspension. Identifying paradoxes without (kicking in that self-protective lizard brain and) losing our cognitive resources in the face of the tension of holding opposing truths in mind at once. There's a place for those who can approach a divisive topic and not blow people's fuses with highly polarized winner-takes-all stance. Just as, apparently, there is a place for those who do! ;)

The other thing I did, was let my understanding evolve. A journal is never done. It maps a person's progress, and doesn't make any claims to completeness. I used this quote last month:

"As a writer, I love the irresponsibility of the short story...." --Tessa Hadley

It playfully captures that spirit. Traces are irresponsible in the sense of thrusting a reader directly into the thought fray without much easing into, like the usual bookends we expect on articles or essays (in blogs or more formal publications). I don't set the reader up with affordances like "this is what we will talk about" and "this is what you learned." We hit the ground running with an idea, with little warm up or warm down. So each trace has that spirit of being something in progress, part of something larger, something ongoing. Which is the state of an open mind.

2/3/16

Ten Years of Tracing

Thank you to those who were encouraging. It meant -- and means -- a lot! Several people, over the years, came back to look in on this Trace from time to time, and that was gratifying. WB Yeats wrote "and say my glory was I had such friends" and it is not nothing that wonderful people read here a little, now and then. Ted Hughes

Now. Time to try a different angle of address. More... like essays, than a journal. Or something.

A Trace was its own thing. Anyway. Time to put it to rest, I think. Not to stop writing. But to try a different style -- force myself into writing more in a more "usual" format.

It is, in the end, just too depressing to write something that is set up to be rejected, because it is too idiosyncratic. Relies too much on a largeness of spirit and inclination to such a peculiar thing as a me, and the way I approach things.

Oh well. Better things to come then. At least, better by the standard of being interesting to, and recommendable by, more than just a (courageous? generous?) few.

People who have been around this Trace over the years, know I've resolved to dump it many a time. Rejection (and silence on a body of work is rejection -- of an insidious unspoken sort that causes a work to be ignored) is a hard thing to face. And hope is a hard quality to kill -- especially when a few people were kind. But, if you write in public view, you obviously hope that it will be valuable, or delight, or be some kind of positive at least to some people. How many people is enough to be sustainable? Well, for example, enough that even on this, the 10th "birthday" of this Trace, it wouldn't be left to ONLY Peter Bakker to rouse the generosity to say something celebratory or appreciative! Oh, I completely understand -- the few people who are responsive, have been responsive in the last month or six, and they are tapped out. But that's the point. If a work is useful and good, worthwhile, but also something special, unusual, above the bar, someone might say so. And someone else. Not always the same someones.

And no. Please don't mention this Trace to create a "sensation" -- let it slumber.

I should add --- I'm not just packing up my toys in a funk. ;) I'll probably use this space for drafts of more topic-focused essay things. Or to note when I've put slides or other material up somewhere else. And so forth. So stop by from time to time.

So less of more and more of less. Or something more or less like that. :)

 

Okay -- almost silence!  

DecaTrace

And breaking:

Thanks Eugene Barker and Peter Bakker!!

Bottom line: appreciate diversity. That doesn't mean just talk about it. It means be inclusive.

 

2/3/16

What''s a Trace? Well, It's like this, see:

 

On Technical Debt

From an earlier trace:

Putting my understanding of Ward's point into my own terms, I'd say that technical debt comes from trading technical learning* and code design improvement for market learning (getting features into user hands to improve them) and product design improvement, in order to gain shorter time to value, and time to market advantages (such as network effects and switching costs) over slower competitors. Other market timing factors may play a role, such as significant trade shows or meeting holiday market windows. For a start-up or small company, creating cash flow can be make or break, but even in well-resourced companies management and investor patience may run short, or their risk sensors may be triggered.

WranglingI think one of the things we stumble on here, is trying to stay loyal to Ward's description, so keeping "[writing] code poorly with the intention of doing a good job later" outside the scope the metaphor. While it is wonderful in spirit to want to honor Ward's contribution to the letter, I think it disserves him and our field to lock down a named concept that wants to leave its parent and strike out on a life of its own. Gain its own nuance, maybe get a tattoo, ... uh... Or something... Language, our named concepts, our understanding(s), are alive and evolve. Sometimes, (at least from some perspectives) errantly. It is good to honor their genesis, but in a balanced way that allows also that change changes change [or "things change, even change itself" ;-)].

"Technical debt" gained currency because it was expressive of something we evidently needed a handle for. Anyway, it's language. Understanding is contextually situated and active -- on assembly for transmission, and on making sense on receipt. But oh my, can we obfuscate with our cleverness. I don't know of any metaphor that hasn't met its match in the mind of its assailant. ;) I'm being colorfully playful; hopefully not too offensively so. I mean only to remind ourselves that our objections are constructions (subject to mischiefs of mind) too. Metaphors are not identities, and in teasing out all the flaws (and there will be flaws; truth is a many-natured thing), we can forget to consider the positive impact of the metaphor on general understanding.

A metaphor is a device for conceptual leverage that we use to aid our thinking and communicating. In the case of technical debt, it is largely a rhetorical device (used to impress upon and illuminate, to persuade and influence, to rally will, to galvanize, to prompt or provoke, etc.). And it's an effective one, for it is a relatable, intuitively grokkable way to make the case that code design is not free and if we defer attention to design of the code we will have to spend more (compoundingly more, the more we defer it) to bring the code back to a state of integrity...

That said, I think we need only the slightest tweak to Ward's cast, to make it work (as in, include its dominant colloquial use). In shaping how we think about technical debt, Ward was concerned not to advocate, or be interpreted as advocating, profligate debt, and he -- and/or we -- collapsed disciplined debt onto debt. But the notion of debt is independent of how the debt is taken on and paid back. We can wantonly take on (technical) debt, miring ourselves in pernicious couplings that impact our ability to earn revenue (by preventing us from adapting to market shifts and opportunities) and pay back debt (it's just too much mess and deferred design decision making/refactoring). Or we can take on debt with discipline -- taking on no more than we can afford and only as much as we need to take an opportunity we would not otherwise have, if we did not "borrow [code design and refactoring] time" so to speak, from the future. Then a kludgy ball of mud is profligate technical debt. Not to be recommended. Better to write the best code we can, given what we know now, recognizing that we learn as we go, but we're going to feed that learning back in after a bit, but we're borrowing time to get a chunk of value to market sooner -- the stuff of disciplined technical debt; cue "what Ward said."

Now, if someone wants to grab hold of technical debt as an excuse to write dirty code, we can slut-sh--err, no. (Judginess usually comes from a place of not understanding the compounds of forces, pressures, uncertainties, strivings, inherent in that situation.) But we can point out that it is indiscriminate, undisciplined debt. And bad things will happen.

@WardCunningham: "Dirty code is to technical debt as the pawn broker is to financial debt. Don't think you'll ever get your code back."

2/24/16: As metaphors as rhetorical devices go, Paul Harland pointed us to this:

Aside: Credit card debt may be a good analogy -- people can get into real debilitating financial difficulty, getting over-extended on credit card debt.

 

Do we still need architects?

Perhaps you saw this tweet poll, since you pay close attention to my tweetstream and I retweeted it. Oh. Yeah. Probably not. Well, here you go:

Need architects

The way that survey question is framed, is not neutral. Is it a straw man? Not really. But it does subtly tip the hand towards No. Because it suggests we have to choose between a bad architect/architecting style and none. And that's not the choice, or shouldn't be.

So let's ask "do we still need (good!) architects?" Steve Merrick, Chad Fowler, Eugene Barker and Rebecca Wirfs-Brock all answered it succinctly -- I like "Hell yeah!" But since there are doubts, here's more to reflect on:

Agile in various of its flavors, feeding into frequent, if not continuous deployments, can raise the profile/visibility of the team and sense of urgency -- we have, for example, "standups" (that make progress, and hold-ups, visible -- good, we can coordinate, but visibility can be pressure; good and bad), "backlogs" and "release trains." We weight early and then frequent delivery to achieve shorter time to value, faster feedback loops. Good stuff. Still, there is a "but" to get past: it takes strong discipline to keep a handle on code quality; to keep the code from falling into (massing, miring) technical debt. Not because we aren't good at our craft, but because we are learning what our system needs to be and become, and we are learning, upon reflection, if we pause to reflect, how to do things better, now that we know more what the system is becoming and how better to enable that (technically). And what strains the system, threatening its integrity (structure of the code and qualities like adaptability and all that is bundled therein, as well as qualities of the software in operation and use including resilience, behavior under load and other issues of scaling, security and more).

[Becoming] involves the transformation of potentialities into actualities -- David Ray Griffin (via @cyetain)

Having strong and organizationally-valued technical leadership throughout the project becomes more, not less, important in Agile (at least, beyond some point of organizational and technical complexity). The (structural and mechanism) design has to be continually advanced -- challenges surfaced, alternative approaches identified and evaluated, approaches tried out. Software development is by nature development -- it moves the system into new territory. We have to create novel solutions to novel (permutations of) problems -- bringing experience, creativity, research and trying things out, much to bear on finding an approach that will address the challenges that arise. This ability to design, to solve hard problems, make tradeoffs, is a key attribute of the architect. Sure, others on the team have it too, but there's valuable experience that comes of seeing many things done, adding to the capability set to draw on, but also adding to a sense of what is risky, or hard. This helps in evaluating capabilities bundled into technologies we're evaluating for our stack, as well as in strategically sensing and evaluating where to build a capability to differentiate. There is a dimension of ingenuity and technical wisdom, experience and ability to see around corners, as it were, to identify the gotchas before they getcha. But there is also a dimension of organizational wisdom and effectiveness.

Sure, the team is participating in the design, impacting the architecture, making architecture decisions – but if architecture is everyone’s responsibility, then architecture is no-ONE’s responsibility and it suffers the plight of the commons*. Someone (with feet to the fire accountability -- and by that mostly I mean they hold their own feet to the fire, managing their attention and responsibility) needs to be attending to the system as a system -- one that exists in the context of other systems, with all of their respective forces and demands. There's paying attention to the structural integrity of the code when that competes with the urge to deliver value and all we've been able to accomplish in terms of the responsiveness of continuous integration and deployment. We don't intend to let the code devolve as we respond to unfolding demands, but we have to intend not to, and that takes time -- possibly time away from the next increment of user-perceived value. There's watching for architecturally impactful, structurally and strategically significant decisions, and making sure they get the attention, reflection, expertise they require. Even when the team periodically takes stock, and spends time reflecting learning back into the design/code, the architect’s role is to facilitate, to nurture, the design integrity of the system as a system – as something coherent. Where coherence is about not just fit and function, but system properties. Non-trivial, mutually interacting properties that entail tradeoffs. Moreover, this coherence must be achieved across (microservice, or whatever, focused) teams. Across a lot of code. And then more code! This takes technical know-how and know-when, and know-who to work with, to bring needed expertise to bear. And it takes technical wisdom, perspicacity, insight into not just consequences of design choices, but watching for design challenges ahead; for impacts across the system invisible to those whose perspective is shaped by working on a locally scoped piece of code (a feature, a service or microservice, anything bounded). Not just "laying down track in front of the train" but getting out ahead enough to scout out the lay of the land and know where we need to put in a bridge (or architectural mechanism) in readiness. (Which, by the way, is why we need to do some upfront architecting -- just enough, which is a judgment call, drawing on experience and a combination of design sense and sensibility, timing and risk awareness.)

Can the team, in a self-organizing way, ensure that architecturally significant decisions get the attention they need? Anything that needs to be done, that is happening organically, or because someone else is leading (perhaps with mentoring), is wonderful, and to be encouraged and facilitated. The likely thing, though, is that dominant concerns of the day (this feature or microservice, now!) shape perception, which will mean playing catch-up on architectural concerns -- they'll be noticed when they bite hard. And architectural concerns are exactly those you want to anticipate, to the extent possible, given technical and organizational wisdom, experience, context awareness,perspective and imagination. (That is, they are, by nature, cross-cutting, have deep impact and high cost of change, are make-or-break structurally and/or strategically.) Further, we shouldn't underplay the importance of technical and organizational gravitas, credibility, ability (and willingness, even desire) to lead, to create concert and coherence, pulling towards system integrity, not pulling apart in turf protecting battles. (For example: Want to simplify? Whose pet idea is going to be culled or redesigned? How will that be managed? Different -- competing -- ideas about how to accomplish something? Who makes the call? How? Stuck, because uncertainty and ambiguity in the fog of confounding change? Who's going to create "ground under the feet" by deciding, despite the risk of being "wrong"? Hard things are hard. Harder when brain cells and emotions get tangled up in them. People are people, people!) Leadership is about doing the hard things we aren't doing, that wouldn't be done anyway, as a matter of course because it takes gathering organizational will, determination, concert of minds and hands, to do. It's bigger than a person, or just a couple or three. It competes for attention with other things that are more "obvious" or near term or de facto part of the organizational habit and routine. And it's saying "no," sometimes -- even when it's hard.

"I wasn't the one pushing things in the wrong direction, but I should have been the one to stop it." -- Chad Fowler (around minute 23)

A team, and even more, team-of-teams, benefits much from a technical leader who deeply understands the technical ramifications and is advocating for the team and helping to shape the context (get resources, protect from political pressure, etc., but also do architectural investigations and design ahead of it becoming a trainwreck for the project, etc.) so the team can do great work, create a system with integrity -- structural and design integrity, but in the ethical sense too. Overly rigid dominance hierarchies that weren't a good fit for agility have left antibodies, but if we overcorrect in the direction of not valuing wisdom and foresight, experience, focus, context awareness, system-level problem finding-solving, ..., and leadership, other problems mount... But leadership in organizations is fractal, and fluidly, dynamically forming and reforming. Further, the architect (by title or simply by responsibility) leads where that is needed -- including leading by the example of following well or stepping out of the way.

What we are paying attention to, shapes what we perceive and pay attention to. -- 10/15/09

* Not sure who referred to the tragedy of the commons in software first... "Siri, who was first to apply commons ideas of Forster Lloyd/Hardin/Ostrom to software development?" Nothing. Watson? (I see I mentioned it in 2009, and it has a c2 wiki page (last dated 2010), so it was in the air before that. Probably goes back to the 70's, like so much else in software. :)

Do architects also code?

Yes architects code. Unless they don't. When we hear "architect also codes," we tend not to hear what the context or frame of reference is. The thing is, someone who may be designated architect on a team working on a fairly small and relatively independent software system, has a different-natured set of concerns and responsibilities than an architect of a product family, portfolio or business domain, or the architect of a large complex system, or system of systems, or the chief architect of a business unit or large company. That is to say, as the architectural scope of what an architect is responsible for increases, we need to rely on the architect's judgment as to where they need to spend their attention cycles. For example, the chief architect for the Civic, say -- cars come in product lines, remember -- may not even have a software background (though the software content in cars is mindbogglingly large, and getting larger). Further, do you want the chief architect for a product family or business domain, popping in on a project team and taking on a critical path piece of coding work? Pairing on it? Sure! Design and coding an architectural mechanism that will be shared across teams? Yes, especially if this is done collaboratively with impacted teams, even potentially mob-modelled and programmed -- at least to get initial challenges and ideas shared, alternatives designed and tried out.

That is, the more complex (technically and organizationally) the space the architect is bringing coherence across, the more the system (of systems) demands attention and the architect has to balance all those demands with learning and staying in touch with the system. The architect has to be able to partner well, delegating and communicating effectively. If we don't understand this, we set the architect up to be questioned on competence terms against developers on the various teams under the scope of the architecture. The credibility of the architect is important, because it impacts the team's orientation to the architecture -- if there is no trust, it will show up in various ways that undermine the architect and the architecture. It's important to recognize and reinforce the value the architect brings in terms of system design, so system integrity (structural and design integrity, coherence, resilinece, etc.) and sustainability. It takes time to understand what needs to be thought through, and then to design, evaluate alternative approaches, document and explain and explain and explain. To persuade, to influence, to shape understanding, to draw attention, to highlight, to inform. To explain in person, dynamically, in conversations. To explain with reasoned arguments we've taken time to sketch out and write down. To share our thinking, and to revise and improve our thinking by getting it out where we and others can see and respond to it. Writing and modeling to think things (aspects of the design, how we approach a design challenge, etc.) through, to get them clear for ourselves, to help us get them clear for others. (You want me to describe and explain what I said on slide 59?)

Code can be used as a design medium -- that is, it is a medium for design thinking and design expression. Indeed, code is the dominant design medium for software systems. But that is not to say it is the only medium, nor for all purposes the best. -- 1/9/14

How does the architect stay credible, with deep knowledge of key areas of the system, especially when this means not just reading knowledge -- we're embodied creatures, and we learn with our mind-hands working on code? We're also social creatures. So we also get a lot from group cognitive processes, like pair programming. Code reviews and code-intensive retrospectives that are really hands-on, where we look at what we did and have learned -- and for alternatives -- can be great too. Besides, for the architect of systems, there's designing and writing code to analyse the system (code, and partnering with (dev)ops to understand the behavior of the software in operation). To understand how the code structure is evolving, to watch for stresses and strains reaching design boundaries, etc. And ongoing teaching, mentoring, coaching -- for teaching is learning; it is being open to new questions and ways of seeings things, being challenged in a good way, discussing choices and impacts, explaining and re-explaining with a new twist, from a new angle, so that the rationale and solution helps shift mental models and make sense. The architect acts as a consultant to the team, as well as to peer architects and the management team -- which means listening. Really listening. And figuring things out -- what is intended, why is that sought, what does it get, how do we get more what they want, but also balancing with what others want. Etc. Demonstrating to the team, and the extended team, that there is value in the other things architects do, beyond writing code. But also writing code, where the architect judges that is what needs their attention. And writing code as a matter of discipline, because hard problems are hard, and take expertise. Not all one thing, or another. Judgment factors. And factors and refactors. Refactoring is good learning too. ;)

Our teams are social too, so human dynamics and power plays and all kinds of crud, along with good stuff, factors. Credibility can be easy to lose as the architect's responsibilities broaden, especially if someone has an agenda and something to gain from sticking points in the architect's back. "Goodwill is a silver bullet" (Dana Bredemeyer) is shorthand for the people factors factor. In great, awesome, productive ways, unless we lose the plot and malfunction together.

The opposite of a vice is not a virtue but a different a vice. -- @JohnDCook
"Virtue is the mean between the extremes." -- Aristotle

Anyway, this is what I wrote on the "does the architect need to code" question back in 2006. I'd say it somewhat differently today, since we have different antibodies -- like "raise the productivity ceiling" might irritate the "Taylor era" rash we've developed. When really, what I meant then and mean now, is enable the team by removing obstacles, helping to ensure the team (or teams) gets what they need, identifying challenges (organizational and technical) that need to be tackled (not necessarily by the architect, or the architect alone). Having responsibility for making sure the system needs are being surfaced and addressed, doesn't mean we make the architect "god" -- if we do, Sisyphus is the more likely result of our attempt! :)

Sometimes, we do big things. Sometimes we do even bigger things (an airliner, an OS, a car!)! Sure, at any scale, we sure apply the discipline of "how do we make this smaller?" -- what does each feature add? each customization option? Is it worth the complexity cost we and users will have to carry?

There is a whole landscape or team and organizational cultures out there -- some more hierarchical than others. Different system and organization complexity, different demands, different team choices in the light of what they value in their architect, and what that architect brings to the team and organization. I have worked with some software architects who are very effective and valued even though they don't have direct code responsibilities on the project. I am not advocating that for every team (obviously), but I've seen cases where it works. So I'd just say it's a big world and pontificating about what other people and teams must or ought to do, is neglecting their abilities and choices. I can point to upsides and downsides and caveats and forces to weigh. I can bring experience and counter-experience to the table. But I'm not going to insist on imposing a righteous notion of how things ought to be done on a team that is working really effectively together! Where the guidance is worthwhile, though, is when a team or broader organization is looking to get a better handle on system architecture and design, and how to enable that interpersonally and organizationally.

"Nothing is an absolute reality, all is permitted". - Bartol (via mfeathers)

"they both strike the balance in different ways." -- Glenn Vanderburg

2/11/16: Two other perspectives:

'The really good architects I knew received genuine respect, authority, and often affection within the team. You could not imagine operating without their sage advice, and they were ready to pitch in when an especially nasty problem emerged. When I first met some IT architects, who lived outside the development teams, my first thought was, “Why would you ever isolate yourself that way?”' -- Tom Grant, Technical Architects With Dirty Hands

and

2/24/16: This study is interesting for the question it asks:

In that article, what are called "architectural tactics" and then "tactics" (which is a confusing conflation), may be instances of architectural mechanisms, in the sense I've been using (in various traces this month). (Where "may" is qualified by significance to the system under design. It depends, for example, on what is brought within the scope of the system, what is on the boundary of system scope, in that the system interacts with and relies on a mechanism, but doesn't implement it, etc. In the latter case, there's a technology choice and boundary-spanning interaction that's likely architecturally significant. etc.)

At any rate, the study is making the case for strong design expertise when it comes to (design-coding) architecturally significant mechansims. It is an area that could stand to have more empirical study, but I generally think of our experience across many many projects (those of use who get to work across many) as being the first level of empirical study -- not the end-state, but a good basis for finding interesting hypotheses and strawmen to put into more scientific contention. :) And my experience tells me that smarts nuanced with experience tends to produce more robust designs -- tends, but anyone can amp up their design ability by adopting more discipline in terms of looking at a design from various angles, pushing and probing at it, trying things out, reworking it. It is just that with experience, we've done that a fair bit already, and we've seen how our decisions and choices played out, formulated "theories" about what works and why, and what it gets us. In other words, we've connected more dots from properties to code, been slammed by perspectives we neglected to take, grown a bit of wisdom of the sort that seeks to understand more of the multiplex of what is going on, needed, and how to make that be more the case.

Good, Right, Successful

Years (many!) ago, Dana Bredemeyer introduced "good, right, successful" as a way to talk about the architect's role in helping to ensure that the architecture is:

  • technically good (conceptual and structural integrity and code-focused properties like understandable and adaptable play into the tradeoffs being weighed and resolved, etc.),
  • right from the standpoint of fit to context and to purpose (contributes capabilities stakeholders value with properties they care about) and
  • successful from the standpoint that it is delivering value, in a sustainable way (for the team, for the users and their stakeholders, for the business and its stakeholders not just in the short term but taking a longer view, and for the broader business and environmental ecosystem).

The last, is speaking to the organizational dimension of the architect's role that is missing from "right system [right], built right [good]." When I think of that organizational dimension, so many things vie for my attention! It is very much about ensuring that conversations that need to be happening are happening -- not always initiating them, nor always helping to focus and navigate them, etc., but ensuring they do happen when they wouldn't otherwise (at all, or in time), and guiding when needed. Which gets me back to another image that derives from Dana -- that of left hand right hand shaping. Where left hand refers to longer view, more strategic and cultural shaping, and right hand is more immediate, tactical. Yeah, culture is hard to pin down in -- if we try to steer it and make something happen deterministically in culture, we're probably going to mess other cultural things up that in turn mess up what we were trying to accomplish... Still, our behavior impacts culture. Where I mean behavior loosely, so that it includes talky things like design meetings or talky-do-y things like mob- and pair-programming (with as needed white-board storming). Where we focus our conversations, as in what we repeatedly draw attention to, illuminate through metaphor and story, etc., influences our behavior. So some of our "left hand" work may be this kind of shaping, these interventions that obliquely influence culture by influencing what we focus on, what we tolerate and don't, etc.

"culture is not a static ‘thing’ but something which everyone is constantly creating, affirming and expressing" -- Mary Douglas

2/26/16: This is wonderful (thanks Paul Harland for pointing it out):

I was struck by the importance of psychological safety and all voices on the team having "equality in distribution of conversational turn-taking" (over the course of work days), rather than one or a few people dominating conversation. Also, the importance of informal content, for establishing comfort and trust. If there are negative consequences to voicing ideas and concerns and so forth, that voice will be hushed, and the team loses that balance.

Here's here are key takeaways from that research, highlighted by Julia Rosovsky in The five keys to a successful Google team:

  • Psychological safety: Can we take risks on this team without feeling insecure or embarrassed?
  • Dependability: Can we count on each other to do high quality work on time?
  • Structure & clarity: Are goals, roles, and execution plans on our team clear?
  • Meaning of work: Are we working on something that is personally important for each of us?
  • Impact of work: Do we fundamentally believe that the work we’re doing matters?

2/28/16: More on the architect and architecting via Glenn Vanderburg:

Also,

3/23/16: I mentioned credibility under the topic of "architect also codes," but there is another factor at play too, which has to do with "skin in the game." If the architect is removed from the pains of refactoring, of working on improving the code in addition to responding to not just new features, but changes, and changes in direction, resentment can build. Sure, this is in the relationship space and there are different ways to build and retain great relationships that don't necessarily mean taking part in the grind of dealing with aging "legacy" parts of the system. But we are human creatures, so mucking in and feeling the heat, living by example of cleaning up messy parts of the code, can help build goodwill and be an opportunity to share know-how and know-when and -why.

3/23/16: This observation about microservices at Spotify is a reminder of the conservation of complexity:

"To conclude, if you have microservices and scale, whatever scale means. It means you have a lot of small things, each one is easy to understand individually, but the combination is hard. How? What’s the big picture? What are even the things that you have?" -- Peter Mahlen (from the conclusion)

See also:

Architecture Workshops

Come take one of our workshops, and tell all your collegues and friends!

ConwaysLawImage (right) source: Melvin Conway, How Do Committees Invent? 1968

Conway's Law

First, Conway's Law:

Conway’s Law: “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations” — Melvin Conway, How Do Committees Invent? 1968

Here are some insights from a post I wrote on Conway's Law back in 2008:

Conway’s Law paraphrased: if the architecture of the organization is at odds with the architecture of the system, the architecture of the organization wins.

Conway’s Law implication: managers are architects of the organization** and they need to partner closely with architects of the system to ensure mutually consistent, synergistic structures (homomorphic) [and mechanisms with associated structures, behaviors and properties, etc. -- the stuff of architecture that isn't just about structure]

Learning how to ask the question 'What does your code need?' -- Lynn Langit (pointing to Michael Feathers on Symbiosis)

** That is, to the extent that managers influence and shape roles and responsibilities and communication mechanisms and paths, etc., it is useful to notice that managers are architects of the organization. Of course, as with other architects, there is a spectrum of possibilities in terms how that architecting is approached; in shaping the organizational structure (at various -- potentially dynamically shifting -- fractal scopes) and dynamics, managers-as-organizational-architects can focus more on shaping the context that enables more emergent self-organizing, rather than determining team structure, individual responsibilities, and interaction paths, fait accompli.

"Structure determines what behaviors are latent in the system." -- Donella Meadows

More?

How about this, titled "Strong Opinions"? ;)

Or, this from another musing on technical debt:

Developing software is transforming a fiction in our heads into something that makes stuff happen in the real world. More complex fictions, more heads, more hands touching the fiction into a realization. What could go wrong? (A tired meme, I grant, but only because it applies so liberally.)

In saying "I'm done with tracing," I demonstrated the value of what I do, just as surely as the silence on what I've done, proves that it has no value. The conundrum of being a me in this field! :)

“You live out the confusions until they become clear.” -- Anaïs Nin

 

2/9/16

Schmonz Appreciation Day

If you're internet-familiar, or better still, f2f acquainted with the Schmonz, you no doubt appreciate some or other facet of the awesome he brings to this world. Here are a few:

  • his Agile in 3 Minutes podcast, which is a wonderful way to take Agile at its word, pushing 3 minute nuggets of high value to its audience on a regular heartbeat (and the book of essays from the podcast -- his voice is awesome, so listening as the podcasts come out is a real pleasure, but the book gives a mechanism for reference and speed reading back over a topic)
  • his blog, which reveals both a lot of pragmatic mindfully-tried-and-practiced (I first said lovingly-worn hand-me-down -- because Schmonz has this wonderful quality of being susceptible to advice and wisdom, taking it on board, making it his own, then making it better and handing it on) guidance and insight, and a person who is real (life deals some stuff we don't seek, we seek some stuff life is intractably stubborn about, and so forth), and a heart-cheering life-affirming wonderful, talented, insightful and questing-seeking-understanding geek's geek (in the best of best ways)
  • his music (he composes and plays) which you can introduce yourself to here

Raising a glass of virtual bubbly in celebration and honor to you, Amitai. So glad you're sharing this little ride on spaceship earth with us!!

Schmonz is ma homie, but I shielded him from actually meeting me for couple of years. But it was a lovely coffee break that extended over 2 coffees. :)

Um...

My sentences are hard to read (hey, GeePawHill done tol' ya, stop yer yabbiting!)? So are people (hard to read because multi-faceted, dynamically shifting and adapting to varying and various contexts), people. We have to allow a little unfolding, no? Read me, and FP will seem trivially transparent. ;)

Also, I should use a spell checker to get my back on typos? You're so nice.

 

Susceptible?

I keep using that word? You don't think I know what it means?

Well. Actually. ;)

I mean it playfully, in the sense of being pervious. So many people are impervious -- not just to me (which is, naturally, understandable), but (me pointing to) to others, to ideas or ... playfulness outside their brand of playfulness. :)

2/10/16

Strategy and Tactics

"THE distinction between strategy & tactic is time: strategic means longer than tactic. Understanding strategy means understanding time." -- Luke Hohman

Well, that's not how I would put it... Strategy is about determining how we will compete (and if competing is the dominant aspect of how we play the game in the first place), so strategy has extended focus not just in terms of time, but takes a wider lens on the current context (ecosystem, emerging trends and opportunities they yield to create new kinds of value, threats, etc., etc.,). Tactics are plays within the context of, and shaped by, strategy. It is, I think, worth noting that an organization's strategy may be entirely emergent, ad hoc, accidental, "shoot first and aim later" kind of thing -- in which case it is simply the emergent thrust that comes from all the tactical action taken without explicit, mindful strategic course setting and context shaping. Alternately put, strategy then has no longer nor wider view than tactics (taken together). It is the degenerate case ... in the limiting case sense -- but can also be in the devolved sense too, though it can be quite effective, or at least effective enough.

Okay. I fixed the internet. We can go back to work. Speaking of which, want to do an Enterprise or Software Architecture Workshop with us? We are awesome at everything from strategy to mechanism design so, you need us. You really do. :) I mean, I can show you how Wardley Maps are great (think capability maps, then also think about where each capability is in terms of evolution towards dominant design and commodity/utility provision) -- and why there is no one map to rule them all, no matter what residue the echochamber leaves you with. There, fixed another problem on the Internet. ;) In seriousness, there is no one architecture workshop to rule them all either, but damn, ours comes close. Just kidding. But it is awesome. Really!

 

2/11/16

Collegiality/Camaraderie and Mutualism

It seems to me that many of us who are/remain active on twitter, are independent consultants and contractors. Of course, we, like architects and developers within larger companies, are on social media to keep our finger on the pulse of change and to tap into serendipity, so to speak, learning about this or that piece of work that busts us out of our thought canals. And so forth. But independents have some unique needs in terms of camaraderie (client engagements may be matters of days or weeks, rather than years) and reputation development (branding, as a path to gaining client access, to pay the mortgage and school bills). Because a lot of us are in the same boat, we tend to act competitively... which is to say, we don't foster one another... We aren't very good at reciprocating, let alone "paying it forward," doing something nice to help one person gain more audience, without any expectation of a return gesture.

But we don't have to orient competitively. There is so, so very much to be learned and done to enhance our effectiveness in creating great software and systems and/or helping teams do that, that we can orient towards one another collaboratively, mutualistically expanding the field rather than competing. Simon Brown and I, for example, may seem like direct competitors because we both emphasize the visual in architecting. Still, we mention each other's work and workshops because the bigger competition we face, is simply the drag against architecture and visual modeling as a medium for architectural expression -- thinking, thinking better, communicating, communicating better. And so on. We already know this, but it is still worth looking to Elon Musk's orientation around "All our patents are belong to you." The ecosystem needs to be grown, and that is bigger than one or two or even many many of us can handle. The ecosystem? All of us involved in creating-evolving software, and getting better at creating great software. Figuring out how to create software that meets needs, and stands up to stresses and strains as demand (features/scope, and numbers of users) grows.

"Indeed, the more people talking about architecture the better. " -- Simon Brown

I also mention or retweet work in our field to help tend the attention commons. To lend support to ideas and their people. It's good to have this cross-pollination happening between architects, developers, coaches, managers, people in other fields, and more. And it is great for independents doing intervention-styled consulting work, because we can be isolated from ongoing, longer term peer networks unless we make it happen for each other. We are social creatures, and we need some ongoing, longer term, social embedding.

So, in short, it's great when we play nice. And arguably more important to do, than to guard a little corner of the field.

Use it, or lose it, as they say:

"Slack will obviously replace Twitter." -- @noahsussman

2/24/16: As voices in the architecture conversation go, here's:

Rebecca reiterates Martin Fowler's characterization of architecture as "things that are most difficult to change" or "the things the organization is most worried about," Those factor; I prefer ours (smile!), naturally. That is, when I say structurally and strategically significant, it's scooping up and including Martin's change and worry stuff, but also going after questions like
"what are the driving concerns and forces, and how do we respond to them?" Change is not the only shaping force impacting design integrity. Resilience (or integrity under duress, over time, even as context shifts) is a factor, so we also seek to reduce the cost/difficulty and consequence of change where we deem strategically significant. But mainly such characterizations miss architecture's most primary role -- enabling the system to do what it does, under the stresses and strains it is and will be subject to. That is, it must enable the desired capabilities of the system (even as we figure out what is most valued and valuable and core and so forth), with (more) the properties stakeholders care about. Anyway, more here:

 

But, but, but

I wanted to write a book called "Getting Past But" with a picture of an elephant's rear end on the cover. I didn't entirely fail, in that I did write an EA executive report (think short ebook) for Cutter by that title. And, if I say so myself, it is delightfully consistent with its title, evading all the usual "buts" like "But, you can't use a children's story in a paper intended for IT executives and architects." I would write it differently today -- but heck, I'll revise what I wrote this morning tonight, so it's just a sign of updating one's understanding. Spin it, Ruth, spin it?!?! There is a lot in it I still like, including the use of, and generally speaking my debrief/discussion of, the children's story; The Wheel on the School is well read as a charming book on dynamic agile teaming.

Now, you need to understand that I very, very -- and I do mean very -- seldomly get paid a compliment. It seems that on the occasions that people engage with me about my writing, it is to tell me what I missed, or misstated, and that is awesome, because it improves my thinking. But the affirmation is indirect -- it is not nothing to be taken seriously enough to be read, on occasion, by someone, and you at that!! So it is an implied compliment. I say all this by way of context: I am no expert in the matter of receiving compliments, and when I do get paid one, my deflector shield pops up automatically in the form of a joke. Like this: "does my but look big in this compliment."

But! Eugene Barker paid my Do we still need architects? trace a compliment y'all and you have to see it!! Look:

'Great description of what us architects do, and more importantly *why*' -- @arctips

 

but

That's from 2008. I've been around all this time, and you didn't even know! And here's a (related) trace from 2009:

3/10/09 Curiosity

The simplicity and sheer brilliance of "wonder why" floored me when I encountered it in The Wheel. The thing is, curiosity is too often a lowly denizen of our corporate worlds. Now, you may be familiar with the "5 why's" problem-solving and root-cause-seeking approach of the Japanese quality movement (developed by Sakichi Toyada, and popularized in application at Toyota). The use of "wonder why" in The Wheel has a different emphasis--it is to get past assumptions that gate possibility, that cloud our options. It is the essence of curiosity. "Why are things not working?" is, indeed, a path to opportunity, and shouldn't only be used to find defects!  And there are many more uses of the simple strategy of asking why (and what if, and what else). The technologist, if she is to bring opportunity to the strategy table, has to be insatiably curious. Curious about how things could be. And curious about why they aren't like that, and about the opportunities that opens up.

The children in The Wheel thought about their world, and something they wanted in it, then wondered why it was not so. And the wondering why, was powerful, magical. We need to learn from the 2 year olds' tendency to ask why, incessantly. You can say you learned it from Lean. I like to say I learned it from a children's story. In truth, it was something I already knew, but my mental models shifted (like the last few moves in a solving a Rubik's cube) when I read The Wheel, and I saw it as an easy mechanism to use to focus curiosity and turn it into a powerful business tool.

Of course, you don't suffer from a deficit of curiosity since you're reading here--but goodness me, you could use a mechanism to channel it! Just kidding! I know my journal has its value as a natural soporific, if nothing else. So naturally you'll be curious to explore further, and you can--entries from previous months may have been written in the past, but for the most part they are just as (ir)relevant today as they were then. Links to previous months are in the sidebar, along with links to other blogs and journals.

If you're curious, the history of "curiosity killed the cat" on wikipedia is interesting, harking back to a play Shakespeare performed!

 

Legacy Switches

Ooo... that sounds like a trace I should write! Feature toggle, but on the trailing end. ;)

If I was tracing still.

I am? Who knew? ;)

What's that? Michael Feathers already wrote that? You mean this:

It is a post worth engaging with, entering into discourse around. Share ideas and practices. Microsoft has some experience with trying to turn off legacy OSs, and that's hard, hard, hard, due to entangled dependencies. It's worth visiting the question though, in the light of microservices. (Don't groan.) I have gotten wind of some who do view DevOps and microservices as the secret sauce in which features are not just brought to life in bounded experiments, full roll-outs and operated through scaling and other challenges, but also decommissioned or end-of-lifed by DevOps, as new versions are brought into the lifecycle. Having "fit-in-our-head-sized" chunks of code that we manage birth-to-death has its appeal, but we know the challenge -- scope creep at the microservice level? I'm thinking that too. And some other things, that I'd write if I was tracing, 'n all.

As a side note on that post, about IT as a cost center: When IT is a support function to a support function (like HR and Payroll, Accounts Payable, etc.), it is readily viewed as a cost center. And that is the legacy framing for IT. Even when the software-intensive systems are the revenue engine of the company, this may have snuck up -- insurance companies, for example, may not conceive of themselves as software companies even as much of their customer interaction moves to digital. Data warehouses are well behind the corporate wall from the customer, but analytics turns the crank on personalization services. The boundary between "obviously" and traditional cost centers, and customer-facing, revenue-generating services, gets blurred.

See also:

  • More (Eco)System Goodies -- disjointed and more suggestive than those who don't speak Hint would be comfortable with, but hey, it's Tracy in here.

And this -- along with the resources at the end:

2/18/16: Python has EoL schedules?

2/18/16: IT as an enabler not just a cost center? See Fractal and Emergent -- yes, you have to give them you contact info to download it free. But please note, you have to do that with LeanPub or Amazon AND you have to give them your credit card info. And yes, it is worth your contact info. It's okay that I wrote it in 2010 -- you only just caught up to where I was then. JUST KIDDING. Sheesh. But I do still like a lot about it (and stuff I cringe at, but hey... I could make you cringe at stuff you wrote today, so let's not go there ;). So there's that. And it's FREE -- well, except for the price of your contact info that we just talked about, and the price of your time reading it... which is an investment. View it as an ENABLER, not a COST. Wink wink. Okay, Okay I have to quaint so you don't kill me for being assertive. The boatanchor we make women swim with, in a world of dominated options... Blah blah.

3/23/16: More on killing code:

And killing projects:

Cost of carrying code:

"It is not that uncommon for the cost of an abstraction to outweigh the benefit it delivers. Kill one today!" — John Carmack

 

2/14/16

Does the title "architect" matter?

"An architect by any other name".... Does the title matter? No. Unless it does. ;) In some situations, the title "architect" has been tainted or sullied, so "senior dev" or "tech lead" or some such -- or even nonesuch -- is used instead. Or we're figuring out how to be more egalitarian, and doing away with titles that implied, or full out meant, hierarchy. I'm down with that. We have to deal with what we have to deal with, and we need to experiment with new organizational ways of being and doing, to progress. If things are going well, why mess with it? And if they're not, we may have more pressing things to deal with... But. It may just be that we need an architect! That we need that strategic and technical leadership and system design mastery and focus.

"Architecture and discipline are the 1st to go during stressful times. And yet are the two things that, when lost, incite needless stress." -- Joe Duffy

The title isn't about making all the decisions and taking the credit for the team's work. It is about a role that works to enable the team and to watch for what needs to be enabled, and what needs to be ruled out, out-framed, constrained. This has a strategic dimension to it -- doing things that really matter; doing more of the right things, and fewer of the wrong things, not by magic, but because experience and strategic sense. ("Because experience"?! Look, I write loosely here, to convey that I haven't hardened my thinking into concrete. As those who interact with me know. Thanks @arctips (Eugene Barker) and @brainwork (Peter Bakker) and @bmichelson (Brenda Michelson). And Dana Bredemeyer, obviously.)

Now, I would suggest, the title "architect" indicates charter and opens up the scope of contribution more than "senior dev" -- both have that dimension of technical expertise drawn from significant coding experience, along with smarts, investigation and curiosity. But architect throws the emphasis onto system design concerns, and, we argue anyway, playing a role in bringing technical considerations to the strategy table (at the level of scope the architect is working at -- product strategy for product architect, business strategy for enterprise architect, etc.), and strategic considerations to setting technical direction and shaping the course of system development. I think we need senior devs -- we need a career path that allows technical people to stay deeply, focusedly technical, and be valued for the mastery they develop. Hard systems things are hard. We need people who are really, really good at technical inventiveness (in small local problem-solving ways as well as large industry/landscape reshaping ways) and solid, elegant coding. And we need architects, who have the credibility in the technical community that comes from that technical mastery, but who also want to lead -- want to work by enabling, organizationally as well as technically, to be strategic, and lead architectural (system focused) design and do the work -- the real and ongoing work -- that entails.

Recall: the system needs to be attended to as a system. For emphasis -- if this is not someone's responsibility, it is no-one's paramount responsibility, and it will lose out in the day-to-day mull, to more local demands. If we focus on parts, without system design context, we are in danger of ending up with "parts flying in formation, trying to be an airplane" (Wim Roelandts). Some teams, especially small teams with lots of organic communication and mind-syncing (over a mindful of code), can navigate this balance, holding in creative suspension, and resolving the tension among, the demands and forces on the system -- and move work forward on the parts. The team really can be, and is, responsible for the integrity of the system. We have to consider how this scales, though, for more complex systems, of multiple parts, features, microservices, whatever the chunking du jour, ...

Further, when something has much to recommend it, but it isn't getting any traction, something else is going on (hat tip there to Dana Bredemeyer). Sometimes, [a figurative] someone has soiled the bath water and we're wanting to toss the baby out with it. In this case, in our binary zeal, we tended to collapse we "value x over y" into "x not y" and tossed architecture under the "comprehensive documentation" bus. I'm oversimplifying our tendency to oversimplify. :) But some of what we've had to deal with here, is the notion that architecture is entirely/best left to be emergent -- from (and documented by) TDD and the code itself. We course-corrected back to "well, actually, some architectural thinking-designing-deciding is important." But we've left some in our field back in the wake of "architecture? not so much," and they'll have to learn some lessons the hard way, and make their way back to "architecture?!--how much?"

Still, some of the push-back against having an architect is more close to the team -- it is political...as in, about decision power and tipping power balances in the organizational landscape. It is worth asking ourselves, in tossing out the "architect" title, are we trying to obfuscate an unwillingness to let someone (else) lead the system design effort? More than one person on the team may have the technical experience, system design talent and organizational effectiveness to be the architect (or tech lead for a subsystem), but it is still useful to have a designated (lead) architect, to break logjams when different alternatives are being advocated, or we just need to move forward, and we don't fully know yet what the "best" avenue of address is. The (lead) architect also serves as a boundary spanner and emissary of sorts, interacting with other architects and executives, on strategic and cross-system, or system-of-system, matters.

I'm fine with everyone on the team doing architecture -- it means more people should take our workshops! ;) But complex systems are complex -- technically and organizationally. Much system understanding is moved in interpersonal conversations, but also written expressions like documented architecture decisions and code (and tests) --- and mulling and mulling over those, imagineering what they mean for where the system needs to go, sketching prototyping and coding this or that throw-out experiment, writing up this or that mechanism or other architectural decision to get it expressed and improve it, making sure those who are differing on a matter of architectural import talk it out and resolve it, and more. Again: this is work, real work! Which takes time. Real time. And the ability to discern where to spend time.

"Architects need to know what they're doing, and *get things done*!" -- Dana Bredemeyer

Many are involved in various aspects of the design and coding, testing, deploying work, but it helps to have a locus of system responsibility to make sure that the system thinking-designing-deciding happens. Does everyone think they can, should and want to do that? Often when push comes to meetings, some want to protect their coding time and let someone else do all that listening and reading. And those who are more senior, have more hooks to hang new learning on -- they have more learning and understanding under their belt. They're at a different stage of integrating a lot of experience of systems -- technical and organizational -- into the understandings and abilities we associate with mastery. Which for an architect is active and ongoing, constantly learning everything from what's happening in the business domain, to investigating technology directions, emerging technology, reading academic papers on mechanisms that look tantalizingly like they might solve this scaling problem or that consistency challenge, reading code including other systems which have done things elegantly, or are hitting some of the same challenges, etc.

Creating value is everyone's responsibility. Quality is everyone's responsibility. Architecture is everyone's responsibility. Yes! But we can still value leadership and a center or locus of responsibility, while at the same time knowing and appreciating that value, architecture, quality is everyone's responsibility, with everyone participating and contributing. Knowing things takes time and attention. So does doing. There are lots of avenues of learning, knowing, doing -- the value network and competitive landscape, user experience and design of system capabilities, system design and code (with tests), integration and system testing, evolving the deployment paradigm and infrastructure, data, oh just ever so much to keep up with understanding the state of, and inventively evolving, to add value and sustain the business. We can't all know all the things and get serious stuff done -- in competitive timescales. Let's take another setting where the work of many mind-hands contributes to something greater than one or just a few can accomplish -- an orchestra. There is no sense that the conductor accomplishes, nor gets, nor deserves all the credit for an orchestra's performance. It doesn't have to be spelled out to us. For groups of musicians larger than the number who can reasonably directly see one another, a conductor is assumed and appreciated. That is true, even given a score and much practice together! So we can argue that we should only do small things -- only have small jazz bands. But orchestras contribute something different. We value that too. Not instead.

If we value mastery, then might we not value technical mastery and effectiveness in leading technical teams? It takes focus and an investment in personal development to develop this extensive and deep (business and technical domain) knowledge, awareness, insight, wisdom! And it benefits the team (of teams) to have this mastery in creating-evolving systems with integrity (structural and design integrity), agility (responsive to change, including sensing what to respond to) and sustainability (in all it's senses, including economically, technically, organizationally, and environmentally). A good architect -- one who really supports the team, mentors and grows the team, and facilitates their success, but gets out front and leads where that's important -- is an asset to the team, in the sense of magnifying the value the team is building.

Magnifying, by helping the business team shape strategy, so that we do what matters -- it differentiates but also contributes value -- and we don't do what doesn't add substantively. [Again, here I am talking about the business team at the level of scope under consideration -- if we're talking about a product or service, then the business team at that level; obviously a product architect isn't working with the executive team, unless it is a one product company.] Strategy demands situational awareness, and technology is part of the context. Architects bring not only knowledge of the state of our current capabilities and systems, but also what we can leverage, to create, shape, and respond to market opportunity. [Aside: One of these days, you will have the good sense to ask me about slide 91 and 93, among others. ;-) But I'm still waiting for that someone who has read the slidedoc e-book, to ask me to go on. I mean, reading 57 pages and paying a few compliments is expensive compared to paying $15 or $30 for a work in progress book on LeanPub, but someone must be willing to pay that price? No? Then I'm not willing to write more. I think that's a fair way to set up trade. It's all about my ego, anyway. ;-)]

We have created these mythical monsters under the appellation of "architecture astronaut," or similar, who are disconnected from project reality yet operate in a highly authoritarian way. Some of us may even have glimpsed such monsters, but usually the reality is more nuanced and textured than the stereotype belies. When you investigate what architects actually do, when they are effective, and trying to be more so, they aren't psychotic authoritarian figures nor highly manipulative authority abusers. Why not? Organizations are political (in good and bad ways) -- we are social creatures and work within webs of formal and informal networks (including hierarchies), and in tech we just tend not to tolerate autocratic rule...for terribly long anyway. We may agree to follow a "benevolent dictator" but much is conveyed by "benevolent" -- if the dictator is acting in ways that disserve the system or its people (developer or user communities), then implicit agreement to follow starts to break down. Smart people are smart. Loyalty and following a dominance assertive leader certainly factors (we're human), but failing to serve raises opportunities for other leaders to emerge. So even at the extreme, extremes don't tend to last.

Creating straw man architect stereotypes is not cost-free fun -- we simplicate real people into the bucket of the stereotype, and make it harder for them to rise to the occasion of their role. It is an intrinsically challenging role. Non-trivial. Strategic. Deeply technical. Organizationally demanding.

The effective architect, works together with others, leading, following, and getting out of the way. Doing a mix of coding, mentoring and pair problem-solving/programming, scouting and investigating, deciding and documenting or simply watching to ensure that gets done, working across the organization as fits the demands of the moment, and more.

We need to do the right things -- a strategic matter that is not just about user needs. And do them right -- which takes technical and organizational savvy. Organizational, because it is not about doing everything or even the coolest things oneself, but growing the team, enabling, coaching, and sometimes leading and showing. As meets the needs of this extraordinary moment.

This trace wants to become my Trace! Here are some snips from past traces:

From 7/2/14:

design*

    • establishes (understands and ascertains what is relevant in) context,
    • understands and articulates/constructs intention (desired outcomes, capabilities and properties),
    • surfaces grounding assumptions, makes assertions (what we hold to be true) and projections (seeks out trends and shaping forces)
    • identifies challenges to be addressed
    • negotiates boundaries (negotiates/assigns shapingly significant roles/responsibilities, makes trade-offs across boundaries)
    • designs interaction surfaces and interactions across the boundaries to achieve the intention, and
    • addresses how the intention will be delivered (and associated challenges tackled) *

    That is, for each design frame, we consider the chunk under design in more "black box" terms, considering its context or containing system, framing its intent (purpose, capabilities or responsibilities and properties or commitments to the containing system), boundaries and interactions across its boundaries, before -- and then iteratively as we learn more -- we get into the matter of designing the guts of the chunk.

Aaaand. As awesome as that is, and all that, it is also wrong. ;-) Well, wrong if you interpret it to mean hierarchical decomposition. There is some aspect of that, but (technical) wisdom applies, and design/technical wisdom has multiple facets. Among them, is insight into side-effects and action at a distance (spatial/organizational and in terms of time), with implications for priorities and significance, and timing and what needs to be tackled sooner and what can be deferred without undermining strategically significant outcomes. You know the refrain: Judgment factors. And factors and refactors. There is working different facets, there is back tracking, considering alternatives and trying different things out, there is sometimes working in the details, and sometimes drawing back out to design at larger scopes of impact, influence and interaction.

Aaaand. If that weren't enough. The very intention changes, as we learn what is possible and desirable. Sometimes more. Sometimes less. And in all this, we need to get ground under the feet, so to speak. Gain traction. Make something that provides value, makes a contribution.

Design ascertains what it is we want to accomplish, and how to do so, addressing the inherent challenges -- technical and non. It is not a linear process. It is not a do-once sort of thing. There is a lot of discovery -- the interaction of the thing with its containing system changes the containing system(s), and hence what we want from the thing. And hence it's inherent challenges. And so on. Messy. Mixing creativity and engineering, working towards creating value, but also engineered integrity -- structural integrity and resilience, adaptability and agility, sustainability. And design integrity, harmony. Even, where it matters, delight.

"To harmonize the whole is the task of art." -- Wassily Kandinsky

And architecture, as system (of system) design and art.

and this on strategy from 12/28/11 (the whole trace is probably worth returning to, but here's a taste):

Strategy explores the organization's context [the competitive landscape, the value networks, the ecosystem(s)] to find and assess opportunities to create differentiating value (and avoid threats). I'm not entirely happy with "strategy shapes the context; design shapes the system" but I think it touches on something important. Strategy assesses opportunities and decides what to design (create or evolve) and how to compete (including whether to compete by dominance or by creating collaborative relationship networks that thrive through mutual interdependence), versus what to influence and what to accept in the ecosystem. Strategy assigns resources and provides other enablers (such as the creation of strategic relationships). Business strategy works at the scope of the ecosystem(s) and even beyond, and can set processes and forces in motion that reshape the ecosystem or create a new one. Business design works at the scope of the business in its context -- responding to its context, adapting to it, shaping changes in it, to be sure. Strategy determines what and how much to take under the organization's design scope (or "umbrella"), articulates desired outcomes and (broadly) shapes approaches to achieving them including choices about what capabilities to build/evolve and where to allocate resources. boring

 

I need to pull these threads (on the architect role) together into something more punchy. The image alongside was Sara's response to my Trace back in 2011 when she was 10 or thereabouts...

Hey, no need to agree with her -- my inner voices have already danced all over my ego enough as it is!

I mean, sheesh. Two people summoned the kindness to say something about my Trace on the grand occasion of its 10th birthday. Let me repeat that for effect: TWO people. It's a lovely world, isn't it? But get this. We aren't all that good at celebrating people on their birthdays, so my Trace is in good company. Gosh, we could be nicer to one another! :)

 

People who give credit, who mention others who have influenced them positively -- they do good in the world.

 

 

2/16/16

(de)Centralized

Still munching on that "do we need architects?" question. We learned that solely relying on emergence of architecture from disciplined refactoring and TDD -- with a strong test driven design flavor added to the usual TDD mix -- may not be the best route. But now in the microservices context, we have a fix for that in the form of DDD, so we're good on design, right? No architect needed. Next!

Not so fast. Architecting is about a lot of things. Not just decomposition (to tackle complexity, with all that it implies cognitively and organizationally). Not just composition and orchestration. Important as that is:

The conservation of complexity point, reminded me of these, by-the-by:

Architecting is also about identifying and addressing technical challenges arising from system properties, interaction with context, more. It is about the design of mechanisms that resolve various interacting forces on the system to achieve more the desired properties and less of the undesired. And it is about sensing and identifying what is strategic -- in business terms, and in technical terms. Strategic, as in: How are we differentiating ourselves? How and where do we add value that is valued (as in, people are prepared to not just spend money, but integrate this system or product into their work or personal lives)? Is this approach sustainable -- for our internal development community as they live in this code base for years, and years; for our operations team as they scale and respond to snafus (I have a story!) and more; for our users as they live with the system and its entanglements into their systems, impacting their days and lives; and the planet more broadly. Have we added positively to the world? Or is the net effect meh, or worse, incurring negative externalities that the planet has to pay for down the line, with nothing more to show for it than some short-term hoopla... (sorry Silicon Valley; we love you, but some of what you do... you should hold yourself to the wall about, and wonder why! ;)

One of the first things we do in architecture workshops is a context map. Not a map of the infrastructure, or immediate technology context of our systems, but a high-level ecosystem map. (Depending on what we're doing, we may do other maps such as value flows and transformations, etc.) This can be quite uncomfortable for entry-level architects or tech leads, because it makes clear all the space in the landscape they have been relying on others to have a handle on. Up until that point, they've been focused on code (and closely related concerns), which is goodness -- it takes time to gain code mastery, and architects should have a level of excellence there, should have developed a sense of what is challenging and how to approach challenges in code. To be considered for the architect role, they need to be good at problem solving, in code and by being great at problem identifying and defining (tests in TTD sense, but also algorithm designing, and deciding what not to do and...)!! Now, if they'd gone all out on understanding competition, they'd be channeling marketing, not developing code skillz. I'm not suggesting that! But as an architect, we need to have strategic sense, to do more what matters and less what does not. That. Contributing to strategy setting, at some point. And making sense of the strategy that has been set -- we have to have some understanding of the playing fields we're adding value to. Many software design approaches go as far as investigating what users need -- and even that is further than where many organizations let their developers go. In counterpoint, we insist that architects gain that "situational awareness" that takes a broader view (just enough -- just enough, for goodness sake!) even than "what do my users want--right now" (or what does my product owner/manager think they want). Why? Because what we do with technology today, is going to enable and constrain the business. We can try to make our systems more responsive to change, but we need to recognize that our starting points have a tendency to limit where we end up. They canalize not just our view, but our users view. And so forth. So strategic direction setting is non-trivial, and just as important for the relationships we set in place in our technology base, as they are for the relationships we set in place in the broader ecosystem. These things interact.

 

We have long (even published as a paper in 2002!) advocated minimalism when it comes to architecture ("less is more with minimalist architecture"). But Jefferson and Lincoln and other US founding fathers were well ahead:

  • "That government is best which governs least." -- Thomas Jefferson (via Gene Hughson; see also other variations and sources on that page)
  • "The legitimate object of government is to do for a community of people whatever they need to have done, but cannot do at all, or cannot so well do, for themselves, in their separate and individual capacities. In all that the people can individually do as well for themselves, government ought not to interfere." -- Abraham Lincoln

 

So. I wouldn't rush to put all of a field's bets on any single pony, whether it be AWS (good as Amazon is at cloud) or DDD (it's an approach, not a panacea), or TDD or anything. We run diversity out of our field if we all go tribal on the same things, or treat any approach as "best in class" "best practice" "safe"... We end up with only "Cavendish bananas"... in our fruit salad... :) We need a little place in the world for the Bredemeyer/Malan goodness to earn its keep, by helping organizations be (more) agile, by having the architectural wherewithal to flex, respond and adapt, where needed. Remember, we were talking "Fractal and Emergent" in print in 2010 (that long ago? I need to write more!) -- that is a lovely paper. Okay, I'd write it differently today, but it speaks well to 2016 all the same.

2/17/16: Neat idea for architects: need to bring the architecture documentation back up-to-date/in sync with code -- hold a "write-design-a-thon" (hat tip Sarah Allen) Oh, we've done working sessions with teams on architecture recapture since the mid-90's -- but it's a contemporary name for something with deep roots.

2/18/16: Matt McLarty was asking "We're discussing your blogs on the link between designing systems and orgs. Are there examples of orgs who have done this well?" To clarify, I asked if this is what we were talking about:

Designing the software system and its people/development system together?

I think this is in the territory of "the answer to how is yes" (to run off with a book title and use it my own way, to my own ends) -- in other words, Conway's Law says there is an isomorphism, and it will be driven by the organization (structure and communication paths and mechanisms). The question is, are there organizations that use knowing this, to get more what they want from the system, by being more intentional about the organization. And again, yes. We used to do this with modular monoliths and upfront design -- whether the upfront design was by managers determining "pizza-sized teams" based on gut feel (and dominant design of similar systems in the organization or the industry) or architects designing the system and helping managers form teams based on the architecture. I can't talk about our clients. Knowing how smart architects and managers I've worked with are, of others, let's just say: I would assume there are those who do this in the context of microservices, and they aren't just waiting around for consultants like Ruth to tell them -- though Ruth's lovely Conway's Law post has been out there since 2008 and that no doubt has had its impact too. ;-) I would also assume that things get messy, so it is hard to find "clean" examples of anything, over periods of time.

Anyway, I think the impetus of Matt's question is right on -- organizations that think about designing the software system and its people/development system together, are going to have these systems work synergistically. Which is the essence of what Michael Feathers is on about these days. (That's a nod to go over check out what he actually wrote -- it's bigger than what I just gave lie to. :-)

Before anyone runs off with all the territorial flags though. ;-) See for example the 2/3/08 trace that then runs on to a Conway's Law trace. And seeking synergy between business and technology is a dominant thread of our (enterprise and systems) architecture work throughout -- bringing technology to the strategy table (at every fractal level) is something you'll see in all of our papers (and my Trace). We have done this from the perspective of the architect acting as a fulcrum in this relationship between organizational strategy and technical strategy, and organizational design and technical system design, and system capabilities design (what the system offers to other systems/users) and system design (how the system is designed internally). Oh, so now you want the notes to Slide 92 and 93? Anyway, we have been addressing this set of relationships, and the need to design across these systems for they influence and depend on one another. We're still learning, but we have been bringing Ackoff and Argyris and Ashby and Beer and on down the alphabet to systems design for 20 years and counting -- actually more in Dana's case, but who's counting, afterall. And a whole lot of unique contributions from the Malan-Bredemeyer duo, to boot. I wouldn't want anyone thinking we just regurgitate classics and contemporaries, just because they can't imagine a woman (and this one at that!) making novel contributions to a field. ;) Although it is definitely not nothing to bring it with Leadership and Self-Deception. :)

Isn't it just so cute how you had me pegged at quaint? ;-)

2/19/16: Why is strategy so important? Because we are so "yes" about this:

"Delivering working software frequently is the most useful political work I know." -- @RonJeffries

that we can forget, that its very effectiveness can be its downfall! Working software has a way of becoming self-determining -- it sells itself to those who're buying it. Remember, "code wins arguments." What am I on about? Working software, systems, start to shape the world around them. "Systems have needs." (after John Gall, I think) Interests become vested. Views become canalized and dug in. It becomes hard to change. Until the chopping block of competition, or higher level determination that the value just doesn't support the cost.

So, yes! But also no. I know. I know. We experiment and adapt, because lean agile. Right. We still do that within a frame, and strategic choices set that initial frame, and shape the context as the thing moves. And strategy is not just a business matter. Technology is so interwoven with the business, we have to think strategically about technology -- what business capabilities we build, enable, shape, with the systems we build and build on. (You can take a look at our Cutter reports -- the second and fourth are especially pertinent, but all have something to add.) And even when we are years into the lifecycle of a system, or family of services and products, situational awareness (evolution of the competitive landscape, value flows and transformations, and underpinning of capabilities including technology), and direction setting-guiding (what we seek to make true in the world, broadly shaping our approach, enabling that and removing obstacles, etc.).-- that is, strategy -- is still important.

So big yes on working software "selling itself" -- shaping the political landscape around itself. And a big caveat. It can create tunnel vision, and the world goes on around it, making its choices. Strategy gets us out of the hole, looking around, seeing where we want to take this thing, longer range, so we tunnel in a direction that fits in to bigger value flows, not something that is a deadend. We can take a "let the future take care of itself" approach, because it will anyway, right? Can't stop the future! So, why aren't you on at the beach today then? You are? Wait for me!

Let me take this moment to appreciate Peter Bakker! When I set down the path of talking about the architect role in the Agile context, my eye was initially on the system integrity ball. Attention shapes what we perceive and pay attention to. So my eye was not on the strategy ball, and Peter gently, obliquely reminded me -- it's about creating value. Value! Doing what matters! I am so all about doing what matters -- having a broader conception of agility, integrity and sustainability and doing what matters. In my talk in London, I told the audience that I drag architects into the strategy and capabilities (system-in-context) design space, not to leave them there, but I insist they go hang out there enough to know what its about, to contribute, and to bring back gut level insight into the broader systems, the system we're designing plays into. What mattersWe wrote about strategy as an architect competency in the first of our papers that became (and still is!) quite influential in enterprise and systems architecture -- in what was that? 2004? But it leveraged work we'd been doing, and informal papers on the Bredemeyer site, from years before that.

So I know this stuff, but Peter did what the architect needs to do -- he got me to lift my head to see the bigger picture, see the things I was missing.

We need to do things that add way more value than we consume, because we're indebted to this planet and its peoples. We extracted and extracted and messed up this lovely world, and we need to do good, right, meaningful things. Or more of that, than the other thing, where we make more pollution and misery. And other social and environmental yuck. At which point I should mention Stuart Boardman and JP De Vooght, who are two of my look-to's on environmental and civic consciousness.

Also, thanks to JP De Vooght for alerting us to this wonderful paper (it would only have been more wonderful if it had mentioned my precursor -- the tweet with the above image notes that it is from a trace in 2014):

But again it runs deeper:

where I said:

"Sustainable" is a compound of customer value (even excitement or delight), economically viable (a matter not just of revenue stream but costs), technologically feasible (taking into account internal capabilities and capabilities acquired in technologies incorporated in the product and its broader value set), and environmentally sustainable (increasingly a matter of organizational concern, but also of concern to a growing proportion of the customer base). With a time element! That is, not just now and for the immediate term, but as all of this (the value stream, technology, etc.) moves into the future. That is, sustainable is about being able to move into the (changing) future, not just thriving in the present. Though that too.

Which also has history, roots in other insights and explorations. Hey, I've been doing this for a long time. Decades. One would hope it was a journey worth taking, with lots to show for it! A whole Trace worth, as you can see, and working directly and indirectly with thousands of architects around the world -- learning from them, stealing their fire. :)

"Move fast and break things" isn't quite so great sounding when it's the business (and people's livelihoods) or safety (and people's lives and livelihoods) or privacy (and people's financial stability) or the environment... Software is high consequence stuff, when it's high consequence stuff! And it so often is, even when we haven't given it much thought -- or especially then!

2/20/16: reminded of Dave Gray's interpretation/presentation of Argyris "ladders of inference" and "self-sealing bubbles."

3/8/2016: This is interesting (on Medium and their experience with holocracy):

"At the same time, the system presents its challenges. Our experience was that it was difficult to coordinate efforts at scale. In the purest expression of Holacracy, every team has a goal and works autonomously to deliver the best path to serve that goal. But for larger initiatives, which require coordination across functions, it can be time-consuming and divisive to gain alignment."

"These people build consensus when it’s possible, and make difficult decisions when it’s not."

-- Andy Doyle, Management and Organization at Medium

 

What's StrategicDoing What Differentiates

Image: from sketchnotes by Remarker on Adrian Cockcroft's talk about Microservices, Netflix, and what he learned:

"Invest in the technology that's unique to your business and buy the rest"

"Remove friction from product development and don't do your own undifferentiated heavy lifting."

To do what differentiates, we need to know what we have to build with and on, and know what is (and is likely to be) valued, and what "unique" is, in the competitive landscape/value networks we play in.

 

Outside the EchoChamber

Wait! I should change the name of my Trace to Perspectives from Outside the EchoChamber. ;-)

"to prepare for the unexpected, you have to weaken the tendency to normalize. You have to encourage ambivalence" Karl Weick

Ambi-valence. Multi-valence. More than one center of attraction. The state of having mixed feelings or contradictory ideas about something, that arises from things not being all one good, or another, but multinatured. With tradeoff spaces.

I risk saying the unpopular thing, but nonetheless: in many contexts, the architect role is too big and too important to be smattered across people. It doesn't mean others don't contribute to the architecture -- they will, regardless! (I am not recommending it, but even if autocratic-architecture-by-dictate is attempted, misunderstandings, accommodations "just this once, just for now", and wotnot will rub at the architectural lines.) It simply means that someone (and for very complex systems -- an airliner, say -- a team of architects and lead architect) is "getting the system's back" so to speak; going to bat for the system, weighing in for the system against the tug of parochial vested interests. Not in opposition to the team, but in support of the team, and gaining support for putting in place what the team will be needing, etc.

Architecture is about seeing (more of) the tradeoffs, seeing beyond the seeming crunch of immediate demands (now seems so urgent, but we have to live with decisions we make under apparent now! stress for a loooooong time). Yes, it takes imagination and perspicacity, and yes, there will be things we'll be wrong about -- but better to be watching for what we are wrong about, than proceeding headlong at full pace into a wall of we-didn't-think-this-through-debt!! But competition! Yes. The tensions in sustainability include surviving the short term and building a future we want to live in -- that is, entails proceeding as wisely as we can, recognizing that even the past is different than we perceive it to be! ;) But yeah, architecture is about tradeoffs and resolving tensions -- technical and organizational. All of this takes attention and development of skill, experience, perceptual acuity. And doing things well, takes becoming good at it by doing it, and doing more of it -- investing in developing the technical wherewithal but also developing in organizational and strategic effectiveness areas too. And all the other stuff I said in several traces this month. Ugh. Must refactor and simplify!! ;)

I love the idea of self-organizing, and have always been very self-determining in all the roles and positions I've held. I think that in software we very often get to define our jobs by what we actually do. And within that value frame, having clear focus has also helped me be self-defining -- I have been able to do that "what at this extraordinary moment is the most important thing for me to be thinking about" thing, which is about judgment and navigation. I've never just done what some "guru" in the software intelligensia wrote in a book or whatever -- and I don't see other people doing that either. We take suggestion, we follow lead. But we use judgment. We apply and blend, hybridize what we already know and do, with what we learn from others. We try things out, and if they don't work for us, we push back and push on.

“But which is the stone that supports the bridge?” Kublai Khan asks.
“The bridge is not supported by one stone or another,” Marco answers, “but by the line of the arch that they form.”
Kublai Khan remains silent, reflecting. Then he adds: “Why do you speak to me of the stones? It is only the arch that matters to me.”
Polo answers: “Without stones there is no arch.”

-- Italo Calvino, Invisible Cities

"Architect" doesn't have to be a position. But to suggest it should not be, is ... oh, yeah, remember this:

We rhetoric hyperbolically to change tack, thrust our weight to this polar position or that, and threaten to capsize the vessel of "progress" -- 10/20/15

 

2/20/16: Yes, this (@Abby_the_IA):

Everyone architects

And it is important to give attention and expertise their due. If we want everyone to be the same -- same level of experience and expertise, same focus, all work on the same thing... that's... one way to go... Generally though, the very people promulgating such an approach, want to be followed. Set themselves up as experts. Want to be heard. And can be quite shouty, assertive, even aggressive (in condescending, defensive, etc., sorts of ways) about it. ;) Humans are social animals, cognitively limited, and time bound. So we figure out how to work together on doing bigger things than one or two or three people can do. This means collaboration, but also coordination -- in the systems we build, and in the systems building them. We can push more into the environment, but environments can then be felt to be coercive too. Imagine being an ant! They achieve amazing things, but have limited degrees of design and social freedom.

Yes, yes, yes. One paragraph became many... Which I need to pound down again. Because people are busy, and brevity has its place. This is where I started:

Agile in flavors like SCRUM and feeding into frequent, if not continuous deployments, raises the profile/visibility/sense of urgency around releases. The clock beats louder, and is more of a master. And it takes very strong discipline to keep a handle on code quality; to keep the code from falling into (massing, until profligate) technical debt. Having strong and organizationally valued technical leadership throughout the project becomes more, not less, important in Agile. The structural design has to be continually advanced. Sure, the team is participating in the design, impacting the architecture, making architecture decisions – but if architecture is everyone’s responsibility, then architecture is no-ONE’s responsibility and it suffers the plight of the commons. SomeONE needs to be attending to the system as a system, to the structural integrity of the code when that competes with the release drum beat. Watching for architecturally impactful, structurally and strategically significant decisions, and making sure they get the attention, reflection, expertise they require. Even when the team periodically takes stock, and spends time reflecting learning back into the design/code, the architect’s role is to facilitate, to nurture, the design integrity of the system as a system – as something coherent. This takes technical wisdom, perspicacity, insight into not just consequences of design choices, but watching for design challenges ahead, for impacts across the system invisible to those whose perspective is shaped by working on a locally scoped piece of code.

I guess it really did the job. I could have saved myself a good dozen paragraphs! But you. Well, if you read along, you got a lot more, right? No? Good gravy, go back, go back! All that stuff about strategy and context? About the social dynamics of binary positions that draw clear tribal lines? Whaaaat? ;)

Ps. Love those architecturally significant design discussions that happen on Twitter and Stackoverflow, etc. :) "The price of"... "source of truth"... smart humor with a pointy point :) and "still need" and back to elaborating the tradeoff space...

Pps. A good architect isn't a (Robert) Moses (all the time)! A good architect, I think anyway, is balancing fractal and emergent unfolding. Doing just enough context setting (so more good things happen, and less of the bad things) and infrastructure work and so forth. Using the lightest touch approach that moves things along, in the direction of getting more what we want. Where "we" means balancing across participants in various systems, internal and external -- the code/dev/ops (eco)system, the system of systems in which our system is used. Where moving things along includes building systems out, but also being deliberative. Making deliberate discovery a way to have "oh crap" moments earlier and more cheaply, from the perspective of lessening their impact (when the (side-)effects are negative) and entanglement in systems and expectations.

“But it’s not very useful to look at them as ‘pure’ Jane Jacobs and ‘evil’ Robert Moses.” -- Joshua Frankel

 

Software Architecture Workshop

Trying to decide... May 10-13 in Chicago/Schaumburg or Boston/Dedham (MIT Endicott House -- out of the way, but lovely venue) for an open enrollment Software Architecture Workshop? The folks on board so far, don't mind either way/are traveling in anyway. So, quick, quick, if you're interested, tell me!

 

 

2/20/16

Constructor Theory

I need to keep trace of this (my Trace being, among other things, a neat memory extension/way to add capacity to my limited onboard memory) on Constructor Theory, via Dave Snowden:

"The basic principle of constructor theory is that all fundamental laws of nature are expressible entirely in terms of statements of which tasks (i.e. classes of physical transformations) are possible and which are impossible, and why. This is a new mode of explanation, intended to supersede the prevailing conception of fundamental physics which seeks to explain the world in terms of its state (describing everything that is there) and laws of motion (describing how the everything changes with time). By regarding counter-factuals (‘X is possible’ or ‘X is impossible’) as first-class, exact statements, constructor theory brings all sorts of interesting fields, currently regarded as inherently approximative, potentially into fundamental physics. These include the theories of information, knowledge, thermodynamics, life, and of course the universal constructor."

 

Complexity

 

Shaping Stories that Shape Us

Interesting, useful musing (for coaches -- including architects) -- and fire worth stealing (just change "patient" to system):

"the purpose of the therapist is not to discover the truth of the patient. it is to discover the story the patient can use to become" -- Michael (Geepaw)Hill

 

 

2/21/16

Aaaaaand... Introducing

Steve Merrick, aka @patternchaser, has started blogging on things firmware and design, but really, it's muchly about software and design so go check it out regardless. It's shaping up to be a wonderful contribution to the space, so clue in on it early. :)

And if you haven't yet checked out Amitai Schlair's Agile in 3 Minutes podcast (and book version of the essays), spend 3 minutes, now and then, and let some Amitai rub off on you. It's a good thing:

"It's #agile poetry from @schmonz. Highly recommended!" -- @ryanripley

And if you aren't following Peter Bakker, aka @brain_work, what in the name of all things design are you thinking?? Follow him! He's is good people! Best people!! I mean, he has been nice to me for years -- almost as long as Daniel Stroe (who alas isn't active on Twitter any more -- but Daniel is in embedded systems, so maybe Steve Merrick and Daniel can create some critical mass on twitter??).

Being nice to me is a reasonable qualifier for following? OBVIOUSLY! I mean, consider -- are you nice to me? If yes, you're in good company -- few, but such fine people! If no, then you have extra reasons to respect those who are! ;)

Okay, in seriousness, Peter investigates topics that interest him -- visual in design, these days, looping back to his work on tubemaps in system modeling. (A few years ago, Peter used my Trace as a demonstrator for his tubemapping work -- which meant he did me the honor of actually reading traces and threading them into storylines. So, you see, he's smart, kind, and...disciplined!)

Oh, Peter's tubemap led me back to reading (a 2006 trace and):

"Besides, when the architect is held in high regard by the team, this does nothing to diminish the other team members--not unless the architect is arrogant, and arrogance and competence, with its corollary of respect, are not joined at the hip!"

And. Gene Hughson's architecture blog. Upon a time, he quoted me. So he's been slacking, but is still worth reading. ;) At least he retweets me more than anyone! :) He's good people, and smart, and doesn't get as much following and community-visible fist-bumps as his insightful blog and conversational peer-coaching in the social-verse deserves. (He gets a lot more than I do, but I'm handicapped... shhhh, don't mention it in public but.... I'm "female." I know right? jk. He blogs. I trace. It's hard to be appreciative and enthusiastic about a Trace. Evidently. jk. His blog is wonderful! Mention it sometime.)

And GeePawHill -- but he got a special featurelet yesterday when I stole a coal from his fire. That musin' was a lovely example of what Michael Hill does on twitter. When I have hope for twitter, it is in good part because people like Michael craft a distinct, compellingly interesting voice on twitter. And he does it to 11!

And Grady Booch has mentioned, retweeted and linked to my Trace. Grady and Peter Morville and Andra Sonea, are, I think, the only tech intelligensia with more than 10,000 followers to ever retweet me. (Well, I could have missed some; follow count and other status markers aren't really something I look at. No really.) So clearly their good sense recommends them to you, and you're following both Peter and Andra, right? You're obviously following Grady! (No?! Take yourself out to your woodshed!) It's almost Grady's birthday, so get your words of celebration ready!

Also, Brenda Michelson, Sally Bean and Andra Sonea are senior architects and architect coaches who combine too many wonderful properties and insights to mention -- you just need to follow them!

Besides, we (women too) all need to try a lot harder to be part of the progress towards valuing women as contributors, not just bit-part players who should be seen (iff eye candy) but not heard, so make sure you aren't holding women to a much higher bar when it comes to who you follow and which tweets you read, and retweet, and so forth.

And lastly, because awesomeness, Eugene Barker, aka @arctips, and this blog post, which is a nice capstone to my architect ramblings this month:

 

2/22/16: I used to think (something like) this (too):

"The web does something more important than connect us, it weaves the space between us together." -- Saul Kaplan

Now? I think the weaving is thin. It is without real caring, without any real sense of mutual responsibility. It is easy not to be considerate, because we connect with some thought projections across the darkness of digital space, not with flesh and blood of emotional people. (And yes emotional! No matter how unemotional people conceive themselves to be; there's even emotion in being judgy of emotional people! :) So it is a dangerous weaving, because we put time into illusory tenuous digital connection, and that takes time away from richer bonding in face to face connections. I don't think it is hopeless, though. I think we have to remind ourselves to be considerate -- to take in, account for, the humanity of people when we only interact with their words -- words on our screen. Words without the personality even of handwriting, without the little choices of paper of different textures and hues. Without seeing a little personal quirkiness... Net net? I think we need to recognize the thinness, and make it a thicker medium, By simply being more considerate -- I'm not speaking to bad behavior. I mean those of us who are trying to be good people (and hard it is! what even is good?). I mean taking the humanness of others into consideration.

 

Different Things, I guess

It's been a really slow month as far as visits to this site go -- that, despite its 10th anniversary, lots of (surely great and useful?) writing, and a kind retweet by Andra Sonea and Peter Morville, among others. I think I have reached peak invisibility. Well, there's you, but you have special powers and can see invisible people...

Generosity is a renewable good.

 

Tipping the Hand

In the trace that set my fingers off on the whole "to architect or not to architect" question, I wrote:

But it does subtly tip the hand towards No. Because it suggests we have to choose between a bad architect/architecting style and none. And that's not the choice, or shouldn't be.

Does that ring any bells? This perhaps...

"Through this work we have come to value:

...Working software over comprehensive documentation..."

Clearly it is not saying there's no place for documentation... So backs are covered. And yet it tips the hand, puts the weight on, working software. When we value good x over really bad y, then y is tossed out, because we don't want bad y. To see my point, let's phrase it like this:

"Through this work we have come to value:

Working software with just enough documentation"

Now people are asking "what's just enough?" "what documentation?" It's not as zingy, but it's not as zingy because documentation is still in the picture and we have to figure out what documentation is critical and suffices. We might argue that the second is the intended meaning of the first, but the first has the effect of tipping the hand in such a way that documentation is tossed off the slate of concern.

So we've been having to drag the just enough documentation rock back up the slopes of Snowbird. Or something. ;)

At any rate, the net effect was tossing the modeling baby out with the BDUF bath water. Poor baby. It hadn't learned to control it's bowels yet. Or we hadn't (all) learned to control our analytic zeal.... It didn't mean that UML is bad, just because we overused it. Sure, it is more expressive than we need for lightweight just enough modeling. But I don't use all of English -- at all, and certainly not in a given piece of work. I do wish we could have had an open forum for managing our Modeling Language definition effort, rather than an elite country-clubby sort of thing. But an evolving "Oxford Dictionary" -- or better Urban Dictionary? -- of software modeling language has merit. [I know, some duplication/overlap with an earlier trace... I need to refactor and cull. Get rid of some bathwater and babies. ;-)]

Okay, okay. Sheesh. I don't really mean a phrase was responsible for a sea-change in the field's trajectory! Even if it was a phrase in a document that was a highly influential reference point for our field. But it is a banner phrase, returned to often, and representative of many conversations and other messaging, that meant that many, and then more, tended to view code, and tests, so code, as the way to design-develop software. Now modeling is back in more and more conversations about software -- as informal sketches we draw before we code some gnarly bit, to get "on the same page" about a chunk of the system, etc. I'm by no means disparaging that! Quite the contrary! I'm saying it is where we should have been in 2002 (and where many of us were, obviously -- minimalist and just enough were deep in our DNA). And as we rebalance further, we'll start to draw more -- draw, and draw on modeling, on visually representing some key facets of the system, to think them through, improve our thinking, make it our thinking, and share it with our downstream--that is, future, selves. More -- as in, just enough. :) [We were "just enough"-ing with Team Fusion in the mid-90's, hence the "deep in our DNA" reference.]

We are intelligent thinking creatures. We can also be influenced. And we can especially be influenced to do what suits us -- that is, seems to be in our interests. That tells us what we want to hear. That gives us a chance to lead our team on mission, chase the holy grail, pursue the silver bullet! Well. Okay, I put that too colorfully. Got carried away. O.o-ps. :) Anyway, our path tends to be kinda zig-zaggy. Object technology comes along, and modeling was coming to a head just in time to help us think about objects? And knit the domain to the code, so that domain changes could transparently shape code changes. How cool was that?! I mean, doesn't that sound cool? So we overdid. So we pushed hard against overdoing, and overdid in the other direction. "Working software over comprehensive documentation" was probably the banner call we needed, to counterbalance too much paper design, and get back to more design by means of code and test -- test in its local and broader sense of testing it out in the "real world." With air quotes, because A/B tests and other experimental approaches, so a testbed for the broader real world demands, values, stresses and strains. Now, in the rear view mirror, we see the broad patterns of zig and zag, but it's also true that many don't go all the way to the extremes, Partly, some are just saved by inertia of big. Others are just using judgment, learning from the shiny new thing, but incorporating it judiciously into what they do. Judgment always factors. Unless it doesn't. Tracing, for example, is both. It's a very rash thing to put thinking in a (potentially) visible spot (off the lesser travelled routes) on the i-way. But it tries to be judicious.

"I tried"

"I give up"

Phrases our culture undervalues! How the heck do we learn if we don't try? How the heck do we take new avenues of address, come at things from a fresh perspective, if we don't give up the approach we're taking?! Give ourselves a break, come at it from a new angle with fresh determination. It is the start of a new twist in the tale, or even a new tale. But endings are beginnings (little beginnings, or more of a departure), too.

Zig and zag isn't all bad. We move to better on the lessons learned by those who went to the extremes and learned what the real tradeoffs are!

"I know. Hyperbole. Rhetoric. We have to get so much energy into something to get the stupid thing to budge at all, it can overbalance. But first to get it moving!!" -- 2/24/14

 

2/29/16: This is a nice reflection on our binary (this or that) zeal, and the way two things can be true but there are tradeoffs and the weight just falls out differently in different contexts (sometimes at the poles, and sometimes as an approach that is a hybrid that combines something of both):

 

2/22/16

Made My Day!

"I’m a big fan of the www.bredemeyer.com content."

Anyone who says "the content is outstanding" -- and thanks me and Dana/doesn't just assume it was Dana -- gets my full and immediate attention. :)

 

And Then

My day got even better! Thank you Andra!

 

A Thing About Boundaries

Remind me to write a trace on boundaries. Oh, not those boundaries. :) System boundaries. Boundaries within our system.

In the meantime, you can acquaint (or refresh) yourself with this post (on the Bredemeyer Requisite Variety blog... which I really should kick back into gear again... maybe?): Mastery and Architecting

boundaries

 

"Each module is designed to hide design decisions from the others." -- DL Parnas (On the Criteria to be Used to Decompose Systems into Modules)

"The purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise." -- Dijkstra

2/28/15:

 

2/23/16

(Conceptual) Integrity

Steve Merrick reminded me of this passage:

"I contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas." - The Mythical Man-Month.

Good call to go back to a classic (again!), and also to pull together and elaborate how I think about integrity in the system/software architecture/design space.

 

 

2/24/16

Michael Feathers on Providing Options

I listened to and highly recommend this: Episode 102 | Michael Feathers - Providing Options

The whole thing is full of insights and images worth stealing/leveraging, but to give you a sense, I'll focus on the last segment:

Transcript of Michael Feathers' responses to 3 tips to deliver more value:Outside box

  • Really listening is the most important thing: it's understanding what people want, what they need, understanding what the effects are of what you propose
  • It's applying all the skill you have to understanding problem, the person and the context, and being able to step outside that box a little bit and think of things that may not have occurred to others;
  • There are many situations where we face problems that we're trying to solve, or we're in a situation were we may have opportunities, and we miss them because we don't think outside the box enough to take advantage of things we haven't seen
  • It comes down, again, to awareness. I try to maximise the options; maximise the different ways of seeing things.
  • And I know that is something I can do at personal level, that can help other people out.
  • So I think that to the degree we can cultivate that sort of thing in ourselves, and help people see that it's fun (it really is fun to see things from a different point of view), it can make life so much better.
-- Michael Feathers, interviewed by Dave Rael

See? Awesomeness!

 

Enterprise Architecture Papers

Here are the EA executive reports Cutter published (you could think long paper, but if you think short ebook you'll be happier -- especially when you can get it for just the price of your contact info, not your credit card and contact info -- winking smile)

 

 

2/28/16

Some Dangling Threads

Tradeoff Spaces

Complexity and Irreversibility

 

 

3/4/16: Metaphors and analogies are not identities, and we use them expecting people not to stretch the point:

"If you want to explain electricity to someone who doesn't know anything about electricity, you say, well, you talk about an electric current. Now, the word 'current' is borrowed from rivers. It's borrowed from hydraulics, and so you explain electricity in terms of water. Now, electricity is not water, it behaves actually in a different way, but there are some ways in which the behavior of water is like the behavior of electricity, and so you explain it in terms of water. Or if you're an astronomer, and you want to explain to people what you mean by an expanding universe and curved space, you say, 'well, it's as if you have a black balloon, and there are white dots on the black balloon, and those dots represent galaxies, and as you blow the balloon up, uniformly all of them grow farther and farther apart. But you're using an analogy--the universe is not actually a black balloon with white dots on it." -- Alan Watts, The Nature of Consciousness

Of course, Alan Watts then goes on to stretch the point. ;) just kidding! But I would argue, we shouldn't make words take all responsibility for their meaning. Nor make metaphors take all responsibility for our understanding and use of them. The thing is, we want words, metaphors, things to mean stuff. So we play an active role in trying to understand, but we also have lots of baggage. History. Context. All conspiring to help and hinder us as we try to bring a larger narrative to bear, to make a strung-together meaning.

So if our narrative is "we do software all wrong," then "construction," as a handle that's been used for producing(/where in the process we produce) code, becomes a ready target to peg responsibility on. Behind that handle, is a whole field, and it is simply, to put it bluntly, insulting to charge a metaphor with how we view and build software. Smart people weren't led by the nose to do things some particular way, by a metaphor! What we do, how we design-code, is a product of so much history, accident, and hard thinking -- a lot quite far from any "construction" tag.

So. I'm constructing this argument. Of course, I'm doing so in my usual iterative, messy, "feeling for how to better express the intuition that is nagging at me" sort of way. And it's still messy and would need considerably work before it could more reliably support safe passage to understanding. I didn't use a pick, though I did try to pick my way through the fallacies, bulldozing the more unhelpful thought detritus... Hm...

Back to construction in software: the word reshapes itself to mean our evolving sense of how we create software. What we do, impacts what we think construction means, when we see it as a label on process diagram, for example. It may not be the best word to use, and may even inhibit some understandings, but we are active in meaning making, and our objections may get more in the way, than the word/metaphor itself! We can obstruct understanding simply by putting objections in the way of understanding. :)

Let me come at it from a different angle. Where we continue to use PDLCs (product development lifecycles) that have waterfall flavors to them, we need to look at the co-evolution of practices and development infrastructure in that space. Recall:

"Let’s remember that continuous deployment was ridiculous five-ish years ago, and continuous integration unachievable fifteen years ago." -- Brian Marick

So where we work in systems(of systems), and we're not just integrating across hardware and software and firmware, but manufacturing lines that have to be co-designed, and supply-chains too... it's just a tad (wink) more complex than getting feature toggles down to a craft form. That is, software is developed in many different contexts, from a bit of ad hoc code one-offed for an experimental analysis, to firmware and software in a product line of manufacturing robots. With what you do somewhere in between ... or... along another dimension, in a multivariate map. :) Anyway, I really have a hard time giving the metaphor "construction" all responsibility, nor even particularly much, for how software is developed. We simply don't go round saying "oh gee, now what does the metaphor tell me to do?" "Oh, it indicates I need a bill of materials. Why, goodness me, I should create one." "Oh my, I need an excavator here." Well, actually. I think I have Mike Mulliganed myself into a big hole. ;)

In short, we need to allow that the meanings of words evolve. We want our process to be agile, to enable our business to be agile? Or at least, responsive and effective at sustaining itself, not just through the moment of now, but the longer now as the present threads itself into the future. Well, then, we might just think that we, the people of software and neighboring fields, are somewhat agile too? Somewhat flexible, and mutable in our understandings, not just in what we shape and give form to in the world?

Yeah, so, we can -- should even -- adopt better, more useful metaphors. Words that draw us towards intended meanings, rather than words that trip us up. I'm down with that (in Buffy/1990's speak, because the teens have me working out to Buffy this winter). But my hackles rise when we treat people like they are idiots we have to look after with better words because they are too clueless to adapt themselves to new understandings. There is give and take. And we naturally tend to gravitate towards new words that are more useful. I think we need to gravitate towards using design instead of requirements, but time will tell. We can take a page out of Lackoff's book, and speed evolution along by picking metaphors that help us progress our understandings and approach. But let's motivate doing so on what a new metaphor offers, rather than asserting that our understanding and approach to software is trapped by a word! It quite demonstrably has not been! Witness successful CD/CI projects.

Speaking of co-evolution. Did you see Mr. Turner? Neat point about art and technology.

Mainly, like this Trace, if you look past all that's wrong with it, there's a lot that's right with it. We progress (often with air quotes -- often seen only in the kaleidescope of history). Our words shapeshift. Morph. Take on new meanings. Or we flat out jump to new ones. I'm fine with drawing attention to moving on from construction as a metaphor to something better.

I just think we misplace attention, if we try to claim it is responsible for too much. Likewise with technical debt -- it is not responsible for projects that take on wanton debt. Something else is going on there. Words are powerful. But they are not all that is powerful. Yes, words frame and draw. They are what we use, besides phyiscal stuff, and images, to get ideas across. To signal. Convey. Reproduce ideas into the minds of others. But a word or phrase does so much. How much, depends on a lot of other things. Context. History. Repeating yourself.

In short,

i. I don't believe the use of the metaphor "construction" caused waterfall. Nor Evo or SCRUM. Nor their mechanisms for determining and coordinating work. Smart people working with an understanding of their context, the forces, technology and understanding/teaching of their time, etc., did that. But we enabled new ways of working, that apply in a lot of contexts -- and more and more of them.

ii. I get antsy when we act like other people are idiots -- like they can't understand a word in one context, because it carries too much definitional baggage from another. Not that it never happens (we're fallible creatures), but it is an unfriendly, derogatory assumption to default to...

What would I suggest? Start to use new metaphors and images. Look for a good (enough) fit, to mobilize understanding and action in a (better) direction.

'The limits of my language means the limits of my world' ― Ludwig Wittgenstein

Change the words and you begin to change the way you think. That in turn changes the way you behave. — Charles Handy, The Second Curve

I also write at:

- Trace In the Sand Blog

- Bredemeyer Resources for  Architects

 

Architects Architecting Architecture'

- Software Architecture Workshop, Boston, MA, June 27-30, 2016

- Enterprise Architexture Workshop, Chicago/Schaumburg, IL, July 18-21, 2016

 

 

Journal Archives

- Journal Map

- storylines tubemap by Peter Bakker

 

2016

- January


2015

- June
- July
- August
- September
- October
- November
- December

 

2014

- January
- February
- March
- April
- May
- June
- July
- August
- September
- October
- November

 

More Archives

 

 

 

 

 

My alter-ego:

 

Architecture astronaut

 

 

 

 


Copyright © 2015 by Ruth Malan
http://www.ruthmalan.com
Page Created:February 11, 2016
Last Modified: August 19, 2016