A Trace in the Sand
by Ruth Malan
Table of Contents: 10th year anniversay month of traces!
This is where I keep a trace of thoughts on wanderings in the landscape of software and systems architecture and those that abut and have bearing on our field. And on February 3, I will have been doing this for 10 years (with a hiatus here and there, where I wondered if the risk of putting an open mind on display for potential public thrashing is a good idea, or just got discouraged because no-one was interested, so why take on the risk).
Okay so. My Trace is pretty awesome -- well, in the sense that if you like this sort of thing, you'll find this the sort of thing you like (after Lincoln, I believe -- at least, F. Scott Fitzgerald thought so too; but I repurposed it, with opposite spin, for in this case, obviously if you like this sort of thing, you're in good company... At least, we think so.)
What? Aghast at me saying it's awesome? Well, you know, it's been TEN years (I get to shout now? Just a little? It's been 10 years!)... And some of us need to place unequivocally positive words out there, to frame the thing in a way such that just the right sort of people notice it. It's a dirty job, so I pitched to help. I'm generous that way. :) I mean, we couldn't leave it all to Amitai Schlair and Stuart Boardman and Peter Bakker, now could we? (Also grateful for RTs: Gene Hughson, JP de Vooght, Paul Harland and Tom Graves -- to name the most recent. And to Michael Hill for "meaty" and "suppress yabbiting for a minute"). And hopefully if I'm just the right sort of rotten, those who would be ticked off at its playfully confident humility will move right along. :)
Having 10 years of traces behind me, is definitely great grist for the humility mill -- my inner critics take such glee in pointing out all the things past me got even a little bit wrong (in expression or understanding)!
To give some sense of the scope under the lens, here is a partial map of topics (and links to traces). For a taste, see for example,
Michael Hill and Amitai Schlair each said something that slanted a new light on how wonderful people like them experience this Trace:
I suppose... if you were inclined to expect a negative, that could sound intimidating.... like it's work to read, albeit with a payoff. So, only worthwhile if really on a topic of direct importance? But then again, there's also this from Oliver Baier
I think -- hope! -- that what all three meant is not that it is hard to read, so much as that it reveals more, the more willing one is to engage and follow-up on the (get-a-)clue-trail it leaves. ;) I do hope that it is encountered as something that sets off sparkles and flashes of insight in the reader's mind. When I approach it as a reader, it works for me when it does that, and I feel like I failed myself when I read some traces where I was wrestling with some understanding and it was getting the better of me!
Once, upon first encounter with my Trace, one of our field's lovely, most lovable people (sincerely!), pointed me at a writing analysis tool so that I might use it to improve my writing, expostulating (I'm approximating from memory, as one does) "I don't know if that's what you think, or what I think!" Meaning, I presume, that I left some things to be worked out, to be reached, in the mind of the reader, rather than landed on him (in that case) fully formed, and unequivocal. And that says so much about my teaching/mentoring philosophy and my writing style. I think we have to wrestle with insights to see their multi-faceted natures, and how they shape-shift and morph in different contexts. If someone tries to distill understanding into a cookbook and lay it on us to imbibe and ingrain in the webs of our brain.... gonna work? I think not. I mean, I do place a lot of stock in education, in learning, in how to do more of that in a lifetime -- we're not neoliths with neolithic culture... not that you could tell, looking at 2 year olds and adults in our (best and worst) moments... :) So our schooling does leave some wake in our individual and collective minds and mindset. But I also think we learn by munching on, not just swallowing knowledge whole. And those things have to interest us enough, to cogitate and try things out -- in our mind's eye, and with our mind-hands working on it. So there is a place for ideas that are more suggestive, than pedantly put. That entice us to skip our own idea stones on the river of knowledge... uh... whoops. Little carried away on a thought stream there... The point is, what engages us? It's not all the same-same same.
Ha. Did I rational argument you into believing my horridly ambiguous writing is entirely justified? You know what I say, because I've said it a few times (here and here) and i. you noticed and ii. you remembered. Hm.
I'm teasing! I mostly think that there are many ways to convey something. And we have much to learn, but it is not all uniform and approachable in a "one best way." Some knowledge can be quite distilled and directly, even assertively, pragmatic -- do this, do that. Other things we've learned, can be conveyed as the kind of checklist that we use as a completeness or attention and memory check -- did we notice this, did we remember to do that? But the topics I grapple with here, tend to be the kind of thing that is more multifaceted, and light needs to be shed from different angles, approached obliquely. Sometimes these are topics we tend to "take sides on," and become polarized around. But things aren't so simple or so easy, and reaching insights from a more holistic perspective may require a style of address that is more tempered, holding multiple simultaneous perspectives in creative suspension. Identifying paradoxes without (kicking in that self-protective lizard brain and) losing our cognitive resources in the face of the tension of holding opposing truths in mind at once. There's a place for those who can approach a divisive topic and not blow people's fuses with highly polarized winner-takes-all stance. Just as, apparently, there is a place for those who do! ;)
The other thing I did, was let my understanding evolve. A journal is never done. It maps a person's progress, and doesn't make any claims to completeness. I used this quote last month:
It playfully captures that spirit. Traces are irresponsible in the sense of thrusting a reader directly into the thought fray without much easing into, like the usual bookends we expect on articles or essays (in blogs or more formal publications). I don't set the reader up with affordances like "this is what we will talk about" and "this is what you learned." We hit the ground running with an idea, with little warm up or warm down. So each trace has that spirit of being something in progress, part of something larger, something ongoing. Which is the state of an open mind.
Thank you to those who were encouraging. It meant -- and means -- a lot! Several people, over the years, came back to look in on this Trace from time to time, and that was gratifying. WB Yeats wrote "and say my glory was I had such friends" and it is not nothing that wonderful people read here a little, now and then.
Now. Time to try a different angle of address. More... like essays, than a journal. Or something.
A Trace was its own thing. Anyway. Time to put it to rest, I think. Not to stop writing. But to try a different style -- force myself into writing more in a more "usual" format.
It is, in the end, just too depressing to write something that is set up to be rejected, because it is too idiosyncratic. Relies too much on a largeness of spirit and inclination to such a peculiar thing as a me, and the way I approach things.
Oh well. Better things to come then. At least, better by the standard of being interesting to, and recommendable by, more than just a (courageous? generous?) few.
People who have been around this Trace over the years, know I've resolved to dump it many a time. Rejection (and silence on a body of work is rejection -- of an insidious unspoken sort that causes a work to be ignored) is a hard thing to face. And hope is a hard quality to kill -- especially when a few people were kind. But, if you write in public view, you obviously hope that it will be valuable, or delight, or be some kind of positive at least to some people. How many people is enough to be sustainable? Well, for example, enough that even on this, the 10th "birthday" of this Trace, it wouldn't be left to ONLY Peter Bakker to rouse the generosity to say something celebratory or appreciative! Oh, I completely understand -- the few people who are responsive, have been responsive in the last month or six, and they are tapped out. But that's the point. If a work is useful and good, worthwhile, but also something special, unusual, above the bar, someone might say so. And someone else. Not always the same someones.
And no. Please don't mention this Trace to create a "sensation" -- let it slumber.
I should add --- I'm not just packing up my toys in a funk. ;) I'll probably use this space for drafts of more topic-focused essay things. Or to note when I've put slides or other material up somewhere else. And so forth. So stop by from time to time.
So less of more and more of less. Or something more or less like that. :)
Okay -- almost silence!
Bottom line: appreciate diversity. That doesn't mean just talk about it. It means be inclusive.
What''s a Trace? Well, It's like this, see:
2/24/16: As metaphors as rhetorical devices go, Paul Harland pointed us to this:
Aside: Credit card debt may be a good analogy -- people can get into real debilitating financial difficulty, getting over-extended on credit card debt.
Perhaps you saw this tweet poll, since you pay close attention to my tweetstream and I retweeted it. Oh. Yeah. Probably not. Well, here you go:
The way that survey question is framed, is not neutral. Is it a straw man? Not really. But it does subtly tip the hand towards No. Because it suggests we have to choose between a bad architect/architecting style and none. And that's not the choice, or shouldn't be.
So let's ask "do we still need (good!) architects?" Steve Merrick, Chad Fowler, Eugene Barker and Rebecca Wirfs-Brock all answered it succinctly -- I like "Hell yeah!" But since there are doubts, here's more to reflect on:
* Not sure who referred to the tragedy of the commons in software first... "Siri, who was first to apply commons ideas of Forster Lloyd/Hardin/Ostrom to software development?" Nothing. Watson? (I see I mentioned it in 2009, and it has a c2 wiki page (last dated 2010), so it was in the air before that. Probably goes back to the 70's, like so much else in software. :)
Yes architects code. Unless they don't. When we hear "architect also codes," we tend not to hear what the context or frame of reference is. The thing is, someone who may be designated architect on a team working on a fairly small and relatively independent software system, has a different-natured set of concerns and responsibilities than an architect of a product family, portfolio or business domain, or the architect of a large complex system, or system of systems, or the chief architect of a business unit or large company. That is to say, as the architectural scope of what an architect is responsible for increases, we need to rely on the architect's judgment as to where they need to spend their attention cycles. For example, the chief architect for the Civic, say -- cars come in product lines, remember -- may not even have a software background (though the software content in cars is mindbogglingly large, and getting larger). Further, do you want the chief architect for a product family or business domain, popping in on a project team and taking on a critical path piece of coding work? Pairing on it? Sure! Design and coding an architectural mechanism that will be shared across teams? Yes, especially if this is done collaboratively with impacted teams, even potentially mob-modelled and programmed -- at least to get initial challenges and ideas shared, alternatives designed and tried out.
That is, the more complex (technically and organizationally) the space the architect is bringing coherence across, the more the system (of systems) demands attention and the architect has to balance all those demands with learning and staying in touch with the system. The architect has to be able to partner well, delegating and communicating effectively. If we don't understand this, we set the architect up to be questioned on competence terms against developers on the various teams under the scope of the architecture. The credibility of the architect is important, because it impacts the team's orientation to the architecture -- if there is no trust, it will show up in various ways that undermine the architect and the architecture. It's important to recognize and reinforce the value the architect brings in terms of system design, so system integrity (structural and design integrity, coherence, resilinece, etc.) and sustainability. It takes time to understand what needs to be thought through, and then to design, evaluate alternative approaches, document and explain and explain and explain. To persuade, to influence, to shape understanding, to draw attention, to highlight, to inform. To explain in person, dynamically, in conversations. To explain with reasoned arguments we've taken time to sketch out and write down. To share our thinking, and to revise and improve our thinking by getting it out where we and others can see and respond to it. Writing and modeling to think things (aspects of the design, how we approach a design challenge, etc.) through, to get them clear for ourselves, to help us get them clear for others. (You want me to describe and explain what I said on slide 59?)
How does the architect stay credible, with deep knowledge of key areas of the system, especially when this means not just reading knowledge -- we're embodied creatures, and we learn with our mind-hands working on code? We're also social creatures. So we also get a lot from group cognitive processes, like pair programming. Code reviews and code-intensive retrospectives that are really hands-on, where we look at what we did and have learned -- and for alternatives -- can be great too. Besides, for the architect of systems, there's designing and writing code to analyse the system (code, and partnering with (dev)ops to understand the behavior of the software in operation). To understand how the code structure is evolving, to watch for stresses and strains reaching design boundaries, etc. And ongoing teaching, mentoring, coaching -- for teaching is learning; it is being open to new questions and ways of seeings things, being challenged in a good way, discussing choices and impacts, explaining and re-explaining with a new twist, from a new angle, so that the rationale and solution helps shift mental models and make sense. The architect acts as a consultant to the team, as well as to peer architects and the management team -- which means listening. Really listening. And figuring things out -- what is intended, why is that sought, what does it get, how do we get more what they want, but also balancing with what others want. Etc. Demonstrating to the team, and the extended team, that there is value in the other things architects do, beyond writing code. But also writing code, where the architect judges that is what needs their attention. And writing code as a matter of discipline, because hard problems are hard, and take expertise. Not all one thing, or another. Judgment factors. And factors and refactors. Refactoring is good learning too. ;)
Our teams are social too, so human dynamics and power plays and all kinds of crud, along with good stuff, factors. Credibility can be easy to lose as the architect's responsibilities broaden, especially if someone has an agenda and something to gain from sticking points in the architect's back. "Goodwill is a silver bullet" (Dana Bredemeyer) is shorthand for the people factors factor. In great, awesome, productive ways, unless we lose the plot and malfunction together.
The opposite of a vice is not a virtue but a different a vice. -- @JohnDCook
Anyway, this is what I wrote on the "does the architect need to code" question back in 2006. I'd say it somewhat differently today, since we have different antibodies -- like "raise the productivity ceiling" might irritate the "Taylor era" rash we've developed. When really, what I meant then and mean now, is enable the team by removing obstacles, helping to ensure the team (or teams) gets what they need, identifying challenges (organizational and technical) that need to be tackled (not necessarily by the architect, or the architect alone). Having responsibility for making sure the system needs are being surfaced and addressed, doesn't mean we make the architect "god" -- if we do, Sisyphus is the more likely result of our attempt! :)
Sometimes, we do big things. Sometimes we do even bigger things (an airliner, an OS, a car!)! Sure, at any scale, we sure apply the discipline of "how do we make this smaller?" -- what does each feature add? each customization option? Is it worth the complexity cost we and users will have to carry?
There is a whole landscape or team and organizational cultures out there -- some more hierarchical than others. Different system and organization complexity, different demands, different team choices in the light of what they value in their architect, and what that architect brings to the team and organization. I have worked with some software architects who are very effective and valued even though they don't have direct code responsibilities on the project. I am not advocating that for every team (obviously), but I've seen cases where it works. So I'd just say it's a big world and pontificating about what other people and teams must or ought to do, is neglecting their abilities and choices. I can point to upsides and downsides and caveats and forces to weigh. I can bring experience and counter-experience to the table. But I'm not going to insist on imposing a righteous notion of how things ought to be done on a team that is working really effectively together! Where the guidance is worthwhile, though, is when a team or broader organization is looking to get a better handle on system architecture and design, and how to enable that interpersonally and organizationally.
"Nothing is an absolute reality, all is permitted". - Bartol (via mfeathers)
"they both strike the balance in different ways." -- Glenn Vanderburg
2/11/16: Two other perspectives:
2/24/16: This study is interesting for the question it asks:
In that article, what are called "architectural tactics" and then "tactics" (which is a confusing conflation), may be instances of architectural mechanisms, in the sense I've been using (in various traces this month). (Where "may" is qualified by significance to the system under design. It depends, for example, on what is brought within the scope of the system, what is on the boundary of system scope, in that the system interacts with and relies on a mechanism, but doesn't implement it, etc. In the latter case, there's a technology choice and boundary-spanning interaction that's likely architecturally significant. etc.)
At any rate, the study is making the case for strong design expertise when it comes to (design-coding) architecturally significant mechansims. It is an area that could stand to have more empirical study, but I generally think of our experience across many many projects (those of use who get to work across many) as being the first level of empirical study -- not the end-state, but a good basis for finding interesting hypotheses and strawmen to put into more scientific contention. :) And my experience tells me that smarts nuanced with experience tends to produce more robust designs -- tends, but anyone can amp up their design ability by adopting more discipline in terms of looking at a design from various angles, pushing and probing at it, trying things out, reworking it. It is just that with experience, we've done that a fair bit already, and we've seen how our decisions and choices played out, formulated "theories" about what works and why, and what it gets us. In other words, we've connected more dots from properties to code, been slammed by perspectives we neglected to take, grown a bit of wisdom of the sort that seeks to understand more of the multiplex of what is going on, needed, and how to make that be more the case.
Years (many!) ago, Dana Bredemeyer introduced "good, right, successful" as a way to talk about the architect's role in helping to ensure that the architecture is:
The last, is speaking to the organizational dimension of the architect's role that is missing from "right system [right], built right [good]." When I think of that organizational dimension, so many things vie for my attention! It is very much about ensuring that conversations that need to be happening are happening -- not always initiating them, nor always helping to focus and navigate them, etc., but ensuring they do happen when they wouldn't otherwise (at all, or in time), and guiding when needed. Which gets me back to another image that derives from Dana -- that of left hand right hand shaping. Where left hand refers to longer view, more strategic and cultural shaping, and right hand is more immediate, tactical. Yeah, culture is hard to pin down in -- if we try to steer it and make something happen deterministically in culture, we're probably going to mess other cultural things up that in turn mess up what we were trying to accomplish... Still, our behavior impacts culture. Where I mean behavior loosely, so that it includes talky things like design meetings or talky-do-y things like mob- and pair-programming (with as needed white-board storming). Where we focus our conversations, as in what we repeatedly draw attention to, illuminate through metaphor and story, etc., influences our behavior. So some of our "left hand" work may be this kind of shaping, these interventions that obliquely influence culture by influencing what we focus on, what we tolerate and don't, etc.
2/26/16: This is wonderful (thanks Paul Harland for pointing it out):
I was struck by the importance of psychological safety and all voices on the team having "equality in distribution of conversational turn-taking" (over the course of work days), rather than one or a few people dominating conversation. Also, the importance of informal content, for establishing comfort and trust. If there are negative consequences to voicing ideas and concerns and so forth, that voice will be hushed, and the team loses that balance.
Here's here are key takeaways from that research, highlighted by Julia Rosovsky in The five keys to a successful Google team:
2/28/16: More on the architect and architecting via Glenn Vanderburg:
3/23/16: I mentioned credibility under the topic of "architect also codes," but there is another factor at play too, which has to do with "skin in the game." If the architect is removed from the pains of refactoring, of working on improving the code in addition to responding to not just new features, but changes, and changes in direction, resentment can build. Sure, this is in the relationship space and there are different ways to build and retain great relationships that don't necessarily mean taking part in the grind of dealing with aging "legacy" parts of the system. But we are human creatures, so mucking in and feeling the heat, living by example of cleaning up messy parts of the code, can help build goodwill and be an opportunity to share know-how and know-when and -why.
3/23/16: This observation about microservices at Spotify is a reminder of the conservation of complexity:
Come take one of our workshops, and tell all your collegues and friends!
Image (right) source: Melvin Conway, How Do Committees Invent? 1968
First, Conway's Law:
Here are some insights from a post I wrote on Conway's Law back in 2008:
** That is, to the extent that managers influence and shape roles and responsibilities and communication mechanisms and paths, etc., it is useful to notice that managers are architects of the organization. Of course, as with other architects, there is a spectrum of possibilities in terms how that architecting is approached; in shaping the organizational structure (at various -- potentially dynamically shifting -- fractal scopes) and dynamics, managers-as-organizational-architects can focus more on shaping the context that enables more emergent self-organizing, rather than determining team structure, individual responsibilities, and interaction paths, fait accompli.
How about this, titled "Strong Opinions"? ;)
Or, this from another musing on technical debt:
In saying "I'm done with tracing," I demonstrated the value of what I do, just as surely as the silence on what I've done, proves that it has no value. The conundrum of being a me in this field! :)
“You live out the confusions until they become clear.” -- Anaïs Nin
If you're internet-familiar, or better still, f2f acquainted with the Schmonz, you no doubt appreciate some or other facet of the awesome he brings to this world. Here are a few:
Raising a glass of virtual bubbly in celebration and honor to you, Amitai. So glad you're sharing this little ride on spaceship earth with us!!
Schmonz is ma homie, but I shielded him from actually meeting me for couple of years. But it was a lovely coffee break that extended over 2 coffees. :)
My sentences are hard to read (hey, GeePawHill done tol' ya, stop yer yabbiting!)? So are people (hard to read because multi-faceted, dynamically shifting and adapting to varying and various contexts), people. We have to allow a little unfolding, no? Read me, and FP will seem trivially transparent. ;)
Also, I should use a spell checker to get my back on typos? You're so nice.
I keep using that word? You don't think I know what it means?
Well. Actually. ;)
I mean it playfully, in the sense of being pervious. So many people are impervious -- not just to me (which is, naturally, understandable), but (me pointing to) to others, to ideas or ... playfulness outside their brand of playfulness. :)
Well, that's not how I would put it... Strategy is about determining how we will compete (and if competing is the dominant aspect of how we play the game in the first place), so strategy has extended focus not just in terms of time, but takes a wider lens on the current context (ecosystem, emerging trends and opportunities they yield to create new kinds of value, threats, etc., etc.,). Tactics are plays within the context of, and shaped by, strategy. It is, I think, worth noting that an organization's strategy may be entirely emergent, ad hoc, accidental, "shoot first and aim later" kind of thing -- in which case it is simply the emergent thrust that comes from all the tactical action taken without explicit, mindful strategic course setting and context shaping. Alternately put, strategy then has no longer nor wider view than tactics (taken together). It is the degenerate case ... in the limiting case sense -- but can also be in the devolved sense too, though it can be quite effective, or at least effective enough.
Okay. I fixed the internet. We can go back to work. Speaking of which, want to do an Enterprise or Software Architecture Workshop with us? We are awesome at everything from strategy to mechanism design so, you need us. You really do. :) I mean, I can show you how Wardley Maps are great (think capability maps, then also think about where each capability is in terms of evolution towards dominant design and commodity/utility provision) -- and why there is no one map to rule them all, no matter what residue the echochamber leaves you with. There, fixed another problem on the Internet. ;) In seriousness, there is no one architecture workshop to rule them all either, but damn, ours comes close. Just kidding. But it is awesome. Really!
It seems to me that many of us who are/remain active on twitter, are independent consultants and contractors. Of course, we, like architects and developers within larger companies, are on social media to keep our finger on the pulse of change and to tap into serendipity, so to speak, learning about this or that piece of work that busts us out of our thought canals. And so forth. But independents have some unique needs in terms of camaraderie (client engagements may be matters of days or weeks, rather than years) and reputation development (branding, as a path to gaining client access, to pay the mortgage and school bills). Because a lot of us are in the same boat, we tend to act competitively... which is to say, we don't foster one another... We aren't very good at reciprocating, let alone "paying it forward," doing something nice to help one person gain more audience, without any expectation of a return gesture.
But we don't have to orient competitively. There is so, so very much to be learned and done to enhance our effectiveness in creating great software and systems and/or helping teams do that, that we can orient towards one another collaboratively, mutualistically expanding the field rather than competing. Simon Brown and I, for example, may seem like direct competitors because we both emphasize the visual in architecting. Still, we mention each other's work and workshops because the bigger competition we face, is simply the drag against architecture and visual modeling as a medium for architectural expression -- thinking, thinking better, communicating, communicating better. And so on. We already know this, but it is still worth looking to Elon Musk's orientation around "All our patents are belong to you." The ecosystem needs to be grown, and that is bigger than one or two or even many many of us can handle. The ecosystem? All of us involved in creating-evolving software, and getting better at creating great software. Figuring out how to create software that meets needs, and stands up to stresses and strains as demand (features/scope, and numbers of users) grows.
"Indeed, the more people talking about architecture the better. " -- Simon Brown
I also mention or retweet work in our field to help tend the attention commons. To lend support to ideas and their people. It's good to have this cross-pollination happening between architects, developers, coaches, managers, people in other fields, and more. And it is great for independents doing intervention-styled consulting work, because we can be isolated from ongoing, longer term peer networks unless we make it happen for each other. We are social creatures, and we need some ongoing, longer term, social embedding.
So, in short, it's great when we play nice. And arguably more important to do, than to guard a little corner of the field.
Use it, or lose it, as they say:
"Slack will obviously replace Twitter." -- @noahsussman
2/24/16: As voices in the architecture conversation go, here's:
Rebecca reiterates Martin Fowler's characterization of architecture as "things that are most difficult to change" or "the things the organization is most worried about," Those factor; I prefer ours (smile!), naturally. That is, when I say structurally and strategically significant, it's scooping up and including Martin's change and worry stuff, but also going after questions like
I wanted to write a book called "Getting Past But" with a picture of an elephant's rear end on the cover. I didn't entirely fail, in that I did write an EA executive report (think short ebook) for Cutter by that title. And, if I say so myself, it is delightfully consistent with its title, evading all the usual "buts" like "But, you can't use a children's story in a paper intended for IT executives and architects." I would write it differently today -- but heck, I'll revise what I wrote this morning tonight, so it's just a sign of updating one's understanding. Spin it, Ruth, spin it?!?! There is a lot in it I still like, including the use of, and generally speaking my debrief/discussion of, the children's story; The Wheel on the School is well read as a charming book on dynamic agile teaming.
Now, you need to understand that I very, very -- and I do mean very -- seldomly get paid a compliment. It seems that on the occasions that people engage with me about my writing, it is to tell me what I missed, or misstated, and that is awesome, because it improves my thinking. But the affirmation is indirect -- it is not nothing to be taken seriously enough to be read, on occasion, by someone, and you at that!! So it is an implied compliment. I say all this by way of context: I am no expert in the matter of receiving compliments, and when I do get paid one, my deflector shield pops up automatically in the form of a joke. Like this: "does my but look big in this compliment."
That's from 2008. I've been around all this time, and you didn't even know! And here's a (related) trace from 2009:
Ooo... that sounds like a trace I should write! Feature toggle, but on the trailing end. ;)
If I was tracing still.
I am? Who knew? ;)
What's that? Michael Feathers already wrote that? You mean this:
It is a post worth engaging with, entering into discourse around. Share ideas and practices. Microsoft has some experience with trying to turn off legacy OSs, and that's hard, hard, hard, due to entangled dependencies. It's worth visiting the question though, in the light of microservices. (Don't groan.) I have gotten wind of some who do view DevOps and microservices as the secret sauce in which features are not just brought to life in bounded experiments, full roll-outs and operated through scaling and other challenges, but also decommissioned or end-of-lifed by DevOps, as new versions are brought into the lifecycle. Having "fit-in-our-head-sized" chunks of code that we manage birth-to-death has its appeal, but we know the challenge -- scope creep at the microservice level? I'm thinking that too. And some other things, that I'd write if I was tracing, 'n all.
As a side note on that post, about IT as a cost center: When IT is a support function to a support function (like HR and Payroll, Accounts Payable, etc.), it is readily viewed as a cost center. And that is the legacy framing for IT. Even when the software-intensive systems are the revenue engine of the company, this may have snuck up -- insurance companies, for example, may not conceive of themselves as software companies even as much of their customer interaction moves to digital. Data warehouses are well behind the corporate wall from the customer, but analytics turns the crank on personalization services. The boundary between "obviously" and traditional cost centers, and customer-facing, revenue-generating services, gets blurred.
And this -- along with the resources at the end:
2/18/16: Python has EoL schedules?
2/18/16: IT as an enabler not just a cost center? See Fractal and Emergent -- yes, you have to give them you contact info to download it free. But please note, you have to do that with LeanPub or Amazon AND you have to give them your credit card info. And yes, it is worth your contact info. It's okay that I wrote it in 2010 -- you only just caught up to where I was then. JUST KIDDING. Sheesh. But I do still like a lot about it (and stuff I cringe at, but hey... I could make you cringe at stuff you wrote today, so let's not go there ;). So there's that. And it's FREE -- well, except for the price of your contact info that we just talked about, and the price of your time reading it... which is an investment. View it as an ENABLER, not a COST. Wink wink. Okay, Okay I have to quaint so you don't kill me for being assertive. The boatanchor we make women swim with, in a world of dominated options... Blah blah.
3/23/16: More on killing code:
And killing projects:
Cost of carrying code:
"An architect by any other name".... Does the title matter? No. Unless it does. ;) In some situations, the title "architect" has been tainted or sullied, so "senior dev" or "tech lead" or some such -- or even nonesuch -- is used instead. Or we're figuring out how to be more egalitarian, and doing away with titles that implied, or full out meant, hierarchy. I'm down with that. We have to deal with what we have to deal with, and we need to experiment with new organizational ways of being and doing, to progress. If things are going well, why mess with it? And if they're not, we may have more pressing things to deal with... But. It may just be that we need an architect! That we need that strategic and technical leadership and system design mastery and focus.
The title isn't about making all the decisions and taking the credit for the team's work. It is about a role that works to enable the team and to watch for what needs to be enabled, and what needs to be ruled out, out-framed, constrained. This has a strategic dimension to it -- doing things that really matter; doing more of the right things, and fewer of the wrong things, not by magic, but because experience and strategic sense. ("Because experience"?! Look, I write loosely here, to convey that I haven't hardened my thinking into concrete. As those who interact with me know. Thanks @arctips (Eugene Barker) and @brainwork (Peter Bakker) and @bmichelson (Brenda Michelson). And Dana Bredemeyer, obviously.)
Now, I would suggest, the title "architect" indicates charter and opens up the scope of contribution more than "senior dev" -- both have that dimension of technical expertise drawn from significant coding experience, along with smarts, investigation and curiosity. But architect throws the emphasis onto system design concerns, and, we argue anyway, playing a role in bringing technical considerations to the strategy table (at the level of scope the architect is working at -- product strategy for product architect, business strategy for enterprise architect, etc.), and strategic considerations to setting technical direction and shaping the course of system development. I think we need senior devs -- we need a career path that allows technical people to stay deeply, focusedly technical, and be valued for the mastery they develop. Hard systems things are hard. We need people who are really, really good at technical inventiveness (in small local problem-solving ways as well as large industry/landscape reshaping ways) and solid, elegant coding. And we need architects, who have the credibility in the technical community that comes from that technical mastery, but who also want to lead -- want to work by enabling, organizationally as well as technically, to be strategic, and lead architectural (system focused) design and do the work -- the real and ongoing work -- that entails.
Recall: the system needs to be attended to as a system. For emphasis -- if this is not someone's responsibility, it is no-one's paramount responsibility, and it will lose out in the day-to-day mull, to more local demands. If we focus on parts, without system design context, we are in danger of ending up with "parts flying in formation, trying to be an airplane" (Wim Roelandts). Some teams, especially small teams with lots of organic communication and mind-syncing (over a mindful of code), can navigate this balance, holding in creative suspension, and resolving the tension among, the demands and forces on the system -- and move work forward on the parts. The team really can be, and is, responsible for the integrity of the system. We have to consider how this scales, though, for more complex systems, of multiple parts, features, microservices, whatever the chunking du jour, ...
Further, when something has much to recommend it, but it isn't getting any traction, something else is going on (hat tip there to Dana Bredemeyer). Sometimes, [a figurative] someone has soiled the bath water and we're wanting to toss the baby out with it. In this case, in our binary zeal, we tended to collapse we "value x over y" into "x not y" and tossed architecture under the "comprehensive documentation" bus. I'm oversimplifying our tendency to oversimplify. :) But some of what we've had to deal with here, is the notion that architecture is entirely/best left to be emergent -- from (and documented by) TDD and the code itself. We course-corrected back to "well, actually, some architectural thinking-designing-deciding is important." But we've left some in our field back in the wake of "architecture? not so much," and they'll have to learn some lessons the hard way, and make their way back to "architecture?!--how much?"
Still, some of the push-back against having an architect is more close to the team -- it is political...as in, about decision power and tipping power balances in the organizational landscape. It is worth asking ourselves, in tossing out the "architect" title, are we trying to obfuscate an unwillingness to let someone (else) lead the system design effort? More than one person on the team may have the technical experience, system design talent and organizational effectiveness to be the architect (or tech lead for a subsystem), but it is still useful to have a designated (lead) architect, to break logjams when different alternatives are being advocated, or we just need to move forward, and we don't fully know yet what the "best" avenue of address is. The (lead) architect also serves as a boundary spanner and emissary of sorts, interacting with other architects and executives, on strategic and cross-system, or system-of-system, matters.
I'm fine with everyone on the team doing architecture -- it means more people should take our workshops! ;) But complex systems are complex -- technically and organizationally. Much system understanding is moved in interpersonal conversations, but also written expressions like documented architecture decisions and code (and tests) --- and mulling and mulling over those, imagineering what they mean for where the system needs to go, sketching prototyping and coding this or that throw-out experiment, writing up this or that mechanism or other architectural decision to get it expressed and improve it, making sure those who are differing on a matter of architectural import talk it out and resolve it, and more. Again: this is work, real work! Which takes time. Real time. And the ability to discern where to spend time.
Many are involved in various aspects of the design and coding, testing, deploying work, but it helps to have a locus of system responsibility to make sure that the system thinking-designing-deciding happens. Does everyone think they can, should and want to do that? Often when push comes to meetings, some want to protect their coding time and let someone else do all that listening and reading. And those who are more senior, have more hooks to hang new learning on -- they have more learning and understanding under their belt. They're at a different stage of integrating a lot of experience of systems -- technical and organizational -- into the understandings and abilities we associate with mastery. Which for an architect is active and ongoing, constantly learning everything from what's happening in the business domain, to investigating technology directions, emerging technology, reading academic papers on mechanisms that look tantalizingly like they might solve this scaling problem or that consistency challenge, reading code including other systems which have done things elegantly, or are hitting some of the same challenges, etc.
Creating value is everyone's responsibility. Quality is everyone's responsibility. Architecture is everyone's responsibility. Yes! But we can still value leadership and a center or locus of responsibility, while at the same time knowing and appreciating that value, architecture, quality is everyone's responsibility, with everyone participating and contributing. Knowing things takes time and attention. So does doing. There are lots of avenues of learning, knowing, doing -- the value network and competitive landscape, user experience and design of system capabilities, system design and code (with tests), integration and system testing, evolving the deployment paradigm and infrastructure, data, oh just ever so much to keep up with understanding the state of, and inventively evolving, to add value and sustain the business. We can't all know all the things and get serious stuff done -- in competitive timescales. Let's take another setting where the work of many mind-hands contributes to something greater than one or just a few can accomplish -- an orchestra. There is no sense that the conductor accomplishes, nor gets, nor deserves all the credit for an orchestra's performance. It doesn't have to be spelled out to us. For groups of musicians larger than the number who can reasonably directly see one another, a conductor is assumed and appreciated. That is true, even given a score and much practice together! So we can argue that we should only do small things -- only have small jazz bands. But orchestras contribute something different. We value that too. Not instead.
If we value mastery, then might we not value technical mastery and effectiveness in leading technical teams? It takes focus and an investment in personal development to develop this extensive and deep (business and technical domain) knowledge, awareness, insight, wisdom! And it benefits the team (of teams) to have this mastery in creating-evolving systems with integrity (structural and design integrity), agility (responsive to change, including sensing what to respond to) and sustainability (in all it's senses, including economically, technically, organizationally, and environmentally). A good architect -- one who really supports the team, mentors and grows the team, and facilitates their success, but gets out front and leads where that's important -- is an asset to the team, in the sense of magnifying the value the team is building.
Magnifying, by helping the business team shape strategy, so that we do what matters -- it differentiates but also contributes value -- and we don't do what doesn't add substantively. [Again, here I am talking about the business team at the level of scope under consideration -- if we're talking about a product or service, then the business team at that level; obviously a product architect isn't working with the executive team, unless it is a one product company.] Strategy demands situational awareness, and technology is part of the context. Architects bring not only knowledge of the state of our current capabilities and systems, but also what we can leverage, to create, shape, and respond to market opportunity. [Aside: One of these days, you will have the good sense to ask me about slide 91 and 93, among others. ;-) But I'm still waiting for that someone who has read the slidedoc e-book, to ask me to go on. I mean, reading 57 pages and paying a few compliments is expensive compared to paying $15 or $30 for a work in progress book on LeanPub, but someone must be willing to pay that price? No? Then I'm not willing to write more. I think that's a fair way to set up trade. It's all about my ego, anyway. ;-)]
We have created these mythical monsters under the appellation of "architecture astronaut," or similar, who are disconnected from project reality yet operate in a highly authoritarian way. Some of us may even have glimpsed such monsters, but usually the reality is more nuanced and textured than the stereotype belies. When you investigate what architects actually do, when they are effective, and trying to be more so, they aren't psychotic authoritarian figures nor highly manipulative authority abusers. Why not? Organizations are political (in good and bad ways) -- we are social creatures and work within webs of formal and informal networks (including hierarchies), and in tech we just tend not to tolerate autocratic rule...for terribly long anyway. We may agree to follow a "benevolent dictator" but much is conveyed by "benevolent" -- if the dictator is acting in ways that disserve the system or its people (developer or user communities), then implicit agreement to follow starts to break down. Smart people are smart. Loyalty and following a dominance assertive leader certainly factors (we're human), but failing to serve raises opportunities for other leaders to emerge. So even at the extreme, extremes don't tend to last.
Creating straw man architect stereotypes is not cost-free fun -- we simplicate real people into the bucket of the stereotype, and make it harder for them to rise to the occasion of their role. It is an intrinsically challenging role. Non-trivial. Strategic. Deeply technical. Organizationally demanding.
The effective architect, works together with others, leading, following, and getting out of the way. Doing a mix of coding, mentoring and pair problem-solving/programming, scouting and investigating, deciding and documenting or simply watching to ensure that gets done, working across the organization as fits the demands of the moment, and more.
We need to do the right things -- a strategic matter that is not just about user needs. And do them right -- which takes technical and organizational savvy. Organizational, because it is not about doing everything or even the coolest things oneself, but growing the team, enabling, coaching, and sometimes leading and showing. As meets the needs of this extraordinary moment.
This trace wants to become my Trace! Here are some snips from past traces:
and this on strategy from 12/28/11 (the whole trace is probably worth returning to, but here's a taste):
I need to pull these threads (on the architect role) together into something more punchy. The image alongside was Sara's response to my Trace back in 2011 when she was 10 or thereabouts...
Hey, no need to agree with her -- my inner voices have already danced all over my ego enough as it is!
I mean, sheesh. Two people summoned the kindness to say something about my Trace on the grand occasion of its 10th birthday. Let me repeat that for effect: TWO people. It's a lovely world, isn't it? But get this. We aren't all that good at celebrating people on their birthdays, so my Trace is in good company. Gosh, we could be nicer to one another! :)
People who give credit, who mention others who have influenced them positively -- they do good in the world.
Still munching on that "do we need architects?" question. We learned that solely relying on emergence of architecture from disciplined refactoring and TDD -- with a strong test driven design flavor added to the usual TDD mix -- may not be the best route. But now in the microservices context, we have a fix for that in the form of DDD, so we're good on design, right? No architect needed. Next!
Not so fast. Architecting is about a lot of things. Not just decomposition (to tackle complexity, with all that it implies cognitively and organizationally). Not just composition and orchestration. Important as that is:
The conservation of complexity point, reminded me of these, by-the-by:
Architecting is also about identifying and addressing technical challenges arising from system properties, interaction with context, more. It is about the design of mechanisms that resolve various interacting forces on the system to achieve more the desired properties and less of the undesired. And it is about sensing and identifying what is strategic -- in business terms, and in technical terms. Strategic, as in: How are we differentiating ourselves? How and where do we add value that is valued (as in, people are prepared to not just spend money, but integrate this system or product into their work or personal lives)? Is this approach sustainable -- for our internal development community as they live in this code base for years, and years; for our operations team as they scale and respond to snafus (I have a story!) and more; for our users as they live with the system and its entanglements into their systems, impacting their days and lives; and the planet more broadly. Have we added positively to the world? Or is the net effect meh, or worse, incurring negative externalities that the planet has to pay for down the line, with nothing more to show for it than some short-term hoopla... (sorry Silicon Valley; we love you, but some of what you do... you should hold yourself to the wall about, and wonder why! ;)
One of the first things we do in architecture workshops is a context map. Not a map of the infrastructure, or immediate technology context of our systems, but a high-level ecosystem map. (Depending on what we're doing, we may do other maps such as value flows and transformations, etc.) This can be quite uncomfortable for entry-level architects or tech leads, because it makes clear all the space in the landscape they have been relying on others to have a handle on. Up until that point, they've been focused on code (and closely related concerns), which is goodness -- it takes time to gain code mastery, and architects should have a level of excellence there, should have developed a sense of what is challenging and how to approach challenges in code. To be considered for the architect role, they need to be good at problem solving, in code and by being great at problem identifying and defining (tests in TTD sense, but also algorithm designing, and deciding what not to do and...)!! Now, if they'd gone all out on understanding competition, they'd be channeling marketing, not developing code skillz. I'm not suggesting that! But as an architect, we need to have strategic sense, to do more what matters and less what does not. That. Contributing to strategy setting, at some point. And making sense of the strategy that has been set -- we have to have some understanding of the playing fields we're adding value to. Many software design approaches go as far as investigating what users need -- and even that is further than where many organizations let their developers go. In counterpoint, we insist that architects gain that "situational awareness" that takes a broader view (just enough -- just enough, for goodness sake!) even than "what do my users want--right now" (or what does my product owner/manager think they want). Why? Because what we do with technology today, is going to enable and constrain the business. We can try to make our systems more responsive to change, but we need to recognize that our starting points have a tendency to limit where we end up. They canalize not just our view, but our users view. And so forth. So strategic direction setting is non-trivial, and just as important for the relationships we set in place in our technology base, as they are for the relationships we set in place in the broader ecosystem. These things interact.
We have long (even published as a paper in 2002!) advocated minimalism when it comes to architecture ("less is more with minimalist architecture"). But Jefferson and Lincoln and other US founding fathers were well ahead:
So. I wouldn't rush to put all of a field's bets on any single pony, whether it be AWS (good as Amazon is at cloud) or DDD (it's an approach, not a panacea), or TDD or anything. We run diversity out of our field if we all go tribal on the same things, or treat any approach as "best in class" "best practice" "safe"... We end up with only "Cavendish bananas"... in our fruit salad... :) We need a little place in the world for the Bredemeyer/Malan goodness to earn its keep, by helping organizations be (more) agile, by having the architectural wherewithal to flex, respond and adapt, where needed. Remember, we were talking "Fractal and Emergent" in print in 2010 (that long ago? I need to write more!) -- that is a lovely paper. Okay, I'd write it differently today, but it speaks well to 2016 all the same.
2/17/16: Neat idea for architects: need to bring the architecture documentation back up-to-date/in sync with code -- hold a "write-design-a-thon" (hat tip Sarah Allen) Oh, we've done working sessions with teams on architecture recapture since the mid-90's -- but it's a contemporary name for something with deep roots.
2/18/16: Matt McLarty was asking "We're discussing your blogs on the link between designing systems and orgs. Are there examples of orgs who have done this well?" To clarify, I asked if this is what we were talking about:
I think this is in the territory of "the answer to how is yes" (to run off with a book title and use it my own way, to my own ends) -- in other words, Conway's Law says there is an isomorphism, and it will be driven by the organization (structure and communication paths and mechanisms). The question is, are there organizations that use knowing this, to get more what they want from the system, by being more intentional about the organization. And again, yes. We used to do this with modular monoliths and upfront design -- whether the upfront design was by managers determining "pizza-sized teams" based on gut feel (and dominant design of similar systems in the organization or the industry) or architects designing the system and helping managers form teams based on the architecture. I can't talk about our clients. Knowing how smart architects and managers I've worked with are, of others, let's just say: I would assume there are those who do this in the context of microservices, and they aren't just waiting around for consultants like Ruth to tell them -- though Ruth's lovely Conway's Law post has been out there since 2008 and that no doubt has had its impact too. ;-) I would also assume that things get messy, so it is hard to find "clean" examples of anything, over periods of time.
Anyway, I think the impetus of Matt's question is right on -- organizations that think about designing the software system and its people/development system together, are going to have these systems work synergistically. Which is the essence of what Michael Feathers is on about these days. (That's a nod to go over check out what he actually wrote -- it's bigger than what I just gave lie to. :-)
Before anyone runs off with all the territorial flags though. ;-) See for example the 2/3/08 trace that then runs on to a Conway's Law trace. And seeking synergy between business and technology is a dominant thread of our (enterprise and systems) architecture work throughout -- bringing technology to the strategy table (at every fractal level) is something you'll see in all of our papers (and my Trace). We have done this from the perspective of the architect acting as a fulcrum in this relationship between organizational strategy and technical strategy, and organizational design and technical system design, and system capabilities design (what the system offers to other systems/users) and system design (how the system is designed internally). Oh, so now you want the notes to Slide 92 and 93? Anyway, we have been addressing this set of relationships, and the need to design across these systems for they influence and depend on one another. We're still learning, but we have been bringing Ackoff and Argyris and Ashby and Beer and on down the alphabet to systems design for 20 years and counting -- actually more in Dana's case, but who's counting, afterall. And a whole lot of unique contributions from the Malan-Bredemeyer duo, to boot. I wouldn't want anyone thinking we just regurgitate classics and contemporaries, just because they can't imagine a woman (and this one at that!) making novel contributions to a field. ;) Although it is definitely not nothing to bring it with Leadership and Self-Deception. :)
Isn't it just so cute how you had me pegged at quaint? ;-)
2/19/16: Why is strategy so important? Because we are so "yes" about this:
that we can forget, that its very effectiveness can be its downfall! Working software has a way of becoming self-determining -- it sells itself to those who're buying it. Remember, "code wins arguments." What am I on about? Working software, systems, start to shape the world around them. "Systems have needs." (after John Gall, I think) Interests become vested. Views become canalized and dug in. It becomes hard to change. Until the chopping block of competition, or higher level determination that the value just doesn't support the cost.
So, yes! But also no. I know. I know. We experiment and adapt, because lean agile. Right. We still do that within a frame, and strategic choices set that initial frame, and shape the context as the thing moves. And strategy is not just a business matter. Technology is so interwoven with the business, we have to think strategically about technology -- what business capabilities we build, enable, shape, with the systems we build and build on. (You can take a look at our Cutter reports -- the second and fourth are especially pertinent, but all have something to add.) And even when we are years into the lifecycle of a system, or family of services and products, situational awareness (evolution of the competitive landscape, value flows and transformations, and underpinning of capabilities including technology), and direction setting-guiding (what we seek to make true in the world, broadly shaping our approach, enabling that and removing obstacles, etc.).-- that is, strategy -- is still important.
So big yes on working software "selling itself" -- shaping the political landscape around itself. And a big caveat. It can create tunnel vision, and the world goes on around it, making its choices. Strategy gets us out of the hole, looking around, seeing where we want to take this thing, longer range, so we tunnel in a direction that fits in to bigger value flows, not something that is a deadend. We can take a "let the future take care of itself" approach, because it will anyway, right? Can't stop the future! So, why aren't you on at the beach today then? You are? Wait for me!
Let me take this moment to appreciate Peter Bakker! When I set down the path of talking about the architect role in the Agile context, my eye was initially on the system integrity ball. Attention shapes what we perceive and pay attention to. So my eye was not on the strategy ball, and Peter gently, obliquely reminded me -- it's about creating value. Value! Doing what matters! I am so all about doing what matters -- having a broader conception of agility, integrity and sustainability and doing what matters. In my talk in London, I told the audience that I drag architects into the strategy and capabilities (system-in-context) design space, not to leave them there, but I insist they go hang out there enough to know what its about, to contribute, and to bring back gut level insight into the broader systems, the system we're designing plays into. We wrote about strategy as an architect competency in the first of our papers that became (and still is!) quite influential in enterprise and systems architecture -- in what was that? 2004? But it leveraged work we'd been doing, and informal papers on the Bredemeyer site, from years before that.
So I know this stuff, but Peter did what the architect needs to do -- he got me to lift my head to see the bigger picture, see the things I was missing.
We need to do things that add way more value than we consume, because we're indebted to this planet and its peoples. We extracted and extracted and messed up this lovely world, and we need to do good, right, meaningful things. Or more of that, than the other thing, where we make more pollution and misery. And other social and environmental yuck. At which point I should mention Stuart Boardman and JP De Vooght, who are two of my look-to's on environmental and civic consciousness.
Also, thanks to JP De Vooght for alerting us to this wonderful paper (it would only have been more wonderful if it had mentioned my precursor -- the tweet with the above image notes that it is from a trace in 2014):
But again it runs deeper:
where I said:
Which also has history, roots in other insights and explorations. Hey, I've been doing this for a long time. Decades. One would hope it was a journey worth taking, with lots to show for it! A whole Trace worth, as you can see, and working directly and indirectly with thousands of architects around the world -- learning from them, stealing their fire. :)
"Move fast and break things" isn't quite so great sounding when it's the business (and people's livelihoods) or safety (and people's lives and livelihoods) or privacy (and people's financial stability) or the environment... Software is high consequence stuff, when it's high consequence stuff! And it so often is, even when we haven't given it much thought -- or especially then!
2/20/16: reminded of Dave Gray's interpretation/presentation of Argyris "ladders of inference" and "self-sealing bubbles."
3/8/2016: This is interesting (on Medium and their experience with holocracy):
Image: from sketchnotes by Remarker on Adrian Cockcroft's talk about Microservices, Netflix, and what he learned:
To do what differentiates, we need to know what we have to build with and on, and know what is (and is likely to be) valued, and what "unique" is, in the competitive landscape/value networks we play in.
Wait! I should change the name of my Trace to Perspectives from Outside the EchoChamber. ;-)
Ambi-valence. Multi-valence. More than one center of attraction. The state of having mixed feelings or contradictory ideas about something, that arises from things not being all one good, or another, but multinatured. With tradeoff spaces.
I risk saying the unpopular thing, but nonetheless: in many contexts, the architect role is too big and too important to be smattered across people. It doesn't mean others don't contribute to the architecture -- they will, regardless! (I am not recommending it, but even if autocratic-architecture-by-dictate is attempted, misunderstandings, accommodations "just this once, just for now", and wotnot will rub at the architectural lines.) It simply means that someone (and for very complex systems -- an airliner, say -- a team of architects and lead architect) is "getting the system's back" so to speak; going to bat for the system, weighing in for the system against the tug of parochial vested interests. Not in opposition to the team, but in support of the team, and gaining support for putting in place what the team will be needing, etc.
Architecture is about seeing (more of) the tradeoffs, seeing beyond the seeming crunch of immediate demands (now seems so urgent, but we have to live with decisions we make under apparent now! stress for a loooooong time). Yes, it takes imagination and perspicacity, and yes, there will be things we'll be wrong about -- but better to be watching for what we are wrong about, than proceeding headlong at full pace into a wall of we-didn't-think-this-through-debt!! But competition! Yes. The tensions in sustainability include surviving the short term and building a future we want to live in -- that is, entails proceeding as wisely as we can, recognizing that even the past is different than we perceive it to be! ;) But yeah, architecture is about tradeoffs and resolving tensions -- technical and organizational. All of this takes attention and development of skill, experience, perceptual acuity. And doing things well, takes becoming good at it by doing it, and doing more of it -- investing in developing the technical wherewithal but also developing in organizational and strategic effectiveness areas too. And all the other stuff I said in several traces this month. Ugh. Must refactor and simplify!! ;)
I love the idea of self-organizing, and have always been very self-determining in all the roles and positions I've held. I think that in software we very often get to define our jobs by what we actually do. And within that value frame, having clear focus has also helped me be self-defining -- I have been able to do that "what at this extraordinary moment is the most important thing for me to be thinking about" thing, which is about judgment and navigation. I've never just done what some "guru" in the software intelligensia wrote in a book or whatever -- and I don't see other people doing that either. We take suggestion, we follow lead. But we use judgment. We apply and blend, hybridize what we already know and do, with what we learn from others. We try things out, and if they don't work for us, we push back and push on.
"Architect" doesn't have to be a position. But to suggest it should not be, is ... oh, yeah, remember this:
2/20/16: Yes, this (@Abby_the_IA):
And it is important to give attention and expertise their due. If we want everyone to be the same -- same level of experience and expertise, same focus, all work on the same thing... that's... one way to go... Generally though, the very people promulgating such an approach, want to be followed. Set themselves up as experts. Want to be heard. And can be quite shouty, assertive, even aggressive (in condescending, defensive, etc., sorts of ways) about it. ;) Humans are social animals, cognitively limited, and time bound. So we figure out how to work together on doing bigger things than one or two or three people can do. This means collaboration, but also coordination -- in the systems we build, and in the systems building them. We can push more into the environment, but environments can then be felt to be coercive too. Imagine being an ant! They achieve amazing things, but have limited degrees of design and social freedom.
Yes, yes, yes. One paragraph became many... Which I need to pound down again. Because people are busy, and brevity has its place. This is where I started:
I guess it really did the job. I could have saved myself a good dozen paragraphs! But you. Well, if you read along, you got a lot more, right? No? Good gravy, go back, go back! All that stuff about strategy and context? About the social dynamics of binary positions that draw clear tribal lines? Whaaaat? ;)
Ps. Love those architecturally significant design discussions that happen on Twitter and Stackoverflow, etc. :) "The price of"... "source of truth"... smart humor with a pointy point :) and "still need" and back to elaborating the tradeoff space...
Pps. A good architect isn't a (Robert) Moses (all the time)! A good architect, I think anyway, is balancing fractal and emergent unfolding. Doing just enough context setting (so more good things happen, and less of the bad things) and infrastructure work and so forth. Using the lightest touch approach that moves things along, in the direction of getting more what we want. Where "we" means balancing across participants in various systems, internal and external -- the code/dev/ops (eco)system, the system of systems in which our system is used. Where moving things along includes building systems out, but also being deliberative. Making deliberate discovery a way to have "oh crap" moments earlier and more cheaply, from the perspective of lessening their impact (when the (side-)effects are negative) and entanglement in systems and expectations.
“But it’s not very useful to look at them as ‘pure’ Jane Jacobs and ‘evil’ Robert Moses.” -- Joshua Frankel
Trying to decide... May 10-13 in Chicago/Schaumburg or Boston/Dedham (MIT Endicott House -- out of the way, but lovely venue) for an open enrollment Software Architecture Workshop? The folks on board so far, don't mind either way/are traveling in anyway. So, quick, quick, if you're interested, tell me!
Interesting, useful musing (for coaches -- including architects) -- and fire worth stealing (just change "patient" to system):
Steve Merrick, aka @patternchaser, has started blogging on things firmware and design, but really, it's muchly about software and design so go check it out regardless. It's shaping up to be a wonderful contribution to the space, so clue in on it early. :)
And if you aren't following Peter Bakker, aka @brain_work, what in the name of all things design are you thinking?? Follow him! He's is good people! Best people!! I mean, he has been nice to me for years -- almost as long as Daniel Stroe (who alas isn't active on Twitter any more -- but Daniel is in embedded systems, so maybe Steve Merrick and Daniel can create some critical mass on twitter??).
Being nice to me is a reasonable qualifier for following? OBVIOUSLY! I mean, consider -- are you nice to me? If yes, you're in good company -- few, but such fine people! If no, then you have extra reasons to respect those who are! ;)
Okay, in seriousness, Peter investigates topics that interest him -- visual in design, these days, looping back to his work on tubemaps in system modeling. (A few years ago, Peter used my Trace as a demonstrator for his tubemapping work -- which meant he did me the honor of actually reading traces and threading them into storylines. So, you see, he's smart, kind, and...disciplined!)
Oh, Peter's tubemap led me back to reading (a 2006 trace and):
And. Gene Hughson's architecture blog. Upon a time, he quoted me. So he's been slacking, but is still worth reading. ;) At least he retweets me more than anyone! :) He's good people, and smart, and doesn't get as much following and community-visible fist-bumps as his insightful blog and conversational peer-coaching in the social-verse deserves. (He gets a lot more than I do, but I'm handicapped... shhhh, don't mention it in public but.... I'm "female." I know right? jk. He blogs. I trace. It's hard to be appreciative and enthusiastic about a Trace. Evidently. jk. His blog is wonderful! Mention it sometime.)
And GeePawHill -- but he got a special featurelet yesterday when I stole a coal from his fire. That musin' was a lovely example of what Michael Hill does on twitter. When I have hope for twitter, it is in good part because people like Michael craft a distinct, compellingly interesting voice on twitter. And he does it to 11!
And Grady Booch has mentioned, retweeted and linked to my Trace. Grady and Peter Morville and Andra Sonea, are, I think, the only tech intelligensia with more than 10,000 followers to ever retweet me. (Well, I could have missed some; follow count and other status markers aren't really something I look at. No really.) So clearly their good sense recommends them to you, and you're following both Peter and Andra, right? You're obviously following Grady! (No?! Take yourself out to your woodshed!) It's almost Grady's birthday, so get your words of celebration ready!
Besides, we (women too) all need to try a lot harder to be part of the progress towards valuing women as contributors, not just bit-part players who should be seen (iff eye candy) but not heard, so make sure you aren't holding women to a much higher bar when it comes to who you follow and which tweets you read, and retweet, and so forth.
And lastly, because awesomeness, Eugene Barker, aka @arctips, and this blog post, which is a nice capstone to my architect ramblings this month:
2/22/16: I used to think (something like) this (too):
Now? I think the weaving is thin. It is without real caring, without any real sense of mutual responsibility. It is easy not to be considerate, because we connect with some thought projections across the darkness of digital space, not with flesh and blood of emotional people. (And yes emotional! No matter how unemotional people conceive themselves to be; there's even emotion in being judgy of emotional people! :) So it is a dangerous weaving, because we put time into illusory tenuous digital connection, and that takes time away from richer bonding in face to face connections. I don't think it is hopeless, though. I think we have to remind ourselves to be considerate -- to take in, account for, the humanity of people when we only interact with their words -- words on our screen. Words without the personality even of handwriting, without the little choices of paper of different textures and hues. Without seeing a little personal quirkiness... Net net? I think we need to recognize the thinness, and make it a thicker medium, By simply being more considerate -- I'm not speaking to bad behavior. I mean those of us who are trying to be good people (and hard it is! what even is good?). I mean taking the humanness of others into consideration.
It's been a really slow month as far as visits to this site go -- that, despite its 10th anniversary, lots of (surely great and useful?) writing, and a kind retweet by Andra Sonea and Peter Morville, among others. I think I have reached peak invisibility. Well, there's you, but you have special powers and can see invisible people...
Generosity is a renewable good.
In the trace that set my fingers off on the whole "to architect or not to architect" question, I wrote:
Does that ring any bells? This perhaps...
Clearly it is not saying there's no place for documentation... So backs are covered. And yet it tips the hand, puts the weight on, working software. When we value good x over really bad y, then y is tossed out, because we don't want bad y. To see my point, let's phrase it like this:
Now people are asking "what's just enough?" "what documentation?" It's not as zingy, but it's not as zingy because documentation is still in the picture and we have to figure out what documentation is critical and suffices. We might argue that the second is the intended meaning of the first, but the first has the effect of tipping the hand in such a way that documentation is tossed off the slate of concern.
So we've been having to drag the just enough documentation rock back up the slopes of Snowbird. Or something. ;)
At any rate, the net effect was tossing the modeling baby out with the BDUF bath water. Poor baby. It hadn't learned to control it's bowels yet. Or we hadn't (all) learned to control our analytic zeal.... It didn't mean that UML is bad, just because we overused it. Sure, it is more expressive than we need for lightweight just enough modeling. But I don't use all of English -- at all, and certainly not in a given piece of work. I do wish we could have had an open forum for managing our Modeling Language definition effort, rather than an elite country-clubby sort of thing. But an evolving "Oxford Dictionary" -- or better Urban Dictionary? -- of software modeling language has merit. [I know, some duplication/overlap with an earlier trace... I need to refactor and cull. Get rid of some bathwater and babies. ;-)]
Okay, okay. Sheesh. I don't really mean a phrase was responsible for a sea-change in the field's trajectory! Even if it was a phrase in a document that was a highly influential reference point for our field. But it is a banner phrase, returned to often, and representative of many conversations and other messaging, that meant that many, and then more, tended to view code, and tests, so code, as the way to design-develop software. Now modeling is back in more and more conversations about software -- as informal sketches we draw before we code some gnarly bit, to get "on the same page" about a chunk of the system, etc. I'm by no means disparaging that! Quite the contrary! I'm saying it is where we should have been in 2002 (and where many of us were, obviously -- minimalist and just enough were deep in our DNA). And as we rebalance further, we'll start to draw more -- draw, and draw on modeling, on visually representing some key facets of the system, to think them through, improve our thinking, make it our thinking, and share it with our downstream--that is, future, selves. More -- as in, just enough. :) [We were "just enough"-ing with Team Fusion in the mid-90's, hence the "deep in our DNA" reference.]
We are intelligent thinking creatures. We can also be influenced. And we can especially be influenced to do what suits us -- that is, seems to be in our interests. That tells us what we want to hear. That gives us a chance to lead our team on mission, chase the holy grail, pursue the silver bullet! Well. Okay, I put that too colorfully. Got carried away. O.o-ps. :) Anyway, our path tends to be kinda zig-zaggy. Object technology comes along, and modeling was coming to a head just in time to help us think about objects? And knit the domain to the code, so that domain changes could transparently shape code changes. How cool was that?! I mean, doesn't that sound cool? So we overdid. So we pushed hard against overdoing, and overdid in the other direction. "Working software over comprehensive documentation" was probably the banner call we needed, to counterbalance too much paper design, and get back to more design by means of code and test -- test in its local and broader sense of testing it out in the "real world." With air quotes, because A/B tests and other experimental approaches, so a testbed for the broader real world demands, values, stresses and strains. Now, in the rear view mirror, we see the broad patterns of zig and zag, but it's also true that many don't go all the way to the extremes, Partly, some are just saved by inertia of big. Others are just using judgment, learning from the shiny new thing, but incorporating it judiciously into what they do. Judgment always factors. Unless it doesn't. Tracing, for example, is both. It's a very rash thing to put thinking in a (potentially) visible spot (off the lesser travelled routes) on the i-way. But it tries to be judicious.
Phrases our culture undervalues! How the heck do we learn if we don't try? How the heck do we take new avenues of address, come at things from a fresh perspective, if we don't give up the approach we're taking?! Give ourselves a break, come at it from a new angle with fresh determination. It is the start of a new twist in the tale, or even a new tale. But endings are beginnings (little beginnings, or more of a departure), too.
Zig and zag isn't all bad. We move to better on the lessons learned by those who went to the extremes and learned what the real tradeoffs are!
2/29/16: This is a nice reflection on our binary (this or that) zeal, and the way two things can be true but there are tradeoffs and the weight just falls out differently in different contexts (sometimes at the poles, and sometimes as an approach that is a hybrid that combines something of both):
Anyone who says "the content is outstanding" -- and thanks me and Dana/doesn't just assume it was Dana -- gets my full and immediate attention. :)
My day got even better! Thank you Andra!
Remind me to write a trace on boundaries. Oh, not those boundaries. :) System boundaries. Boundaries within our system.
In the meantime, you can acquaint (or refresh) yourself with this post (on the Bredemeyer Requisite Variety blog... which I really should kick back into gear again... maybe?): Mastery and Architecting
Steve Merrick reminded me of this passage:
Good call to go back to a classic (again!), and also to pull together and elaborate how I think about integrity in the system/software architecture/design space.
I listened to and highly recommend this: Episode 102 | Michael Feathers - Providing Options
The whole thing is full of insights and images worth stealing/leveraging, but to give you a sense, I'll focus on the last segment:
Here are the EA executive reports Cutter published (you could think long paper, but if you think short ebook you'll be happier -- especially when you can get it for just the price of your contact info, not your credit card and contact info -- winking smile)
Some Dangling Threads
Complexity and Irreversibility
3/4/16: Metaphors and analogies are not identities, and we use them expecting people not to stretch the point:
Of course, Alan Watts then goes on to stretch the point. ;) just kidding! But I would argue, we shouldn't make words take all responsibility for their meaning. Nor make metaphors take all responsibility for our understanding and use of them. The thing is, we want words, metaphors, things to mean stuff. So we play an active role in trying to understand, but we also have lots of baggage. History. Context. All conspiring to help and hinder us as we try to bring a larger narrative to bear, to make a strung-together meaning.
So if our narrative is "we do software all wrong," then "construction," as a handle that's been used for producing(/where in the process we produce) code, becomes a ready target to peg responsibility on. Behind that handle, is a whole field, and it is simply, to put it bluntly, insulting to charge a metaphor with how we view and build software. Smart people weren't led by the nose to do things some particular way, by a metaphor! What we do, how we design-code, is a product of so much history, accident, and hard thinking -- a lot quite far from any "construction" tag.
So. I'm constructing this argument. Of course, I'm doing so in my usual iterative, messy, "feeling for how to better express the intuition that is nagging at me" sort of way. And it's still messy and would need considerably work before it could more reliably support safe passage to understanding. I didn't use a pick, though I did try to pick my way through the fallacies, bulldozing the more unhelpful thought detritus... Hm...
Back to construction in software: the word reshapes itself to mean our evolving sense of how we create software. What we do, impacts what we think construction means, when we see it as a label on process diagram, for example. It may not be the best word to use, and may even inhibit some understandings, but we are active in meaning making, and our objections may get more in the way, than the word/metaphor itself! We can obstruct understanding simply by putting objections in the way of understanding. :)
Let me come at it from a different angle. Where we continue to use PDLCs (product development lifecycles) that have waterfall flavors to them, we need to look at the co-evolution of practices and development infrastructure in that space. Recall:
So where we work in systems(of systems), and we're not just integrating across hardware and software and firmware, but manufacturing lines that have to be co-designed, and supply-chains too... it's just a tad (wink) more complex than getting feature toggles down to a craft form. That is, software is developed in many different contexts, from a bit of ad hoc code one-offed for an experimental analysis, to firmware and software in a product line of manufacturing robots. With what you do somewhere in between ... or... along another dimension, in a multivariate map. :) Anyway, I really have a hard time giving the metaphor "construction" all responsibility, nor even particularly much, for how software is developed. We simply don't go round saying "oh gee, now what does the metaphor tell me to do?" "Oh, it indicates I need a bill of materials. Why, goodness me, I should create one." "Oh my, I need an excavator here." Well, actually. I think I have Mike Mulliganed myself into a big hole. ;)
In short, we need to allow that the meanings of words evolve. We want our process to be agile, to enable our business to be agile? Or at least, responsive and effective at sustaining itself, not just through the moment of now, but the longer now as the present threads itself into the future. Well, then, we might just think that we, the people of software and neighboring fields, are somewhat agile too? Somewhat flexible, and mutable in our understandings, not just in what we shape and give form to in the world?
Yeah, so, we can -- should even -- adopt better, more useful metaphors. Words that draw us towards intended meanings, rather than words that trip us up. I'm down with that (in Buffy/1990's speak, because the teens have me working out to Buffy this winter). But my hackles rise when we treat people like they are idiots we have to look after with better words because they are too clueless to adapt themselves to new understandings. There is give and take. And we naturally tend to gravitate towards new words that are more useful. I think we need to gravitate towards using design instead of requirements, but time will tell. We can take a page out of Lackoff's book, and speed evolution along by picking metaphors that help us progress our understandings and approach. But let's motivate doing so on what a new metaphor offers, rather than asserting that our understanding and approach to software is trapped by a word! It quite demonstrably has not been! Witness successful CD/CI projects.
Speaking of co-evolution. Did you see Mr. Turner? Neat point about art and technology.
Mainly, like this Trace, if you look past all that's wrong with it, there's a lot that's right with it. We progress (often with air quotes -- often seen only in the kaleidescope of history). Our words shapeshift. Morph. Take on new meanings. Or we flat out jump to new ones. I'm fine with drawing attention to moving on from construction as a metaphor to something better.
I just think we misplace attention, if we try to claim it is responsible for too much. Likewise with technical debt -- it is not responsible for projects that take on wanton debt. Something else is going on there. Words are powerful. But they are not all that is powerful. Yes, words frame and draw. They are what we use, besides phyiscal stuff, and images, to get ideas across. To signal. Convey. Reproduce ideas into the minds of others. But a word or phrase does so much. How much, depends on a lot of other things. Context. History. Repeating yourself.
What would I suggest? Start to use new metaphors and images. Look for a good (enough) fit, to mobilize understanding and action in a (better) direction.
I also write at:
- Bredemeyer Resources for Architects
- Software Architecture Workshop, Boston, MA, June 27-30, 2016
- Enterprise Architexture Workshop, Chicago/Schaumburg, IL, July 18-21, 2016