A Trace in the Sand
by Ruth Malan
This Trace is just where I "work out loud," jotting notes as I explore in and around the territories I associate with architects architecting architecture. Since I work with architects across the spectrum from applications to enterprises, the scope of my exploration flexes accordingly. For a playful orientation to this Trace, you can read the characterization I assembled here, or selection of posts I assembled here, or visit my in-progress Journal Map.
The work I've been doing on business ecosystems and characterizing change and the implications for business agility, and fractal and emergent strategy and tandem architecture, will be tossed over the wall for review in the next week or so. Then I can shift my focus to system design/development of the next big thing. :-)
Image: By Sara B.
This got a fair bit of play in the tweet stream today:
This struck me:
I agree. And I live that principle. On the Bredemeyer site, in this journal, in the papers I write, I have done probably more than anyone else in this regard in our field. I am constantly referencing and frequently applauding other people's work.
But I would like to point out that it is not just about building networks of knowledge. There is also the matter of professional standing. We invest our time. And if we get nothing back from that investment, it is harder to keep doing it.
Will we see ecosystems fail to thrive because people take without thought to fostering what they value?
It's not just the music industry (link via Grady Booch) that is still in the tumult of a watershed transition from consolidated power in corporations that controlled gateways to value flows to a more democratized ecosystem with diffuse access and highly distributed, individualized value flows. There is still a need to create healthy livelihoods at the base of all the ecosystem transfers of value.
Quote-as-image source: 2011 Shift Index, Deloitte
6/2/12: Attention and trust are scarce resources in this age of info glut and digital distraction, and respected advocates who evidence enthusiasm can draw attention to excellence that would otherwise be overlooked.
6/5/12: More on the threat of not passing through value to sustainability in internet ecosystems -- through the lens of the music industry as the pre-eminent guinea pig of our era:
7/1/12: Maria highlighted a wonderful and pertinent piece from Toffler's Powershift:
More "bread crumbs on the empathy trail":
Commenting on a biography she read about Laura Ingalls Wilder, my daughter remarked that Ingalls Wilder's life was so much more boring when someone else was telling her story. That brought to the foreground for me how much of our life as we experience it is internal -- our thought or mindlife. And it is only by embodying, speaking, drawing, writing out our thoughts that others gain access to our mindlife. Of course, a biographer, chronicling the events of Ingalls Wilder's life had more access to her imagination, her interpretations, her emotional, spiritual, emotive and thoughtful encounters than is generally true, given Ingall's Wilder's delightful autobiographical novels. But who could compare in the telling to Ingalls Wilder herself?! At any rate, I think fiction is so important in the development of empathy because it is able to take us into the thoughtscape of others, imaginary or real, and give us the chance to feel their experience, see through their eyes, and practice that. Sure, we can do this through our own imagination, but reading takes us into encounters with other's thoughts -- the imaginative casts of other minds -- which develops a greater span of perspectival flexibility in us. Our imagination is given all the more grist when we immerse it in the imagination and vividly recounted experiences of remarkable others.
6/8/12: The language of emotion, David Kozatch
6/9/12: Five Keys To Building A Business That Doesn’t Bury The Humans At Its Core, Tim Leberecht, June 8, 2012
6/26/12: Kids ask amazing questions. Sara asked how deaf people experience thought. Our experience of our conceptualization is so full of "heard" -- in our "mind's ear" -- words. We talk about "the voices" in our heads. In all my years, I don't believe I'd thought, or heard anyone else ask, that! I can't believe it, for after it is asked, it is so obvious a question to ponder. And yet we know thought is sometimes visual. And visceral. Still. How do apes multiply? Numbers, I mean!
I enjoyed Tom Graves post:
All that advanced probability theory I did is seemingly more relevant. ;-) We have emerged from the age of naivete, from the "childhood" of mankind, I think. At any rate, we're pummeled with realizations of our ineptitude, having messed up everything from the climate, to the global economy and... goodness gracious, even created a junk yard in space that threatens the communications blanket we've come so quickly to depend on! Add to that all we're learning about our cognitive biases and the ability of our "lizard brain" to hijack the more recent cognitive layers and we're being disabused of our sense of rationality (even if we saw it as bounded) and command of our world. Not to mention The Edge 2011 answers to What Scientific Concept Would Improve Everybody's Cognitive Toolkit? Here is the fast read version:
Ah but! Just in time, we have the likes of Watson and the Google self-driving car to the rescue, giving us augmented capabilities and ever more replacing us with less error-prone integration control points of complex systems of systems. Freeing us to revel in augmented reality that feeds our brains with the quick fix of digital stimulation... Uh. I mean, freeing us to weave knowledge in ever more creative ways, extending the frontiers of what we are capable of.
And this, from The Edge 2012 edition:
puts things in perspective again. Phew.
That said, it's good we have some things we're certain about:
Those plans you were making 4 billion years out. Not gonna happen. Sorry.
First we have to deal with ourselves in the 21st century. Pandora's box comes to mind:
ps. I Freudian-slipped my response to Tom, saying "flexible variety" instead of "requisite variety." Noticing my error, I deleted the tweet to correct it. Then, in the process, noticed that flexible variety was actually a neat twist. When I write up why, we'll have the Ashby-Malan law. ;-)
6/5/12: Related: Complexity and Philosophy, Giorgio Bertini 11/11/2010
6/8/12: Not to mention the junk yard in the ocean. Goodness me. The combination of survival and little self-indulgences in modern life do mount and mount. To such devastation.
Sustainability comes at a price. But it is an investment in a viable future. If we don't pay as we go, whether it is creating technical debt or a disengaged workforce or trashed and disabled planet, our older selves, our children, their children will live with waves of pain as the the environment convulses and sorts out a new order. Depressing. But Peter Bakker's droodle comes to mind. We stuck our head so far in the sand it's come out the other side and we look at the world we've created in staggered surprise.
Image source: I can't find the Dan Willis uxcrank sketch that inspired my sketchnoted version, but this image is cool: life as visual illusion. Looking for the original uxcrank image, I stumbled on this great visualization of "imagineering" (my framing) by Dave Gray.
6/18/12: On complexity (in software systems):
My, my. Remember this:
Well, we can help. We're doing that bridge to great thing July 9-12 in Chicago. :-)
6/2/12: HP has strayed so far from its roots -- the roots whence we came, but which the quick succession of hapless CEO's all worked to rip out. Let's get HP and the economy going in the right -- jobs growth, not 27k retrenchment -- direction! We can do innovation leadership. Agility.
This is an awesome presentation: Mission-Critical Agility - Dr. Jeffrey Norris, Oredev210 (via Ernest Beauchamp) Great use of stories to convey key insights. I'll be using the door visualization (with attribution, of course) around minute 38 as it is a great way to make a key point central to Visual Architecting!
The phrase "going Dutch" came up, and on explaining that it means paying one's own way, quick as a flash Ryan said "In that case, I'll go Greek." Very funny. With a dark side.
6/9/12: I should add that I do realize that it's a cheap joke and various nations have handled civic responsibility and dignity differently, creating models we can learn from.
Pinterest is now showing who follows your boards. And with this new transparency, suddenly the follower count on my Business Ecosystems board dropped from 12 to 5. Was Pinterest lying about the count (or if "lying" sounds too provocative, did Pinterest have a bug), or did people bail on following the board once who they are became visible? Hm.
Ah. Twas Pinterest. Bad Pinterest. If I was into conspiracy theories... ;-) But I'm not. I mean, we're fallible enough to take all the bugs squarely on our well-intentioned shoulders. As bugs go, that's some rather muddy algorithmic thinking...
Well, at least they were upfront in explaining. There's system integrity and there's organizational integrity and it is pretty hard to achieve system integrity without organizational integrity (at least at the dev team level, but often throughout).
You can see how little interest Pinterest has in the tech community... I had to go searching for what happened... Usually news like that comes to me on the tweet-stream...
Pinterest has a strong bias towards visual and organized. Is that why it appeals so to women? There women go, cleaning up the internet and putting it into neat visually indexed boxes. Ha. Gotcha. ;-) Sure, it is slanted towards topics that are inherently visual. But that doesn't make them inherently feminine.
Well, I do like Peter Bakker's Inspiration board (and promising "droodles" board). Jack Martin Leith's Models, concepts and methodologies Pinterest board was a neat visually indexed "cubby" of links to useful models -- so disappointed to be met by a 404 error on that...
A Rubrik's Cube Moment of a Post!
A Rubrik's Cube Moment? You know, an "ah ha" generator. This:
I'm so happy Jennifer Sertl pointed it out!
Uh. Right. Code.
I'm being told to turn off the distractions and just. write. code.
(Yes.... I have to finish the chapter.)
Ok. Ok. ;-)
This is important to read as a citizen of the world and as an architecture case study, that of the European Union:
Talk about the Age of Disillusionment in the face of our great ineptitude, or the failure of our models of the world. It is a time where we need great leaders, but in the US at any rate, there is such determination to thwart Obama that there's no good following, or not enough to avert disaster. Much then depends of Europe, and Merkel. What a time to be living in!
Luckily the brain is wired to be optimistic, right?
Consider this from the New York Times:
I would also like to add, with respect to "Without men, we would never know what our friends were doing five minutes ago", again there is a woman to include in our debt of gratitude -- Ruchi Sanghvi is one of the (3?) people responsible for Facebook Newsfeed.
What a way to begin an article that is about the tender issue that beleaguers our field, for it so well demonstrates the blind spot! I thought the opening must surely be making exactly the point that our field favors men even in our telling of our field's history, but as I read further there was no hint that the opening was intended to be any such head fake. Instead, the article's main thrust seemed to be to discredit Ellen Pao, more patently implying a "victim mentality" or "litigious bent" than investigating how a field can simultaneously be friendly to women and not. A field where we have problems like not hearing or seeing or valuing the contributions of women. The opening lines being a case in point.
Well, I wonder if Vint Cerf ever used a pocket protector? Or Bob Kahn. Two men. Either the credit roll is specific, and then clearly limited to a very specific contribution, or we include the broad swathe of people in the various paths of our field's history. And include, even, beyond the stereotypically "nerdy" people -- like Hedy Lamarr. Who, if we "know what our friends were doing five minutes ago," we would likely also need to draw into our credit net, for WIFI probably played a role in that connectivity. And so it goes. History, including of technology, cuts broadly across our social roots.
Our technology field has drawn women against the odds, and they have made contributions, some cornerstone (like Elizabeth Feinler), some less foundational yet still important.
6/5/12. This is cool: Women entrepreneurs on front page of USA Today. Putting women tech entrepreneurs on the front-page helps shift the daily messaging we all receive. I think there is a lot to draw women to technology careers. The gauntlet at entry is high (being one of just a few women in a scoffy classroom is tough), and the thick-skin requirement isn't just a barrier to entry. But the work is thrilling and there are so many great people, it is worth it!
6/9/12: Hidden in plain sight: Important Demographic in Tech, Alexis Madrigal, Jun 8 2012.
Rather than taking offense at diminution of others (aka moving on from the last post), I think it is far more productive to take inspiration from Cory Booker and live the positive experience we want to impress upon the world. Ah, Cory Booker. We can be so cynical that when a man like Cory Booker comes along, we look for the tell-tales signs of Machiavellian image-mongering. First, let me say that to even preface my point with chastising myself and you for a tendency to cynism when we encounter a great person in our times, it is a sign we are jaded and world-worn in the worst ways. But, since I opened that question, let me hasten to add this is a man who lives in peril to live his convictions. A man who has been so busy serving, that he still doesn't have a family to give him the joys of intimate love and mutual dependence. So you know this is an authentic man, fallible -- but less so than the rest of us; and great -- more so than the rest of us. I urge you to watch this speech. It will (ok, it can -- if you allow) teach you what no book on communication and presentations and leadership can teach, and only take 30 minutes. That's a great deal, if you ask me. So, watch this:
Then if you are inspired to a random act of kindness involving giving away money, Harmony School is raising scholarship funds. No child is turned away for lack of financial means. I'm told by a reliable source that the teachers took a 25% pay cut when the recession hit and more families couldn't afford the tuition. When your pay check is meager and you still take a 25% pay cut that is laid bare, honest evidence of the deep merit in what you're doing! This is an alternative education choice school that gives a place to kids who thrive when their indivduality is honored. The teachers, much like Cory Booker, lived their convictions by making a personal sacrifice. The school is called Harmony, which is fitting because it is a school that prepares kids to achieve harmony in themselves and in the world. Mission critical for the planet at this point, don't you think? Donations from within the US qualify for a 501(c) (3) tax credit, and from within Indiana also for state tax credit under the "Scholarships for Education Choice" program. There is a donation button on their site, but I can email you the donation form if you want to make a difference this way (preferably before June 15th). I realize there are already too many demands on your budget, but this is one of those authentically good things to do.
That is a vivid -- and I think very useful -- turn of phrase! And it sure highlights why Google is feeling heat in the "Hunger Games" of modern competition. In the early days of the information superhighway we first relied on phonebook-like indexes such as the early incarnation of Yahoo! to find stuff in the quickly growing info-mass. Then Google became the pre-eminent routing device and because it was the main mechanism for attention flow, it could make a lucrative business of pipelining in advertizing as it conducted our attention through the info-space. But as Facebook and Twitter started to siphon off more and more of our limited attentional bandwidth into affinity and trust networks as the discovery or "Serendipity" engine, Google's lock on the advertizing feed was broken.
Or something like that. Makes for a colorful story anyway. ;-)
I'm much more interested in reading Clayton Christensen's new book after reading this interview with him:
Here's an excerpt:
That is, I think, an important positioning statement for business agility -- agility isn't a indeterminate random walk* where the trajectory changes at each step depending on local context for that is taking a long route to nowhere in particular. Sure, innovation is an unfolding journey, but there is a greater purpose we look to as we make adjustments and even outright pivots to the course we take. Here is a key to being agile in that intentional and emergent way we advocate:
In our context mapping work, Dana Bredemeyer has called these "assertions" and I agree that it is important to be explicit about the assumptions we want to keep in view so we notice when the unfolding future deviates from our expectations. This is also true for deep internal-to-the system assumptions, and it is why we (must) document assumptions in the context of architectural decisions that depend on them holding.
6/6/12: This is a great framing (or, in this case, positioning statement) for strategy:
6/10/12: * Charlie Alfred's tweet has a re-useful (amusing) illustration of the random walk point, and a complementary point. Of course ad hoc agile isn't really so undirected -- or should we say, it is hard for agile to exist without strategy. Both agile and strategy are what people make them. And people are flawed as all get-out. And amazing. How amazing? This amazing! And much more. It is hard to be completely unintentional, to just be bumped along by feedback, not attempting to make sense and figure out where value and market and doability lies. Which is strategy. Of course, we have tried to build isolation tanks around waterfall development, which is failure prone because business ecosystems are being actively reshaped and disrupted. But the extremes illustrate and push thinking. And it is a nice juxtaposition. Demonstrating the need for some "opposable mind" thinking (in that Roger Martin sense). Like this:
But it isn't a guarantee. It might even be an encumbrance, having carved deeper ruts, become coupled into more messy entanglements of complexly nested dependencies and entrenchments attaching it rigidly to an old order in changing times. We see that longevity is decreasing, for example.
But size matters. Like this:
I think the interesting question goes more along the lines of: has how size matters changed? Also, what are the structural features (mechanisms and forces) that allow small to be able to compete in new ways that enable small to thrive even in the presence of large? And can small bring about the demise of large without becoming large, or is there just a rolling over and replacement of large? Like Borders is gone, but it has given way to Amazon, along with Barnes and Noble and Apple... Not exactly the demise of large, but a change in who controls passage and extraction of value from the ecosystem flows with still more consolidation under Amazon, squeezing out not just brick and mortar (remember that phrase? how quaint it seems now...) but also publishing houses... Now, if Amazon could just get Mechanical Turk to write the fiction we want... or... Hm. Ok, extrapolating a bit far there. Or are we? This stuff is crazy serious, so its more fun in this backwaters place to play a little with an idea so we don't get too ... despondent! ;-) I know, I know. Lots and lots of exciting developments in our tech-augmented capabilities and opportunities.
And changing market dynamics, at least in so far as social gives long tail consumers access to long tail producers who have access to long tail funding sources (including long tail customers who will pre-order product if it excites them... hey, did you see Collusion? Wow!)...
Anyhoo... just thought I'd explore that a bit... I liked where things were going with the Elasticity thread, but "there once was a time" seems a tad premature... We already see industries that were locked tight through the advantages of size (learning curve effects, barriers to entry, etc.) have been rendered penetrable, and even disrupted. But it is one thing to have a PayPal and a Kickstarter be successful, and another thing to say that they mean big is done. For one thing, PayPal isn't small, and was absorbed into something still larger. The impetus to power and control of ecosystems is still there. Sure, this is a time of flux. And sure, big has to learn to be organic and responsive. The tension, perhaps, is more between the mechanistic structures that enabled us to create the large enterprises of the last century, and the more flat, more organic, less territorial (less power tree oriented), fluid, fast, dynamic, responsive, reflectiive, human-centered organizations we can create with the new ecosystem infrastructure of social technologies. This, I think, is more the direction Dave Gray is exploring, in Connected Company. Ultimately, perhaps we will see a return to more community centered ownership of much smaller production serving local consumption, as we're seeing with farm products, but again, I don't see that being exclusive. Some things are just big things to do!
Boy wisdom. I do admire it. As for me, I mow the lawn along the contours of the yard, not in straight rows. Girl wisdom. Same diffs. We just have to figure out what those forces are. Not so easy when it is a market in tremendous flux, upsetting the underlying dynamics in ever-changing ways... Or. Is that the case? Is there a pattern? Are there so many interactions...
6/13/12: This is interesting:
Of course, start-ups have a different vestment and work ethic, not just shorter decision chains. But that is not generally sustainable year after year, as people want to "get lives" and spend time with families and such. As is this:
6/20/12: In the beginning:
Growth. Inexorable attraction. Opportunities and dragons.
How do you like them apples?
When the families of the Watsons and Siris of this world grow up, we're hosed!
Ok. We'll still have some advantages. I suppose. Won't we?
Oh, I'm teasing. We are headed for planetary abundance and Watson and Siri and the like will help us get there. If we put our heads together with theirs. We're creating capability here that will change the possible! It is very exciting.
The thing that concerns me is the mean, the evil. The petty. The scared. Because we could wind ourselves so tight around selfishness and greed and defensiveness that we set more pain loose in this world than we can imagine.
6/7/12: Also interesting:
As things augmented reality go:
Think we need to read this: James Beniger's The Control Revolution (1989)?
These are interesting (food for thought and useful stimulus);
What's in it for you? And why should you tell your peers and even your manager to come? Well, goodness, look at Kodak. RIM. The list goes on and on. Scare tactics? Wave after wave of digital transformation is reshaping entire industries. That's either scary or hugely, invigoratingly exciting because there's so much opportunity! Still, these are times that demand great leadership and it is strategic-technical leadership -- not simply eking out past successes to get by for a while, but boldly building the future out.
So. More about the workshop:
The workshop follows the Visual Architecting Process because we find that that is a good way to practice the thinking-modeling-iterating-collaborating of developing a (draft) architecture. Which is to say, it is very much a doing workshop, with brief “lecture” sessions followed by working sessions in small teams, debrief of the work done/lessons learned/more lessons and experiences shared; rinse repeat.
The Visual Architecting Process is pictured here and here. We regard process as “scaffolding” (it supports us and enables us to build more complex systems, but the effort and focus is on building the system) and you use just as much as you need to fit the design and organizational challenges of your system/organization and the maturity stage of your system. We talk about tailoring the process at the end. In other words, it is a “just enough” highly iterative and agile process. But mainly the process provides us the contextual grounding to make other points about architecture, so there are sections on Conceptual (abstractions, coupling and cohesion, design for evolution kinds of concerns), Logical (interfaces, etc) and Execution/Physical (design for operations) Architecture, as well as architecture strategy/meta-architecture (and architectural patterns), and so forth. We take the orientation that the architect is responsible for right system built right, so we address both right system (fit to purpose and fit to context), and built right.
Visual Architecting emphasizes:
The short form of which would be to quote an architect from a recent workshop: "was fun and rocked."
What's in it for me? Well, getting to work with you of course. And the wiffm for you when you pass on the word? Your peers learn the "lead, follow, or get out of the way" principle. That's success-shaping. You'll thank me/Dana. ;-)
Oh. Just in case this surprised you: Why tell your manager to come? It's strategic and it's organizational. So many think architecture happens between a mind and the code it creates and evolves, but so much of what is really important about architecture has to happen between minds. Alternately put, architecture as a self-aware discipline becomes important when the system must be built with and through people, often many people. So we're figuring out how management needs to change in a more agile, flat and less command-and-control world, as well as how the role of architect and tech leads change, and what that means for "architect as just one hat" versus "architect as role" and so forth, and all these questions are best addressed when more of the team is exposed to the issues, challenges, concerns and how they relate to desired outcomes.
Just one more thing. ;-) Another of the principles Dana gave name to in how we talk about architecting is "a commitment to objectivity." Figuratively, it looks like this and like this:
You'd like our workshops. Don't you think?
Oh, oh, oh. Another other thing. In the workshop we consider what capabilities the architect needs to hone (in her or himself, in the team, and in the system) through deliberate practice, and start to do that. And we consider what deliverate discovery entails, at various points of system conceptualization, development and evolution.
In all seriousness, we cover a lot of ground, but in a fun format.
Well, some may have that "meh, that's not so special" reaction. There's the zen parable of a cup too full. Oh, a cup full of experience and knowledge and ideas gives us a lot to work with. But a cup too full of hubris and know-better, too full of entrenched orientations, doesn't leave room for the sheer humbling experience of all the frontiers we have yet to push in ourselves and in the world of the possible, the world we make real through software and ideas -- most often, simply novel combinations of ideas put to new uses, with some leaps of inspired inventiveness in the light of new perspective, perhaps. On occassion, someone comes with fixed ideas about what will be useful to them and cannot open themselves to the lesson in The Useless. But mostly we all just pitch our learning into that crucible the workshop forms, and make something great happen.
I read the introduction and look forward to reading the book -- excused, of course, by John Gall (author of Systemantics) admonishing us to take up biology as a hobby and study it for clues. Reading that introduction (and the part about the cat), situations like this came to mind.
Tom Gilb's JavaZone2011 talk -- Real Architecture: Engineering? or Pompous Bullshit? -- got a nice shout-out by "Uncle Bob" Martin.
Tom has been making the case for characterizing requirements -- system properties or qualities or characteristics or non-functionals -- and showing us how to do so for years and he has a rich and informative site, not to mention a number of books that pioneered and shaped various avenues of the software field. Tom and Kai Gilb's site is a great resource.
Most BI toolsets focus on queryng and reporting, dashboards and alerts, and predictive analytics -- or operational health monitoring and forecasting -- kinds of areas (and the data gathering or extraction and analysis to that end). And EA focused on IT infrastructure and resolving untoward proliferation in the technology landscape. In each case, we've need to widen the lens, to better serve the business. So we want to drive our questioning in (at least) two directions here: what forms of intelligence serve our business? and what does compute/digital/information technology present in terms of intelligence-focused opportunities to enhance our ability to execute or create new strategic opportunities for our business?
So I think of business intelligence (BI) as being made up of:
(See our Fractal and Emergent paper.) Or looking inward, outward, and to foundational capabilities. Each of these has a tactical and a strategic dimension. And looking back, and monitoring and sensing in the moment, to provide continuous feedback and enable "pull" responsiveness to the market, and to anticipate and make projections about the future. System "health monitoring" for "preventative health" and continuous improvement -- to eliminate waste and quickly spot errors and to make course corrections in the near term. Pattern finding and simulating to better understand the operant mechanisms and forces, and to make predictions for the longer term. Sensing, learning, responding and anticipating.
Yes, I'm casting a wide net there. BI efforts may be cast in terms of intelligence that can be gained from data warehousing, which is saying what value can we tap from the data we hold? But what if we turn that around, and ask how we want intelligence to serve our business? Then we want the lens of our "intelligence" to be wide enough to know not just how well we're doing, and to build in the ability to anticipate and shape, to lead smartly as we contribute to building out the future through our actions in the present, shaped by what we want to make real in the world. The identity we wish to have, what we want to be known for, what we want to make possible. Whether our focus is on the next quarterly statement, or some bigger future. Or some balance of excellence in execution to short term goals, and excellence in building the next wave of success.
Several years ago, I played with this image. The point being that we want to apply "intelligence" to develop "wisdom" -- to improve our judgment, informing our actions with a rich understanding of context. Reiterating, context has these dimensions:
While wisdom, we might say, involves penetrating hindsight, and highly developed insight and foresight. Wisdom, then is not just born of experience, but of actively making sense of experience, learning, reasoning about, making connections, creating and evaluating or testing heuristics and models that help understand and predict, etc. So that wise -- systemically appropriate and timely -- responses and determinations and goals are made or set.
More traditionally scoped BI functions focusing on analytics, powering dashboards and alerts, doing risk profiling and predictive production forecasting, etc.. are already dealing with the stuff of rather big data -- integrating, filtering, working with large amounts of data from a variety of sources. If we treat "Business Intelligence" more along the lines of intelligence gathering to understand the threat and opportunity landscape, to uncover risk, to respond more finely, targetedly to opportunity, to spot emerging trends and strategize, we look to, but also beyond, the traditional transaction data generated in the conduct of our various business processes. And that takes us into the territory of "Big Data" -- the stuff of tracking and monitoring and collecting huge volumes of data (from the internet-of-things of connected sensing devices, RFID tracking through the supply chain, ... to tracking user actions and the content they generate online). In addition to volume, there is variety (not just of sources, but forms -- structured and un, quantitative and qualitative, subjective and emotion laden, interpretation challenging, etc.) and velocity (meaning fast changing so there is a critical time element to capture and analysis, which further compounds the volume of data) to contend with. Not to mention the challenges of using this data to inform strategy, or more directly to actuate and control systems (of eco-socio-technical systems). In a world of change, uncertainty, complexity, ambiguity, ... we are seeking to discern, to inform, to understand, to make sense, to resolve, to act, to shape, to get more the outcomes we want (and less of those we want to avoid, that threaten, harm, impede, ...) ...
We might also like to think of intelligence in terms of opportunities along these trajectories:
That is, we tend to think of business intelligence in terms of sensing and sense-making to improve judgement -- to determine actions to take, enhance the outcomes of actions and avoid risks or reduce the consequences of environmental threats.
But we can also think of business intelligence as enhancing knowledge and capability, so that we can tackle more, advance frontiers, address or harness complexity, etc. For example, through augmented intelligence, or transferring capabilities to machines -- especially those equipped to learn, or adjust responses based on data acquisition and analysis, hypothesis forming and testing -- we can compensate for the human tendency to limited attentional capacity, cognitive biases and other human error to handle more complex tasks. For instance, handing over more and more of the matter of piloting an aircraft to compute intelligence and control systems. Likewise in trading systems. And we see IBM Watson's offshoots moving into medical diagnosis, starting out as an assistant, but who knows what the career path of a Watson' will be?
Relevant? Well, what about sentiment analysis? Shouldn't natural language processing tools for discovering sentiment trends in your market be part of your BI toolset? To what end? Predictive demand forecasting? Discovering new product opportunities? Market threats? Ok, so if we include sentiment analysis, what other opportunities present themselves now that we've opened our intelligence box up to natural language processing, and to data mining in internal social networks or across the available information spaces of social and formal media? How do we leverage people's drive to share information, to check in, to share images, thoughts, experiences, assessments and reactions? Even, often, allowing their devices to act as more granular sensors, providing geolocation. Not to mention the increase, albeit slower than anticipated, adoption of RFID and ubiquitous sensors.
Now, Your Turn
I'm interested to hear from you -- what conceptual models or maps are you using? How do think about BI and related tooling, the relationships among various BI areas, and coverage you supply to your business leadership? What do you think falls under the BI umbrella? What else should I be thinking about, in this area? Is this useful? What am I missing, or misunderstanding?
Also interesting and relevant to this discussion:
6/18/12: Not sure who tweeted the heads up to this, but it slots right in:
6/18/12: The other day I wrote: Sensing, learning, responding and anticipating. And today I was interested to see the same essential framework in the diagram in John Allspaw's wonderful post:
The diagram draws on Hollnagel et al.'s model of the "four cornerstones of resilience." It is indeed a wonderful, useful characterization of the organizational intelligence we're building up through the technology capabilities of BI (focused on operations):
Source: Resilience Engineering in Practice: A Guidebook By Erik Hollnagel, Jean Pariès, David D. Woods, John Wreathall
Knowing, for example, "what to look for" is built on previous cycles of learning -- built on cycles of anticipating and generating hypotheses or mental models of how things work (their expected operating model and characteristics, an understanding of the mechanisms and their behavior given contexts, etc.) and monitoring and observing what happens given actions (responding) and the relationship with what was expected, etc.
Of course, if we're talking about intelligence, and learning, models like Agyris's double loop learning come to mind.
An aside on my use of sensing rather than simply monitorig: When I say sensing, I mean to include measurement and monitoring. Essentially, when we're not just looking at operational resilience, but also at direction setting (innovation, business agility and strategy setting), our lens is wider than monitoring (placing sensors -- physical or digital, and taking readings, tracking, noting trends and anomalies, and alerting) and also includes scanning. In that case, more imaginatively anticipating, and broadly scanning (here I'm distinguishing, for example, broad sweep scans where we don't know what we're looking for a priori from metrics gathering). So of course our sources differ across the more operational and tactical lenses versus more strategic lenses scanning for opportunities and threats more broadly and qualitatively.
As intelligence goes, it is interesting to look at the CIA as an analogy since they have a considerable amount of learning under their belt. So this observation (and associated diagram) struck me:
That too is a wonderful paper! I was jotting notes earlier about the role of augmented intelligence in overcoming cognitive biases and human error, so it is well-timed. Interesting that, like John Gall, Rajiv Desai is a medical doctor. And interesting that John Allspaw and Rajiv Desai reference James Reason, who wrote a book of the same title -- Human Error (1990). This is more recent (but not available on Kindle): The Human Contribution, James Reason, 2008. Digging around a bit on James Reason, I saw this on James Reason's elements of safety culture, which identifies a just culture -- which also harks back to John Allspaw's post and the culture at Etsy. Neat to see the "founding father" influence there.
6/18/12: Of course, if you're running a BI program, chances are good you're going to be asked how you show the value of your program. Or its architecture. Have you read this (and does it help?):
6/18/12: Facebook is buying Face.com. What Facebook knows, is seeking to know, and why (to target push-selling and ad stalking) is getting... uh, well, what do you think:
On the other hand, it's yet another avenue where digital tech is getting smarter -- almost as good at guessing age from photos as humans!
And then there's Narrative Science and:
Watson's off-shoot doing diagnosis. And robot writers of news. Augmented may seem a bit euphemistic in a few years, no?
More links here:
This is interesting:
As is this:
6/20/12: As we learn more about ourselves through behavioral, cognitive and neuroscience (and the meeting thereof), through social science (having a hey day on big data), and so forth, we open Pandora's box still wider. And if we are to see Hope rather than Denial (the leading shadow of Despair) at the bottom, we need to think about ethics and the meaning of all this, and imagine how we want the world to be, so we do more of that.
Anyway, this is another thought provoking companion piece to Jonathan Harris's "medicine man" -- or woman, Jonathan -- essay:
6/21/12: Interesting: “Precision diagnostics will lead to better outcomes” -- positions IBM Watson's diagnostician future and "Here’s where big data meets life science meets healthcare: Lab tests + big data integration + medical evidence = clinical recommendation. That’s evidence-based medicine." -- Jim Golden, Cancer, Data and the Fallacy of the $1000 Genome
6/22/12: When you have the data, it's not just about understanding and being more responsive to and better serving your (current) market. It's also about creating new business. Using data as ecosystem firmament (a carrier or host-bed for an ecosystem, or part thereof -- like a pond, or desert. Creating the space within which species adapt or speciate to thrive, in their own way, in niches they find and create and husband or enfertile*). Like Google and Google Maps Co-ordinate. (Still... Co-ordinate? The euphemism of the year? Why is Google stirring up Big Brother angst -- to inure us? Is this like a vaccine against caring about what's coming? Ok. I tease. And yet...)
* Ah, yes. Indeed. I indulged in a little neologism there. I wanted the word that means make more hospitable to life, and couldn't remember it. So made one up. ;-)
6/25/12: Trying to find the word that is playing hookey in my memory, I came across this:
How in the name of all that is scientific, could that have anything to do with architecture? I dunno. Blame John Gall. If blame you must. Inspiring man. But if you are interested in sustainability -- of any kind -- the "would Nature approve?" question is a HUGE bonus. Put it on your checklist! Nature? Yes! By analogy. And directly. Am I nuts? Probably. But what I mean is, is there precedent for the mechanism in nature. Because if Nature already designed it, Nature has already taken its agile path to (life)test-evolving it. Just... agility in Nature's timeframes is different than ours. Nature is the great systems thinker. [Of course, when I write Nature, I allow that I, and you, may place God, emergent or intentional or both, in the capitalized hallowing.]
Well, anyway, I'm looking for how to say organizations that actively enrich the ecosystem, foster flows of information and value -- including by increasing trust which lowers barriers and lubricates transmission of empathy and information/communication and ignites innovation yadda yadda -- serve not only to increase what they reap, but what is available to nourish other organizational organisms hosted on and colonizing the ecosystem. The whole system thrives until a wave of change alters the structure and relationships of the space, giving way to a new order. And so it grows.
6/26/12: Predictive analytics and using the patterns found in "big data" to steer and tailor what is presented to customers is relatively new, and so still a learning ground. Just as "ads that stalk" one from site to site are viewed by many as sordid, there can be a queasy reaction to using personal data (such as user activity) to "bias" what a user is shown online. So this is part of the territory:
Advertizing has long been accepted, and clearly advertizing has long sought to, and benefitted from honed manipulation of emotional state and content of mind. Business is now using more about us as individuals, and the groups we fall into, to further target that manipulation. We stand to gain, with better -- that is more targeted -- navigation of the info-glut and options space. That is how these devices are motivated. It makes alternative mechanisms like personal word-of-blog/tweet/etc referrals all the more important, for these agents of Serendipity can side step algorithmic categorization and demographic bucketing. With the combination of tuning of results and "answers" to our queries, and more options for Serendipity, we'll be ok. Right? ;-)
7/3/12: Useful post (and pointers):
7/19/12: Event Processing in the Enterprise - a Waypoint on the Path to the Warehouse or the Launchpad for new Analytics Solutions (Industry Keynote) Balan Sethu Raman
When we're documenting architecture decisions, we need to sufficiently describe the context within which the decision makes sense. What constraints were we taking into account? Explicitly? But what about implicitly? And if we don't make those implicit constraints explicit, does the decision still make sense when considered from another point of view (that of another stakeholder -- for example, another team that will be impacted by the architecture)? Yes, yes, yes -- we want to keep to the discipline of just enough. Where enough, I'm afraid, is a judgment call. So you're going to take into account who will be impacted, organizational complexity and barriers to fluid, organic communication, and so on.
This flicked by via the tweet-stream today, I laughed out loud at this point, it was so charmingly un-self-aware:
Let's see. Arrogant means "having or revealing an exaggerated sense of one's own importance or abilities."
Actually, Nokia was good at product families -- at platforms in the product family sense. Very good. Not so good at seeing what was coming in terms of the smartphone ecosystem though (it saw the future, but not with enough urgency and conviction to get on it and make it happen -- at the expense of the existing product set). Elop saw this:
Well, of course the jury is still out, and Nokia is not dead yet! Not by a long shot. The point though, is that product family platforms can take a company down the path of optimization for niche refinement and value capture, blinding it to the entry of a new capability set that can send a disruptive wave across the ecosystem, introducing a new competitive order, rending the prior relationship bases obsolete, and introducing new ones. What Apple did was not just disrupt the mobile phone device market, but introduce something more powerful -- an ecosystem or relationship platform. Some have called this a software platform, and though software is critical, there are other elements that are important too, like new IP arrangements and expectations as occured in the music industry. Apple wasn't the first to do this. Annabelle Gawer and Michael Cusumano cover the strategies of Intel, Microsoft and Cisco, among others, in creating Platform Leadership. Demonstrating that the rolling waves of disruptive innovation can undo even dominant and fairly deeply embedded ecosystem platforms.
More background on Nokia here:
Happy Fathers' Day!
This photo (right), which I took some years ago, is my favorite visualization of fatherhood in my family! To all fathers reading this, and to all who love fathers, I hope your day is that fun and happy!
Children are a responsibility and serious work, but they are also an excuse and a provocation to have way fun. Have it! Turn something upsidedown (figuratively or literally) and relish fun!
There is ever so much I am grateful to Dana for. Fun and laughter high among them! Generous caring, thoughtful partnering, wisdom, empathy and kindness, much more. But FUN! All-consuming fun! That is a signal achievement I want to honor today! It is too little appreciated and venerated, and fun in our family owes ever so much to the Dad in our lives. He enables fun. Has fun. Is fun.
Fun takes a curious combination of work and spontaneity. An attitude and an opennness. Moments seized and moments made.
Here is an informal map of the Visual Architecting Process (skematically showing threads of reasoning):
It occured to me that it would be useful to step back and just talk about the structure of the map:
It looks like I'll have to clean that up and make it readable. Well I would, if I had reason to suppose that would be useful (given that, surprisingly I grant, I can read it).
But if you're interested in that, you might be interested in:
Blog Post on VAP
I added a post on Visual Architecting to my blog -- it leverages from an earlier post in this month's Trace so I only mention it in case you prefer to follow the blog where the posts are distinctly fewer and hold a bit more of a "finished thought". Also, it has the blog paraphenalia supporting "conversations."
Put these together, and you have an opportunity:
I really like that "stealing sheep" analogy -- it, for example, explains how FedEx undid the US Postal Service, seizing lucrative elements and specifically not going after the whole postal customer space. And how Zappos got to grab a chunk of online retail in the age of Amazon (sure, then Amazon made the Zappos Purchase and brought it within the Empire).
Of course, Amazon is also stealing sheep right from under IT's nose... Those who are reassuring themselves that all is well in their pastures, might use this piece of news to abet their case:
But SoMoClo is colonizing, and Amazon is a formidable host, floating in sheep stealers in the guise of employees dressed in BYOD extensions they've happily hooked up to their brains, extending their memories, their currency and access to answers and even questions, generated by powerful Serendipity engines that hold also the promise of innovation through novel connections (albeit with detours into distraction), ..., their very identities... and life management systems...
But... then... will IT wake up one day to find that Amazon, and the SoMoClo colonies it plays host to, stole the emergent organizational brain right from under IT's nose... In plain English: the social players ride off with the business analytics opportunity. Of course I could have used Facebook. But Amazon has such a nice biological ecosystem analog -- of the same name -- to play with. :-)
I smell an Animal Farm in this post. ;-)
Teasing Out the Metaphor Thread
As things predatory go, this is interesting:
Which gets us to mutualism and:
Bacteria, ant colonies, social networks, and the emergent global brain.
Connections. Associations. Analogies. Leverage.
Now isn't this an interesting image (via Maria Popova) -- using the imagery of, and so vividly characterizing, the age of automation (atop of the age of mechanization). Wouldn't today's version be interesting? Social. And mutualism of human and compute, or augmented intelligence that goes both ways.
Networks from bacteria to brains to people to teams to organizations to ecosystems to.... the emergent global organism which has a collective intelligence drawing directly on our plugged in brains, and the knowledge we have worked to unfetter from the confines of a single mind...
And then there's:
On Changing Minds (by Changing their Content)
Source: Paying to be Nice: Consistency and Costly Prosocial Behavior, by Ayelet Gneezy, Alex Imas, Leif D. Nelson, Amber Brown, and Michael I. Norton (via Jennifer Sertl)
So, why not get someone else to argue the case for your architecture/decision? Oh. Right. I say that from the frame of reference that values integiryt, humility, commitment to objectivity (and recognizing fallibility) and participation. Among other things.
More than 140 chars
Anne-Marie Slaughter has added her powerful voice to raising awareness.
Her article (short book :-) -- but a defining topic of our times!) in The Atlantic is worth the read as it raises awareness (nudging change):
The stories Anne-Marie tells and points she makes, equally make the point that men don't have it all either -- the intimate co-dependence (in the best, most healthy sense) of loving family is something that both parents should be part of, and both should share the work of. Should, although the burden of guilt and embarrassment at the missed hairballs falls more -- generally -- to the working mom when in-laws visit. ;-)
But I think this is important too:
It is not that women aren't ambitious -- our ambitions aren't necessarily to scale corporate and institutional power stacks (especially in organizational cultures where alpha-dominance styles are favored, making for more fractious interactions). Many men and women aspire to make a difference in the world through other vehicles than traditional authoritative power. Anne-Marie's article on collaborative power is a great way-pointer! And women tend to be good at collaboration -- we have to be. For many of us, our physical stature and voice does not "speak" authority, even if our minds and actions readily determine our competence and contribution.
6/30/12: Here's a gracious level-headed response to "Science: It's a Girl Thing."
Consumers have power: Why People Are Gaining Power Over Organizations in the Age of IT, Irving Wladawsky-Berger, April 23, 2012
Consumers have no power: What Happened to Silicon Values?, Bill Davidow, June 25, 2012 (via @Grady_Booch)
Bill Davidow's article is arresting. Remember Jonathan Harris' point that Facebook asks "Where is the serotonin in this design?” in design reviews? Lock-in. Like Pincus and Zynga's "compulsion loops." And "Activity" and "follower count" on Twitter, subscribers on Youtube, etc. We're gamifying our selves, and social has our number!
We can be cynical. Or we can take the view that this serves us. That our social persona has avenues to information and affirmation that grow our faculties, enable and empower us. We're all self-interested. The question has more to do with how effectively we manage to rise above the petty and self-interested, to share, to collaborate and serve bigger social good. To create a more sustainable -- heck even viable -- habitat for this humanity we're growing within and among us.
It is remarkable, don't you think, how little people do that? What? Take the trouble to say something appreciative.
As for method:
Source: Intuition and Creativity: A Pas de Deux for Qualitative Researchers, Valerie J. Janesick
We've made a similar point (it is the heart of the "This Extraordinary Moment Principle"). And it is also echoed in Grady Booch's Professional Architect piece in his IEEE On Architecture column. That is, we learn the techniques and principles of our field, we deliberately practice them. And mastery is a combination of developed unconscious competence, the sensibility to know when we need to pay more close attention (a point we make with the story of the Master Butcher -- here and here), and artistry -- aesthetics and unique individual creation and expression of meaning.
Beyond system complexity, there's the matter of building systems with and through not just our fallible selves (embarrassing when we catch up with our past selves in the code we wrote 5 years ago), but other people. Great. Still human. So process as scaffolding, and method than isn't only about individual technique and principles comes into play, so that we can build more complex systems. On top of the abstraction strata of our tooling and platform. Important that the method (technique elements and process that weaves them into a flow) is simple and its presentation clear, given all that!
So. It made my day, week, month, year that someone noticed, and bothered to say so! :-)
HOW TO USE CONSCIOUS PURPOSE WITHOUT WRECKING EVERYTHING By John Gall
Once you've read it, I expect you'll understand both the content and the enthusiasm of my reaction. :-)
I tweeted on Tom's heads up. This revision is even more stunning than the version I read a few months ago. It begins:
Mistaken for Truth
Which reminds me of:
Aside: I hope you don't think, when I point back into this Trace's archive stack, I'm doing so to "self-promote"! It's just that there are points back there that add something to the conversation now, so I leverage them. If you don't like it, you could always tell me. Nah. My internal critics have a head start on any criticism you could come up with -- not that you're not most able, just further from my head. :-)
Stuart Boardman's New York as a Virtual Enterprise post got a refresh of attention thanks to being mentioned in Tom Graves "This" post. In his post, Stuart kindly opened with "Ruth Malan produced some more pearls of wisdom." It is rare in my experience to get that kind of graciously positive reference. Thanks!
And then there is camaraderie. I'd like to thank Mark Appleby and Arnon Rotem-Gal-Oz for retweeting or passing on a couple of my tweets -- it's nice when a tweet resonates with someone, and especially when it is a tweet about something so deserving of wider attention as John Gall's keynote/paper! (Indeed, if it hadn't met with some resonance, I'd fear that my calibration of your awe and utility sensors was way off!)
Such inclusion is important! Consider:
It's not just old media! It's new media. It's who we talk up, and what we read. These cycles propagate. So I want to thank the gentlemen who spun some up-cycles for me this week!
You might like to follow the above-mentioned gentlemen. And Tom Gilb -- one of the giants of our field who stands all the taller for his infectious enthusiasm and kindness.
And Brenda Michelson because she ranks up there with Michael Feathers for LOL-tweets and insight.
And Peter Bakker. Because I'm enjoying his sketches and visually expressed humor, but miss his pointers and blog posts. He has the ability to make one take notice of stuff in the peripheral vision that is important but hasn't drawn our attention.
Why don't I say all that on Twitter? Well, gracious me. That's more than 140 chars. ;-)
6/30/12: Thanks also to Rolf Götz for tweeting the John Gall paper onwards. John Gall wrote that paper for our field -- he is retired from the medical profession, but his book, Systemantics, is still a must-read classic in our field. The keynote script -- a paper, really -- is wonderful; insightful and thought-provoking and exquisitely written. We could do our field, ourselves, and John Gall more service to be excited by such a piece of work as that! Of course, as system theory goes, both medicine and software can do with more of John's understanding!
Do we stop being delighted because we become aware of our mortality? Or because, not being appraised with delight ourselves, we withhold it from others, becoming jaded by looking for the tarnish, the weakness, the not-so-great-ness of what we encounter? Is the indifference that stultifies not that of the universe, but of those who withhold joy at and affirmation of one another? And is the light we must supply not our own sorry little lattern held to light our self, but an emergent Light that arises from the goodness, the goodwilling and striving to shed light and love and joy, of the mass of creatures on this planet. And others? Then life is not meaningless, but instead it is joining a chorus of learning and affirmation that ever evolves an emergent Light or Goodness of a universal or metaversal ever Becoming Being. (Thanks to Dana for his part in prompting such realizations in me.)
Let me hasten to say, that is a lovely article! I'm just drawing attention to "girls aren't confident" meme for I think it is one that hurts us.
I wish we could get beyond telling girls and women we're not ambitious and not confident! We are simply biologically primed and socialized to exhibit our confidence differently than men do. See how confident I am? I have done no research beyond a lifetime of close observation of the world about me, and I make so bold a claim! But I am aware of this, and would want to do said research before I make the claim more broadly than in this external memory space for the ideas in my head. ;-)
Indeed, we may back away from confrontational posturing and argumentativeness, but this is not "lack of confidence" but simply seeking a more mutual ground, where common outcomes can be advanced more mutualistically. Most women avoid fractious confrontation, not because we don't stand our ground, but because we find other ways to work towards a viable solution than talking over others, occluding their perspective, until they give up. Most men do the same -- but we don't penalize them by telling them they are not confident or ambitious! We honor them for being humble and playing well with others.
Do I not write my mind with confidence? Do I not exemplify an ambition to make a difference to the experience and effectiveness of others in our field? Sure, I don't care to sound a megaphone of ME across the lands, because this is a plural world where it is a tremendous goodness that there are many approaches.
My only somewhat informed sense is that girls start to move away from STEM in their teens because it is an age where group norming plays a strong role. They move to where the girls are, because that is where there is a density of camaraderie that is extended to girls. And that, I think, is why GirlsLearningToCode is so exciting and important -- a density of camaraderie. Of social inclusion.
If girls, following the girls ahead of them, move away from STEM to stay with supportive peers at an age of great emotional, physical and social change, that creates a situation where they start to track away from STEM subjects which require much work to learn the foundations -- even laying down connections in the brain. They can recover that distance, but few think of encouraging them to do so, because there are so few women in STEM that the perception becomes self-fulfilling.
With a tweener in the family, I watch girls in the arts -- performing arts like musical theater, ballet and music; photography and movie making; manga and anime; writing and reading fiction, and so forth -- and they are so magnificently good at using social media and direct interactions to not just help each other problem solve and break barriers of capability, but also they form a beautiful support system for each other in a time of momentous change. These are supremely confident girls. Doing high school math in elementary school! But the place where they coalesce for mixed gender camaraderie is Minecraft (yay Notch) and for self-esteem boosting camaraderie is the arts.
Telling women we have to man up to confidence and ambition just ignores our strengths. In a diverse world, let's enjoy the ways men project confidence and learn to recognize and appreciate the ways women are confident and convey that.
And focus on creating more interest in STEM by showing women in STEM roles doing work that makes a difference for life on this planet. And focus on creating support pools of camaraderie. For girls, and boys too. We create such opportunities for sports, for music and the arts. We need to do more in STEM and software engineering in particular.
Uh. Not sure about the title to this post. It's so not me!
I also write at:
- Bredemeyer Resources for Architects
Strategy, Innovation, Ecosystems
Scanning Trends and Other
Architects and Architecture
- Todd Hoff (highly recommended)
- Anna Liu
- JD Meier
Architect Professional Organizations
Agile and Lean
Agile and Testing
Other Software Thought Leaders
- CapGeminini's CTOblog
CTOs and CIOs
CEOs (Web 2.0)
- Don MacAskill (SmugMug)
- Wired's monkey_bites
Social Networking/Web 2.0+ Watch
- Dan Roam
- David Sibbet (The Grove)
Strategy, BI and Competitive Intelligence
- Freakonomics blog
Um... and these
- CNN Money Business of Green videos