Empiricism makes everything better

I am fully aware that making broad and far-reaching proclamations like the one in the title of this post in dangerous. Yet, having thought about this topic long and hard, my credence in the value of empiricism is rather high and my hope is that if you manage to read the whole post you will come to a similar conclusion. Let’s get started.

What is empiricism

As Wikipedia explains:

Empiricism is an epistemological theory that holds that knowledge or justification comes only or primarily from sensory experience.

As a philosophical theory, it has a long and rich tradition dating back to the the sixth century BCE with many famous thinkers making major contributions to the core concepts and ideas over the last two and a half millennia. The modern version of empiricism that people typically think of originates chiefly from 17th and 18th century empiricists like John Locke, Francis Bacon, Thomas Hobbes and David Hume.

A wise man proportions his belief to his evidence — David Hume

On Hume’s view, our knowledge of the world is based on sense impressions and such “matters of fact” are based on experience. In practice, empirical research would then typically progress along the following path:

David Hume

1. Observation — Gather empirical data

2. Induction — Frame a general conclusion from the gathered data

3. Deduction — Create a conclusion from the experiment

4. Testing — Analyse and validate collected data with appropriate statistical methods

5. Evaluation — Present the gathered data and the conclusions of the experiment and assess their validity, value and consequences

It is worth contrasting empiricism with the philosophy of rationalism which posits that knowledge can also be developed by exploring concepts and through deduction, intuition and revelation and that knowledge can be created even without prerequisite sensory experience. In my experience both rationalism (sometimes known as intellectualism) and empiricism are important avenues through which we acquire new knowledge and understanding. As such both need to work hand in hand in most human endeavours to drive progress.

But enough of philosophy — my aim in this post is not to provide a full and holistic overview of empiricism as a philosophical concept (which I am not qualified to do), but instead I would like to show that having empiricism as a mindset is important and valuable both in life and in how we build, run and grow organisations.

Empiricism as a foundation of progress

Scientific method and empirical research are perhaps the most transformational ideas humanity has ever come up with. These ideas and methods have lifted humanity from poverty, powered the development of technology and medicine and enabled a fundamental transformation of our society resulting in a creation of a completely new epoch, Anthropocene.

It was the mindset, ethos, practices, and methods of empiricism that enabled people to think more deeply and more honestly about what is real, what is true, what they should (and could) believe and why. It is empiricism that also dramatically accelerated society’s departure from the supernatural and the spiritual (recent, hopefully temporary, regression in some countries notwithstanding) and enabled a gradual liberation of whole peoples from the millennia of religious dogma. For all these reasons, it is hard of overstate the impact empiricism had on the society as a whole and on the pace and direction of human progress.

When empiricism is absent

Now, so far all we have been talking about empiricism purely at a conceptual level. Let’s now explore how ideas of empiricism translate into a business context. In my mind, a powerful way to illustrate the value of empiricism is to describe a fictitious organisation, let’s call it Legacy Ltd., where empiricism is entirely absent. At Legacy Ltd, the following is very common:

• Most decisions are made exclusively by people with the strongest opinion, those who are most assertive at advocating their views and those who are paid the most

• Strategy is often based on hopes, assumptions, gut feelings and ‘strong intuitions’

• True level of organisational maturity is not understood or is based on unfounded beliefs

• Major structural problems and issues continue to ‘bubble’ deep down in the organisation without senior management ever admitting there’s a problem (even if people are fully aware of them). Issues and incidents are explained away as bad luck, unforeseen circumstances, unexpected human errors or are downplayed as unavoidable.

• People’s career growth is very positively correlated with their ability to ‘manage the message upwards’ and ‘tell a story’.

• Dubious, irrelevant, unreliable, or incomplete data is used as justification for decisions or investments

• Vanity metrics used to measure organisational and team performance

• The organisation spends a lot of effort on reinforcing existing beliefs and ideas

The list goes on. In various guises, many of us have experienced one or more of such things in our careers. If any of the above looks familiar, your organisation or team would certainly benefit from more empirical thinking and more empiricism-based practices.

Why is practicing empiricism hard?

So if lack of empiricism tends to result in poor outcomes, why don’t all leaders, teams and organisations fully embrace empiricism and thus gain the many benefits it brings? Well, as is usually the case, there are many reasons. In this post, I will outline only three of them.

Firstly, practicing empiricism requires a culture that values seeing things as they are not as people would like them to be. I am talking about a culture which values truth above everything else regardless of how painful, unpopular, or difficult to accept it might be. Way too often people have incentives to distort reality in a specific direction — perhaps they feel threatened in their role and feel that any admission of a problem may weaken their position. If the cultural context they operate in requires careful ‘management upwards’, navigating complex politics, dealing with power struggles, or being reprimanded for bringing up bad news or challenging things, it is only natural that people prefer to employ creative ‘spin-doctoring’ instead of empiricism.

Empiricism implies facing reality, digesting it, and shaping our actions based on what is reasonable to believe to be true.

Empiricism also requires having means to assess things in some empirical manner. This usually requires data, insights, analyses, feedback, and other types of inputs all of which need to be reasonably reliable. Without such inputs it is not possible to establish what is true and real what is fiction or wishful thinking. Organisations often either lack trustworthy data and insights or, and that could be worse, knowingly or unknowingly work with data that’s misleading, irrelevant or incomplete. This is usually a consequence of measuring what you can vs. measuring what you should.

Finally, empiricism requires intellectual humility, curiosity, absence of hubris and appreciation of one’s fallibility. These are all exceptionally valuable qualities to have and any team formed of people with such qualities is destined to grow and succeed. Alas, the world around us often rewards and incentivises different behaviours.

Empiricism in practice

As we said, empiricism implies facing reality, digesting it, and shaping our actions based on what is reasonable to believe to be true. It means grounding our actions and strategies in what is real and continuously validating if our understanding is correct. Here is a list of basic principles, approaches and ideas that, in my experience, support empirical thinking:

• Don’t distort facts — good or bad. See things as they are, even if you don’t like what you are seeing. That means not hiding from bad news but also not mislabelling mediocrity as excellence.

• Reward and recognise people who are honest and transparent and who help you see what’s true. Publicly recognising people whose work is grounded in empiricism enables others to gain confidence to do the same.

• Promote and foster a culture of open enquiry and curiosity. Lead by example.

• Use (the right) data, insights, and feedback to help you establish what is real and what is not. Pay attention to what is measured, why and how and be conscious of Goodhart’s law.

• Continuously evaluate how valid and representative your measurements, data and insights are and to what extent your conclusions really follow from the data (premises) — see Inductive Fallacies.

• Get used to working with credences and degrees of confidence as opposed to absolutes. Your confidence in something should always be proportionate to the evidence you have and the quality/reliability of that evidence.

• Be wary of confirmation bias and Dunning-Kruger effect — nobody is immune to it. Be careful about the questions you are asking yourself e.g. Can I believe it? vs. Must I believe it?

• Make Bayesian thinking part of your mental toolbox and use it to update your understanding of reality.

Empiricism is, of course, not a panacea, but I strongly believe that it needs to have a much more prominent place in the minds of leaders both in government and in business. The agile community, for instance, has done a great job at promoting empiricism as one of the foundational ideas of what agile is all about and reminding organisations how valuable it is. Another example might be Ray Dalio whose leadership philosophy is strongly driven by empiricism — just read his ‘Principles’. Yet, I still maintain that we don’t have enough senior leaders whose ethos is grounded in empiricism and related ideas.

I hope I succeeded in making a good case for empiricism as an approach and mindset. So now it is over to you: How can you lead by example and bring a bit of David Hume and Ray Dalio into your team or organisation? Where and how can you best introduce empiricism in your work and your life?

The Yahaba Technique

Introduction

How are our future concerns represented in the decisions we make today? How can the people who will inherit our world influence actions we are taking and the policies we put in place? How shall we balance the needs of today with the needs of tomorrow?

In 2015, representatives of residents of Yahaba, a small town in northeastern Japan, went to their town hall to take part in a unique experiment. Their aim was to develop policies that would influence the future of Yahaba. But this time, there was a twist. Before debating municipal policy, half the participants were asked to put on ceremonial robes and imagine they were from the future, representing the interests of the current citizens' grandchildren. Specifically, they were told to imagine they were from the year 2060 and therefore representing the interests of a future generation. The results were very encouraging. Not only did researchers observe a "stark contrast in deliberation styles and priorities between the groups," the concern for future generations was infectious — among the measures on which consensus could be achieved, more than half were proposed by the imaginary ‘grandchildren’.

Organisations have a similar challenge. Like councils and governments, businesses need to think very carefully about the long-term. This is not only because most businesses hope to be successful not only today but also tomorrow, next year and in five years’ time and thus create a long-term value for shareholders but also because some policies, strategies and investments take many months if not years to yield results.

Concerns of tomorrow are therefore very relevant today. And whilst the time horizon most organisations today consider is typically in years rather than decades, the challenge remains the same: How do we practically bring considerations and perspectives of the future people into our conversations? How do we ensure our decisions reflect and take into account the viewpoint of people who may be in our roles in one, five or even ten years’ time? As I wrote elsewhere, long term success is very closely linked to the organisation’s ability to grow its overall maturity and that often involves making difficult trade-offs between the short-term and the long-term. Only organisations that get this balance right can hope to remain viable in the long-run.

What is the Yahaba technique?

In business and especially in tech there’s very often a tension between things we absolutely must complete here and now (e.g. because our commercial success directly depends on it) and things that we have to focus on in parallel so that we can continue to grow, remain adaptive, scale our operation without impeding agility and be able to innovate also in the medium and long-term.

To turn this tension into an advantage, to make more informed decisions and to properly examine what really matters to us and why, I propose the Yahaba technique.

Here’s how to use the Yahaba technique in practice:

1. When considering a strategic, organisational, structural, technical or other significant decision, bring together a small group of people whose input is relevant for the decision. In practice these could be experts in the field, senior leaders and managers who will need to lead the implementation of the decision, people who will be impacted by the change the most, customers, stakeholders etc. Keep the group small, ideally between three and eight.

2. Designate up to a half of the group participants (i.e. typically 1-5 people) to be the Shōrai (将来, Japanese for ‘future’). The rest of the participants stay in their ‘normal’ roles i.e. represent their perspective on the matter as they would normally do.

3. The role of the Shōrai is to put themselves in the shoes of the people who will have to live with the decision being discussed and to represent the view of these future people during the discussion. The Shōrai are expected to come up with arguments, ideas, challenges and proposals to shape the decision being discussed in a way that they believe would be most favourable from the perspective of people who will be leading the business, enhancing and operating the platform or running a business process in a few years’ time.

4. The group as a whole explores the decision in question from various perspectives and aim to reach a consensus about a set of specific proposals, actions, ideas and approaches that, in their view, best represents the varied views in the room and balance the different outcomes and concerns voiced during the conversation.

A few things to keep in mind

Running a good Yahaba session is not difficult, but there is a number of considerations you should keep in mind when using this technique:

• Avoid making the group too large. My strong recommendation would be to limit the participants to the maximum of ten people, ideally less.

• How do you decide who will play the role of a Shōrai? It is best to start with asking people to volunteer. Typically, people who are most concerned about the future implications of the decision(s) in question will happily put their hand up. At the same time, it is also insightful (both for the group and for the individual) to invite someone who typically tends to favour short-term concerns over the long-term ones to be one of the Shōrai. It not only forces them to think differently, but it also makes it possible for the rest of the group to obtain a new perspective on the matter in question.

• If are a Shōrai, your job is to be a passionate but considerate and reasonable advocate of the concerns of future people (colleagues, customers, leaders, stakeholders etc). Your task is not to overwhelm the rest of the group with a set of purely ‘self-serving’ demands or to dismiss other people’s concerns for the urgent issues of ‘today’. Also, consider that too much emphasis on the long-run may result in failing in the tasks of here and now. This in turn may mean there will be no ‘long-term’ anymore and thus taking a position that’s too biased towards the future may be self-defeating.

• It is a good practice to rotate the role of a Shōrai to different people. This can be done for every decisions/session or even once or twice during the same session. The aim is to come up with the best possible argument from both short and long-term perspective and allow the group to discuss these fully.

• It helps to have a facilitator who will ensure the discussion is kept focused, that arguments are presented clearly and that people stick to their roles. The facilitator should also help capture different arguments being presented so that the group can explore them properly as well as actions agreed and decisions made.

• Yahaba sessions could be as short or as long as you see fit — it all depends on the number of participants, nature of the decision being discussed and complexity of perspectives. As a rule of thumb though, my recommendation is to keep the session between 30 and 120 minutes. A shorter session is unlikely to enable all perspectives to be meaningfully considered. On the other hand, sessions longer than two hours tend to gradually lose focus and you may find yourselves going in circles.

• It may not be practical or desirable to run a full-fledged Yahaba session for every decision. Consider the significance of the decision you are making and the risks involved. Based on that you can determine if and how the session should be run, who should be invited, how long it should be etc.

• When running the meeting remotely (e.g. via Teams, Zoom etc) you may find it helpful for the Shōrai to temporarily add the word ‘Shōrai’ to their name. Alternatively, the facilitator may post who the Shōrai are into the meeting chat.

A Yahaba session in practice

Here’s a simple made-up example of how a team-level Yahaba session may look like in practice. In this example session, taking place at a on-line clothing retailer Peak Cloud Ltd., we are observing one of the teams trying to decide about the best approach when implementing a new major capability. At a senior level, the nature of the discussion, the time horizon people are concerned with and the trade-offs involved would, of course, be different. But the example below should illustrate how a simplified Yahaba session may look like and how people in the Shōrai role can play a vital role in illuminating the consequences of decisions being taken.

The participants in this session are:

• John — Scrum Master and session facilitator

• Claire — Solution Architect

• Sandra — Lead Developer

• Marcus — Product Owner

• Tom — Senior Developer

• Anthony — SRE

John: Hi all and welcome to our Yahaba session! Today, we will be talking about a rather significant decision we need to make. As you know, over the next few sprints Marcus would like us to develop a new multi-level filtering capability for customers who are looking to buy one of our products and want to filter based on a range of criteria. We have an existing solution in place and can look to enhance it to support the new requirements but Sandra raised a concern about that due to the complexity of the code in that area. We all know there’s a deadline we are working towards and there’s a lot to do. To make sure we are making an eyes wide open decision about the implementation approach, we decided to run this Yahaba session. We have about 45 minutes so let’s get started. Is everyone ready?

(people nod)

John: Great, thanks! So, first things first, I am happy to be the facilitator today. Is everyone happy with that? Awesome. Next, we need to appoint our Shōrai for today. Any volunteers?

Sandra: Yes, happy to be one of the Shōrai today.

John: Thanks Sandra.

Anthony: Me too.

John: Thanks Anthony! Anyone else? No? Ok, fine, let’s do this. Marcus, do you want to set the context please?

Marcus: Sure. As you know, we have a set of major new product product launched coming up in July and that will significantly increase the number of products and their variants on our website. Having a really good filtering capability is essential and we need to have it on day one to maximise the conversion from the outset. I appreciate there are challenges in implementation of the filtering logic, but we just need to get it done.

Sandra (Shōrai): Thanks, and I get that Marcus. As a Shōrai though, I am now thinking how this will feel like in 6-9 months’ time. Because of the decision being discussed today to simply bolt on a lot of additional logic on top of an already complex filtering mechanism, I am now stuck with a massive amount of complexity in this area of code, it keeps constantly breaking and filtering is now very slow. Our customers complain about it and it is affecting our conversation significantly. The team is now spending a huge amount of time just fire-fighting issues with this logic and we can’t support any more enhancements.

Anthony (Shōrai): I agree Sandra. Operationally, this is nightmare. Because the filter logic is tightly coupled with the rest of our system, any deployment is now very slow and error-prone and we tend to introduce defects which are hard to detect and even harder to fix. The amount of testing we now need to do is huge even for basic changes which have nothing to do with filters!

Claire: Hang on. It can’t be that bad! Yes, things are coupled but it’s working fine now and it’s not causing us that much trouble, is it!

Marcus: Exactly! Plus, as I said, we really can’t afford to delay this — it has to be done by July!

Tom: I agree with Claire that this is not causing issues now, but I want to hear more from Anthony as one of the Shōrai, about the operational issues. Can you please elaborate?

Anthony (Shōrai): Of course. What I am seeing now, speaking as a Shōrai about nine months in the future from the rest of you, is that the code is now so complex that none of the teams are prepared to touch it and the we struggle to test it effectively. Our deployment cycles have increased because of the constant back and forth around broken filtering functionality which nobody can fix. And when John asked us how long it would take to refactor it, the answer was at least three months!

John (facilitator): OK, thanks everyone. Sandra, Anthony, what are you proposing we do?

Sandra (Shōrai): As I said, I now see major issues with the way filtering works because we took the shortcut nine months ago to meet the July deadline…

(Marcus interrupts…)

Marcus: Well, Sandra, if we don’t get this done by July, your concern about complexity etc. will be irrelevant as we will miss our yearly targets, so…

John (facilitator): Hang on, Marcus, let our Shōrai speak please. Sandra, you were saying?

…

(Sandra explains her concerns from a perspective of someone working on the code in the future and together with Anthony adds a number of examples about things that are causing problems)

…

Sandra (Shōrai): …and to avoid this scenario, I propose to split our work over the coming weeks into two streams. Stream one will be about separating the existing filtering logic from the rest of the system which will alleviate a significant part of the operational pain Anthony is talking about as we will remove all the coupling. The second stream will be focused on building the new requirements Marcus mentioned.

Claire: Yes, but that does not solve the code and logic complexity issues you called out Sandra.

Sandra (Shōrai): No, it doesn’t, but it does significantly reduce the operational impact AND means we can still get close to July with the new functionality.

Tom: I am ok with that, but when will we refactor the rest of it? I mean, we will still have a hugely complex code in this area!

Marcus: If it helps, I am fine with blocking up to two months of our roadmap from August to allow for a proper refactoring of this.

John (facilitator): Would that work Sandra? Anthony, Claire — what are your thoughts?

Sandra (Shōrai): Yes, with encapsulation done I think two months works.

Anthony (Shōrai): Works for me. As long as we have an independently deployable and encapsulated module I am happy to go ahead.

Claire: Fine with me too. But I will need time over the next few weeks to work with Tom and Sandra to shape how the design for the new filter will look like so that we can hit the ground running in August. Can we put some stories in for that John?

John (facilitator): Sure. You happy with that Marcus?

Marcus: Yep, I am fine. Thank you.

John (facilitator): OK, great. Seems like we have a way forward. I will capture the actions and the summary of our conversation and share it with you. Thanks all for this session!

Conclusion

As you can see, this technique can be used at various levels of the organisation (from the exec board, to teams and down to individuals) and in many different contexts and to varying degrees. My recommendation is to try to include an element of Yahaba in any decision you are making. After all, considering future concerns is a healthy habit to develop and it will also help you hone your systems thinking skills. I hope I managed to make a good case for the Yahaba technique and encourage you to try it and make it part of your toolkit when trying to make important decisions.

The Yahaba technique is, of course, not a panacea and it will not magically enable you to solve all your problems. For it to work, people need to be engaged, argue in good spirit, be prepared to listen to each other and consider a range of competing perspectives. Having said all that, just the act of giving a ‘voice’ to the future people can be immensely valuable and is very likely going to help you and your team or organisation to make much better decisions and achieve better outcomes. And in my mind, that’s well worth it!

How I think about: Uncertainty

This is part one of a series of posts focused on a range of topics, concepts and ideas that, in my experience, significantly influence the level of overall success in any organisation. Some posts will be more about specific practices and approaches while others, like this one, will be more conceptual and aimed at creating a foundational understanding and a basic mental model of a specific idea. In this post, I hope to provide a perspective on the phenomenon of uncertainty and its different flavours and introduce some basic concepts and approaches which help to navigate uncertainty in business and in life.

The urge to ‘know’

As humans, we like to know. We like to feel we are in control. We want to know that we are in charge of our destiny — be it in our personal lives or in the context of the organisation we are part of. And we go to extraordinary lengths to reduce uncertainty. We analyse and compare scenarios, we devise intricate plans, and we identify and manage risks. We do all these things and more in a hope that we will reduce or ideally eliminate the uncertainty. And yet, despite all this effort, we continue to be surprised by what actually happens and how difficult it is to be truly ‘in control’ in reality.

The human desire to be certain, to be in control is natural and it stems from our evolutionary history. Being able to reduce uncertainty and predict the outcomes of our actions or decisions would be hugely valuable. In many areas of life, we pay experts to help us reduce uncertainty (think investment bankers, financial advisors, pollsters, political advisors etc.). Some people even pay charlatans to make them at least feel that they ‘know’ (think astrologers, psychics, fortune-tellers etc). And most of us spend money to reduce possible negative impacts of uncertainties in life (think insurance).

We crave certainty because the feeling of being uncertain is something deeply uncomfortable, distressing and altogether undesirable. It creates anxiety, makes decision-making harder, and forces us to keep changing our plans. Uncertainty can be so uncomfortable that we are prepared to fool ourselves into feeling confident or certain about something even if such feeling is not grounded in any meaningful data or evidence (think most religions). Moreover, an uncontrolled quest for certainty resulting in holding one’s opinions too strongly hinders progress, impedes innovation, perpetuates falsehoods, ignorance and injustice and shuts down conversations (think religious or cultural dogmas, Taylorism applied to knowledge work, superstitions, creationism etc.)

In a nutshell, feeling uncomfortable with uncertainty is very human. And while working effectively towards reducing uncertainty is logical and can be hugely advantageous in life and business, our discomfort with uncertainty can and does often lead to waste and creates real-world harm. Getting good at living with, managing and exploiting uncertainty, therefore, makes a lot of sense.

Where does uncertainty come from?

Uncertainty is all around us. The more you look, the more uncertainty you will find. In fact, even events that we normally tend to consider ‘certain’ (the sun will rise tomorrow) carry a degree of uncertainty. So where does all this uncertainty come from? At the most basic level, uncertainty is both a physical property of the universe (e.g., quantum mechanical events that can only be described in probabilistic terms) and of human society and the world we live in (e.g., moral uncertainty about what is the right thing to do in a given situation or the impossibility to practically anticipate all effects of all our actions). As we shall see later, while some uncertainties can be reduced, others can’t.

In a business context, uncertainties are inherent to running a business and, as such, they have many forms — political, economic, social, structural, financial, organisation, technical etc. As a result, businesses spend billions on dealing with uncertainty. Yet, in many cases, a lot of this investment is wasted and yields zero or even negative value.

Types of uncertainty

It’s important to recognise that there are several distinct types of uncertainty and dealing with each type of uncertainty requires a different approach and a different mindset. In addition, not all uncertainties can be meaningfully reduced or eliminated, some not even in principle.

The main types of uncertainty include:

• Epistemic uncertainty — being uncertain due to a lack of knowledge or understanding e.g., I am uncertain about the colour of the t-shirt my son is wearing today

• Aleatory uncertainty — being uncertain in principle e.g., when flipping an (ideal) coin, I am uncertain if I will get heads or tails.

• Ontological uncertainty — different parties in the same interactions have different conceptualisations about what kinds of entities inhabit their world, what kinds of interactions these entities have and how the entities and their interactions change as a result of these interactions.

• Semantic uncertainty — different participants in the same interactions giving different meanings to the same term, phrase and/or actions. The words and concepts that we are using are inadequate to describe what we are trying to explain.

In the case of epistemic uncertainty, if we gain more information (e.g., by doing research, taking more measurements, conducting tests etc) the uncertainty can be reduced.

Aleatory uncertainty, on the other hand, is irreducible in that there will always be variability in the underlying variables. These uncertainties are characterised by a probability distribution and need to be dealt with as such.

Ontological uncertainty is about different people (teams, organisations, social groups) having different mental models about the state of the world, and/or different perception of how cause and effect work in the given context.

Semantic uncertainty, similarly to Ontological uncertainty, results in communication and interaction issues, confusion, misunderstanding and conflict as people talk cross-purposes. The latter two types of uncertainty can be reduced e.g. by explicit definition of terms, their meaning and public ‘validation’ of these terms to create a shared language and taxonomy across a team or an organisation. One should also openly examine and discuss existing perceptions and mental models that different parts of the organisation have (e.g. using Causal Loop Diagrams and other techniques) and seek a common interpretation.

In organisations, we encounter all four types of uncertainty regularly, yet we rarely think more deeply about what type we are dealing with, and if and how can it be effectively reduced. Way too often we implicitly assume that all our uncertainties are epistemic in nature.

Finally, uncertainty is, of course, very closely related to the concept of complexity. Highly complex systems, organisations, processes, and interactions often tend to create or amplify existing uncertainty as relationships between cause and effect are unclear and emergent and most actors have only a limited understanding of the system as a whole. In such systems, most actions have a range of second, third and higher-order consequences some of which are highly uncertain. Meaningfully reducing such uncertainty often requires a more fundamental rethink of the organisational model, process, structure, technology, the flow of work or other core ‘properties’ of the system as a whole and reducing the overall complexity.

What can we do?

Organisations need to get much better at embracing uncertainty.

By ‘embracing’, I specifically mean taking steps to:

• recognise when a specific uncertainty exists

• understand the nature of the uncertainty

• clearly describe its nature and properties and carefully think about them

• assess the cost AND the value of reducing/eliminating the uncertainty

• where it is meaningful, effectively reduce the uncertainty (but only to the extent warranted by the cost/value analysis)

• establish a culture and technological, organisational, structural, and systemic capabilities which enable working effectively in the presence of uncertainty and respond to change

• create and foster organisational ’antifragility’ on multiple levels

“Not to be absolutely certain is, I think, one of the essential things in rationality”

— Bertrand Russell

The above should be read less as a checklist and more as a set of practices, qualities and organisational traits — a mindset about uncertainty. In this context, I would like to emphasise that reducing uncertainty comes at a cost. To reduce the level of uncertainty, we will need to invest time, effort, money, and other resources. This, of course, promises a reward in a form of greater confidence, better predictability, or more optimal and less risky decisions. However, it also comes at a cost which can be very significant. Way too often costs of reducing uncertainty remain hidden and are not explicitly talked about. This can result in an excessive (and ultimately futile) focus on uncertainty reduction (e.g., through more analysis, planning, detailed specifications etc) even though a focus on organisational, technical and process changes enabling effective uncertainty management and real organisational responsiveness/adaptability/agility would have been a much smarter and more productive strategy. Finding the balance is hard and, as is often the case, it all starts with the right conversation.

In practice, there are many ways, tools and techniques which enable us to work with uncertainty better. Context, as always, matters but, in general, when navigating uncertainty I find ideas like the Cynefin framework, Bayesian inference, Decision Theory (including basic concepts like Minimax, Maximin), Systems thinking, and Probabilistic forecasting very useful. When used in the right way with the right level of understanding of the uncertainty we are dealing with and the overall system or area we operate within, these approaches yield remarkable results and enable us to fundamentally change the conversation and the way we look at dealing with uncertainty.

Uncertainties, like taxes, are here to stay. Trying to completely eliminate them or ignore them is dangerous, can be very costly (and futile) and leads to bad outcomes in the long term. Uncertainties also create opportunities. Exploiting these opportunities requires teaching ourselves to live with, embrace and take advantage of uncertainties around us. It means getting comfortable with the discomfort, thinking more deeply about the uncertainties we need to work with and reframing the conversation we are having. Organisations that embrace uncertainty tend to reliably outperform those that don’t — both in the short and the long run. Good luck!

PS: Found an error? Or a typo? Did I get something wrong? Or do you have an idea you’d like to share? Please let me know!