An Institute for High Impact Decision Making (IHIDM)

Brief outline proposal

SUMMARY

Most high impact decisions in government, industry and the third sector involve radical uncertainty where no clear optimal solution can be identified at the time of making the decision.

Currently, most of these decisions are made using Optimal Choice

Frameworks (such as Cost Benefit Analysis or Randomised Control Trials). When this reduces the complexity of the problem by masking assumptions about hard-to-imagine uncertainties, it can lead to poor decisions and strategic surprise. It may be especially unhelpful if the aim is to achieve innovative outcomes in new situations not already well understood, particularly those not easily accessible to a command and control framework.

Recent research has proposed a new approach to decision-making based on the latest findings from the social, mathematical and brain sciences that explicitly recognises the ambivalent nature of these decisions and explicit role of “imaginaries” (tools and concepts that explicitly or implicitly influence the decision-making process) these include: modelling, narratives, imagination and emotions.

Because research has identified a key knowledge gap in this area, the UK is well placed to exploit it. Successful research in this area will enable the UK government, industry and third sector organisations make more efficient and effective decisions in complex, uncertain and devolved settings and there will be significant export potential for the new knowledge and tools developed.

PROPOSAL

Create a new cross-disciplinary Decision-Making Institute bringing together experts, from a broad range of academic disciplines, advisors and decision makers to develop new tools and techniques to support complex decisions based on real-world problems.

The Institute would provide support to senior decision-makers with current issues, develop and deliver an ambitious research agenda, co-created between the institute’s stakeholders, as well as developing human capacity in this domain.

The cost of the institute would be £10-20m over 5 years (depending on ambition, scale and location). This would cover overheads, the core leadership team, and a number of junior and senior research fellows, domain advisors and administrative support as well as fund a targeted research programme.

It is envisioned that the core research programme would be supplemented by additional income from research grants and industrial contributions.

The Institute and its research agenda would be overseen by a high-level board made up of senior stakeholders from academia, government and industry. This will ensure the work of the institute is both academically rigorous and remains focused on real-world problems.

This proposal has already received significant support from the Chief Scientific Advisor community and within the Research Council as well as from key figures across the Banking and Insurance industries.

BACKGOUND

Most high impact decisions made in government, industry and the third sector, involve radical uncertainties. Hence, they is no clear optimal solution when they are made. However, most of these decisions are currently made using tools based on an Optimal Choice Framework (OCF) (such as Cost Benefit Analysis or Randomised Control Trials) that use a probabilistic framework to determine the best solution. Although formal OCF frameworks can be useful for answering or thinking through some questions, in cases of deep or radical uncertainty there is a clear imbalance between the required decision support tools and what is currently available from consultants or academics.

The CRUISSE network (Confronting Radical Uncertainty in Science, Society and the Environment) [1] was set up by the UK research councils in January 2017 to produce a report on the current landscape of research in real-world decision-making under uncertainty and to suggest a future research agenda. The network was built up of academics and policy makers with a wide range of backgrounds in physical, mathematical, social, political and psychological science [2].

The key CRUISSE finding was that focusing on optimality in decisions characterised by radical uncertainty is likely to be misleading. Models based on an OCF approach risk leaving out key information and lead to the threat of strategic surprise. Optimal choice approaches generally reduce large world problems, characterised by ill-structured problems, unknowable outcome sets, and complex dynamic environments to small world problems with well-defined goals, known outcome sets and probabilities. Uncertainties or poor assumptions in the supporting data and modelling assumptions used to arrive at this ‘optimal’ choice are often lost or ignored during the decision-making process. This often leads to poor decision making.

The CRUISSE work identified a new approach to addressing these complex decisions based on the explicit recognition of the failures of OCF approaches. The new approach encourages decision makers to recognise the intrinsic ambivalence [3] experienced when making decisions under uncertainty. This requires instead of using existing data, models and frameworks as inputs to an optimisation problem, they are used as “imaginaries” (together with other tools such as narratives, emotions and heuristics) to understand the problem context. They then become tools that enable decision makers to face and manage the ambivalence intrinsic to decision making under radical uncertainty.

To further develop, test and implement this new approach, requires both the breaking down of the silos between academic disciplines and the forging of much stronger links between the research community and real-world decision makers. The co-creation of research agendas between experts and decision makers is key to better understanding what drove poor decisions in the past, identifying examples of good practice and developing successful tools to support better decisions in the future. This includes understanding the strength and weaknesses of the available tools within a given context, how to select them appropriately and ensure they are used appropriately.

A new UK networked institute is proposed to further develop and apply this new decision-making approach and to build stronger links between academia and real-world decision makers across government, industry and the third sector. This cross-disciplinary institute would support clients in making complex decisions, develop and deliver a new research programme on the science of decision making under uncertainty build human capability in the area.

The cost of the institute would be £10-20m over 5 years (depending on scale and location) covering overheads, staffing and the core research programme. The institute would be led by a Director and Deputy Director and six diverse part-time professor level appointments drawn a wide range of expert theoretical communities. The budget would also provide funding for a core research programme addressing academic research issues identified for deeper study. It is envisioned this core research budget would be supplemented by additional contributions from research grants and industrial funding. The institute would also employ a number of junior and senior research fellows and domain advisors to work on specific projects and/or the wider research programme.

The Institute and its research agenda would be overseen by a high-level board made up of senior stakeholders from academia, government and industry to ensure the work of the institute is both academically rigorous and remains focused on real-world problems.

This proposal has already received significant support from the Chief Scientific Advisor community and within the Research Council as well as from key figures across the Banking and Insurance industries.

---

Annex A – The Proposed Institute

We believe there is both an urgent need, and the capability available, to construct an ambitious programme of research; which brings together academics from multiple disciplines and senior practitioners facing problems involving radical uncertainty. There is great potential in delivering a programme of co-constructed research that uses real-world challenges to develop both new tools for practitioners and a new science of decision-making under radical uncertainty founded in an understanding of human capabilities.

We propose creating an Institute for High Impact Decision-Making designed to enable and facilitate close, iterative collaborations between decision-makers and leading researchers.

Institute Principles

(i) The Research Agenda will be co-created by Real-World Decision-Makers and Academic Researchers.

(ii) The Research Agenda will be implemented by highly Cross-Disciplinary teams.

To implement these principles, the Institute would bring together a range of expert communities that have “theoretical”, “domain”, “knowledge broking” and “decision-maker” expertise.

 

Theoretical Expertise: Academics who study decision-making under radical uncertainty from a range of disciplinary perspectives.

Domain Expertise: Leading academics or practitioners with expert knowledge of a particular decision- making domain being examined.

Knowledge Brokers: Leading practitioners (sometimes academics) with significant experience in the specific decision-making environments being supported.

Decision-Makers: Members of the C Suite (and their advisers) in companies, government departments and organisations. Decision-makers will co-create the research agenda and are the ultimate audience for the Institute – the target ‘clients’ for project-based collaboration and support.

We propose that the Institute be based around a small number (5-6) interacting teams that will build expertise covering:

a)  Co-creative modelling and Applied Mathematics into practice (working with decision-makers to design and put into practice data analytic devices for problem-solving.)

b)  Experiential Simulation (e.g. Scenario building, foresight, gaming.)

c)  Design-Experimental (e.g. adaptive decision-making, design-based thinking).

d)  Narrative, Naturalistic and Heuristic Approaches (e.g. conviction and identity narratives; simple tools.)

e) Social and Group Behaviour (e.g. leadership, institutional; group dynamics, science and technology studies.)

A Decision-Making under Uncertainty Research and Evidence Hub (the ‘Hub’) would lie at the heart of the proposed Institute and be the primary means for achieving the co-creation principle that CRUISSE pioneered with its pilot projects. The Hub would comprise an outgoing team, comprising a leadership team and domain experts linked to the interacting communities, whose function would be to create interactions between decision-maker practitioners and diverse theoretical communities and to collate and publish relevant materials and manuals to meet the need for immediate,intermediate and long-term [4] consultation and collaboration requests from business, government and the third sector. In each case, the approach begins by listening and observing – perhaps a visit to the decision-making site by a combination of creative mathematical modellers, psychologists and sociologists to decision-makers in government, business, regulatory and third-sector communities to discover which decisions and analyses they are finding challenging and why they think this is so.

Institute Research Agenda

A significant limitation of current research knowledge in the area is that it is the outcome of questions posed in academic research silos. What is required is a research agenda with scope for co-created research challenges, formed through a process of academics identifying with decision-makers the decision-making problems the latter experience. In context, research workers would be enabled to design questions and ways to investigate them that address real-world problems and incentivize research workers to work with relevant investigators from other disciplines and to use all their imagination and skill to innovate. Two forms of research are currently most likely to be successful:

a)  Descriptive and Documentary Research answering the questions “What is going on here?Because research to date has mainly been based within an Optimal Choice Framework, it has often excluded both the study of real-world decision-making process and the study of howradical uncertainty influences real-world decision-makers. We lack basic knowledge of what is going on.

b)  (Mostly) Field Experiments. Descriptive work can produce hypotheses and build a case for their initial validity, but field experiments (working with decision-makers) are likely to be the best way to test.

Organization and Staffing

The Institute requires a core leadership team comprising a Director and Deputy Director and six diverse part-time professor level appointments drawn from all five of the expert theoretical communities mentioned above. They will have both the ability to assemble the communities/networks for research and the experience of developing and implementing the techniques that link them together effectively. Other staffing needs include a range of junior and senior research fellows, domain advisors and administrative support.

Budget Estimates

At current University rates, housed in a university, the Institute decribed would cost £10,656,000, over a five year period, so that, under the usual assumptions, the price to the funder would be £8,525,000. Work over this period could also include establishing a practice organisation membership model designed to support a path to funding sustainability.

---

Appendix 2. The CRUISSE Approach

Confronting Radical Uncertainty: A New Research Framework for Risky Decision-Making and Decision Support.

All simple decisions are alike; every truly risky decision confronts radical uncertainty in its own way.

Some decision-making questions can be answered with recourse to scientific model-making in a relatively simple (though not necessarily easy) way. First, define the boundaries of the problem and the qualities of a good solution. Next, optimize the decision quantities such that the best solution is found. This strategy has powered the quantitative sciences to the unparalleled social position they enjoy today and solved many problems from the microscopic (developing inkjet printers) to the global (curbing CFC emissions). In many arenas this strategy is still a viable source of useful insight and development.

Other decision-making questions, however, are not amenable to the above paradigm. Perhaps the boundaries are uncertain, the values are contested, multiple models disagree, causal mechanisms are unknown, or the results show extreme sensitivity to inputs or the choice of framing criteria. In business and government, almost all decision questions fall into this category, and they are high-impact decisions that involve the possibility of significant loss or significant success. We call this context “radical uncertainty”.

We aim to improve the state of cross-disciplinary support for evidence-based real-world decision-making given radical uncertainty. The CRUISSE collaboration began with a deeper conversation between physical and social scientists and decision-makers at the heart of business and governments. The kinds of questions we consider include:

• whether to build a high-speed rail link;
• when to abandon a flood defence;
• when is an engine component more likely to fail;
• how accurately to predict the properties of materials and structures;
• which NHS treatments to prioritise;
• when to restrict growth in credit to protect financial stability;
• when should company management expand to a new product/technology;
• how should companies prepare for yet unforeseen cyber-security threats;
• how should a government balance protection of its citizens from terrorist threats;
• what significant infrastructure should be built;
• what should government and industry do to adapt to climate change;
• what level of research, training and innovation spending to choose for maximum benefit of the UK economy;
• how to change the planning system to increase affordable housing;
• how to mitigate humanitarian crises;
• what levels of immigration to allow.

For any of these questions, a complicated mathematical model could be constructed which simulates outcomes given different inputs and assumptions. But however well the model is constructed, there will remain necessary questions about reliability, confidence and robustness, and always a scope for surprise, model failure, or incompleteness.

Quantitative research in decision-making seeks either to assess the “optimal” choice in agiven situation or to analyse or assess how individual and organizational performance could be improved. In fields such as engineering, management, economics, and psychology, normative approaches such as cost-benefit analysis and rational choice frameworks have dominated the study of individual decision making with optimality assumptions, where research into real-world decision-making in groups, organisations and societies has become seriously neglected. Increasingly this is becoming recognised, partly by the evident non- optimality of many high-profile events like the financial crisis of 2008.

Artificial intelligence (AI) and humans posing as AI are fragile under radical uncertainty because of the rigidity of optimal-choice frameworks and their inability to live with ambivalence and contested schemes of values. Exciting new developments treat decision- making not as a computational process conducted by machine-like algorithms but an activity conducted by intelligent human actors drawing on their full range of capacities including imagination, emotion and intuition. The CRUISSE approach aims to integrate both the internal processes involved in taking decisions (psychology, neuroscience, management science, microeconomics) and the evidence basis which contributes to the assessment of potential outcomes (quantitative science including modelling, future projection and lab and field experiments or descriptive experience).

We classify decision support tools such as scientific models, spreadsheets, or narratives as“imaginaries” or prosthetics, which are used to reduce the complexity of a situation and to extend the capability of the mind itself. Narratives, in particular (and we note that where quantitative models are used, they are generally supported by explanatory narratives) are important. They have the property of creating a shared cognitive and emotional experience which helps to overcome ambivalence (a cognitive and emotional state of being pulled both towards and away from commitment and investment at the same time) to generate the conviction to act. The resilience of that conviction is for study. Most decisions are social: made in groups that have cultures, preferences, ways of operating, preferred narratives, and ways of debating and resolving argument.

With this in mind, we identify quantitative strategies for the model-builder as well as the decision-maker. Assessing the sensitivity of models to the inputs and assumptions is a first step. Expert judgement (on the part of the model-maker) must then assess the link between the model and the real world. Instead of single-use optimisation vehicles, then, this paradigm sees models designed to be used as active exploratory tools by social groups of decision-makers, in conversation about values, outcomes, and insights.

Our framework does not reduce epistemic fallibility to a statement that all tools are useless or nothing can be done; rather it provides an environment in which fallible tools can be used to the limits of their capacity, but do not become the arbiters of a decision themselves.

It creates the conditions in which we can face ambivalence without paralysis. Given the use case, it also becomes clear that co-creation of these tools is essential: to develop, select and use decision support prosthetics requires detailed understanding of context. There is no one-size-fits-all tool or technique, since decision questions confronted with radical uncertainty are all unique – but the insights generated by the deeper conversation facilitated by CRUISSE have led to broad principles and a radical new research agenda for decision support.

---