Why quantify risks? Good and bad reasons for quantifying risks

The practice of quantifying risks is perhaps almost as old as the deliberate taking of risks for financial gain.

Img_20160916

Small margins in business, finance and insurance encourage precise calculation of risks. One of the best-known definitions of risk in the world of finance is “measurable uncertainty” 1. A modern book on risk 2 defines it in explicitly quantitative terms as “uncertainty associated with an event that can be quantified on the basis of empirical observations or causal knowledge”. 

Modern risk assessment practice is able to quantify risks in immense detail. Reports of quantitative risk assessment (QRA) studies are often thick with data analyses, fault trees and consequence models. This comes at a substantial cost. So it is worth asking, before embarking on such a study, “Why are we doing this?” 

Regulatory compliance
One simplistic answer might be “because we have to”. For a few countries and hazards, regulators have been so convinced of the benefits of QRA that they have made it compulsory. Nevertheless, this still leaves the question, “Why did they do that?” 

After all, risk assessment does not have to be quantitative. The current international standard on risk 3 makes clear that risk analysis “can be qualitative, semi-quantitative or quantitative, or a combination of these, depending on the circumstances”. Outside the world of finance, much if not most risk assessment today is qualitative. So why, if risks can be managed qualitatively, would anyone want to go to the extra effort of quantifying them? 

Lord Cullen, whose inquiry into the lessons of the Piper Alpha disaster 4 resulted in a requirement for QRA of offshore installations in the UK, endorsed the view that “QRA is only one input to the decision-making process, though an important one”. Yet he concluded that QRA “gives a better understanding of the hazards and the measures needed to control them”. 

Still one might ask: “How does it do that?” In the following paragraphs, I consider several arguments that could be made in response – some more valid than others. 

Precision
One potential advantage of quantifying risks might be the greater precision that it offers. Qualitative expressions of risk are typically rather vague and subjective, whereas quantitative risks appear to be precise and objective. 

In reality, contrary to the impression sometimes given by authors of QRA studies, quantified risks are often highly uncertain and sensitive to many choices that are rooted in the judgement of the analyst. 

In the chemical process field, comparisons of QRAs of the same installation by different teams have revealed widely varying results. While models of the consequences of chemical releases can be precise and well-validated, much greater uncertainty is introduced by the attempt to combine with release frequencies in order to quantify the overall risks. In every discipline, risk estimates are uncertain. Hence, in my opinion, the precision of QRA is not sufficient to justify the approach. 

Unimaginable risks
QRA may be beneficial in the analysis of very rare but potentially catastrophic events. These are particularly difficult to assess qualitatively. Most of us have no experience of them, and our imaginations are affected by the biases of fear and dread on one hand, or unwarranted optimism on the other. These low-probability high-consequence events are to some extent what QRA was developed to address, and major accidents have often been the spur to wider adoption of the technique. Unfortunately, the uncertainties in QRA are greatest when accidents are infrequent. This requires the analysis to rely on theoretical models or quantified expert judgement. As a result, I think the advantages of QRA in assessing such hazards are less than overwhelming. 

Unacceptable risks
Many attempts to define the limits of acceptability for risks have been expressed in quantitative terms. For example, acceptance criteria may be based on the background risk of death from all causes, or the rate of accidents in comparable activities. Such acceptance criteria inevitably require QRA to show whether an activity complies with them. This might be considered a case of “the tail wagging the dog”, but it is wise to consider how a decision will be concluded before starting an analysis of it. 

Similarly, many assessments of the need for risk reduction measures compare their costs to the benefits that they provide, in order to reduce risks cost-effectively. Such comparisons of risks and costs are very difficult to perform qualitatively. Hence QRA is usually an essential input to any cost-benefit analysis of a risk reduction measure. 

In finance, risks may be expressed as net present values, insurance premiums, price volatilities or even amounts at risk. For these, quantification is so intrinsic that it becomes part of the definition of risk. 

Combinations of risks
Subjective judgement is a very powerful method of assessing individual scenarios, perhaps because humans are good at constructing stories to make the world meaningful. Quantitative analysis, by contrast, is good at adding things up. One sign that QRA might be needed is therefore if we want to combine risks from different hazards into an overall measure of risk. This is a task at which qualitative judgement typically struggles. One reason why QRA is appropriate offshore is that hazards such as fire, explosion, structural failure and helicopter transport all need to be added together in order to see the overall picture. 

QRA can also help when there are multiple safeguards preventing an accident from occurring. This type of complexity suits QRA more than qualitative assessment, although the results are very uncertain in such cases. 

Dynamic risks
Subjective judgement is best suited to assessing risks that are relatively constant and homogenous. To assess a risk that is changing, whether through some trend or a dynamic response to changing or local conditions, we need some form of predictive model. Such models are invariably quantitative. However, they are also very uncertain. Some argue that the failure to predict the financial crash of 2008 resulted from over-confidence in quantitative models of risk 5. Nevertheless, in this area there are few alternatives to quantification. 

Conflict resolution
In a very few cases, QRA can resolve conflicts that can arise in qualitative risk assessments, through different parties having different understanding of the significance of unlikely events. Merely stating the proportion of cases in which accidents have happened can sometimes cut through long-running discussions. On the other hand, QRAs themselves provide plenty to argue about. 

Trivial risks
Paradoxically, QRA can be effective at analysing very small risks. By making simple and pessimistic assumptions, it can sometimes demonstrate that the risks are low. In these cases, a very simple QRA can limit the scope of safety activities and show that a more detailed study is not needed. 

Belt and braces
The case for QRA does not rely on any proof that it is superior to qualitative assessment. In reality, the two can co-exist very well. Qualitative judgement is a very efficient way to assess a wide range of hazards and identify the most critical issues, while QRA is able to combine accident data and theoretical models with expert judgement to analyse these critical hazards in depth. A well-specified risk assessment therefore typically starts with a qualitative assessment of the risks (sometimes called hazard identification) before undertaking a QRA of the most critical issues. Ideally, the two approaches should inform each other. Different conclusions, at least during the analysis, may be considered a healthy mutual challenge, providing it leads ultimately to agreement on the main results, or at least a shared understanding of uncertainties. 

Because we can
In fields where data is plentiful, it might be considered negligent not to attempt at least some reckoning of the numbers involved. This might be the fundamental reason why financial risks are routinely quantified, whereas other business risks are normally not. In the UK at least, where employers are required to do all that is “reasonably practicable” to ensure safety, this might seem to require QRA, providing its cost is commensurate with the potential benefit. 

Cost-effectiveness
This brings me to one key reason why QRA is undertaken today in the absence of any compulsion: because it is expected to pay for itself. The cost of a QRA may be large, but the cost of an accident (or even the cost of an ineffective safety measure) may be so much larger that a QRA becomes a cost-effective business strategy. Conversely, this is sufficient to explain many cases where QRA is not undertaken: the risks of making the wrong decision are just not large enough to make QRA worthwhile. 

Personal preference
The cost-effectiveness argument has a logical catch: it implies that the decision whether to adopt QRA requires a knowledge of the risks, which inevitably returns to the question, “Quantitative, or not?” This is normally resolved by judgement, which is really a matter of personal or corporate preference. Decision-makers who feel comfortable with or helped by QRAs typically commission them, while others who are confused or exhausted by them typically do not. This may not be a very helpful answer to the question, but it explains a lot.

Discipline
In my opinion, the main advantage of quantification is not the quantification of the risks themselves but the greater degree of rigour that is required to produce them. Quantifying risks requires systematic consideration of the causes and consequences of each type of event, giving each a numerical weight. This process often delivers a remarkable insight into the risk. It also makes the results open to scrutiny and reconsideration. Qualitative risk assessment raises the same issues but leaves them relatively vague. In that vagueness, important issues can be lost or fudged. Lord Cullen 6 quoted the HSE’s view as being that, “Quantification, or in some cases just the attempt to quantify, imposes a discipline beneficial for safety”. 

It is perhaps a paradox that the main benefit of quantifying risks should ultimately be a better qualitative understanding of the risks. The numbers, in the end, may not be very robust or believable, but the effort of trying to produce them does deliver a greater insight.

  1. Knight, F. (1921), “Risk, Uncertainty and Profit”, Boston: Houghton Mifflin.
  2. Gigerenzer, G. (2002), “Reckoning with Risk”, London: Penguin Books.
  3. International Organization for Standardization (2009), “Risk Management – Principles and Guidelines”, ISO 31000
  4. The Hon. Lord Cullen (1990), “The Public Inquiry into the Piper Alpha Disaster”, London: HMSO, p284.
  5. Silver, N. (2012), “The Signal and the Noise”, New York: Penguin Group.
  6. Cullen, op cit p 285.

11/17/2016 8:00:00 AM