Documenting Uncertainty

A4.1 Introduction

The authors have assessed a wide range of information in the scientific literature and various technical reports to arrive at their Key Findings. Similar to the 2014 NCA¹ and the Intergovernmental Panel on Climate Change’s Fifth Assessment Report,² this assessment relies on two metrics to communicate the degree of certainty in Key Findings:

• Confidence in the validity of a finding based on the type, amount, quality, strength, and consistency of evidence (such as mechanistic understanding, theory, data, models, and expert judgment); the skill, range, and consistency of model projections; and the degree of agreement within the body of literature.
• Likelihood, or probability of an effect or impact occurring, is based on measures of uncertainty expressed probabilistically (in other words, based on statistical analysis of observations or model results or on the authors’ expert judgment).

Key sources of information used to develop these characterizations are referenced in the Supporting Evidence section found at the end of each chapter. The Supporting Evidence sections include “Traceable Accounts” for each Key Finding that 1) document the process and rationale the authors used in reaching the conclusions in their Key Finding, 2) provide additional information to readers about the quality of the information used, 3) allow traceability to resources and data, and 4) describe the level of likelihood and confidence in the Key Finding. Thus, the Traceable Accounts represent a synthesis of the chapter author team’s judgment of the validity of findings, as determined through evaluation of evidence and agreement in the scientific literature. The Traceable Accounts also identify areas where data are limited or emerging or where scientific uncertainty limits the authors’ ability to estimate future climate change impacts. Each Traceable Account includes 1) a description of the evidence base, 2) major uncertainties, and 3) an assessment of confidence based on evidence.

A4.2 Evaluation of Confidence in the Validity of a Finding

Assessments of confidence in the Key Findings are based on the expert judgment of the chapter authors. Authors provide supporting evidence for each of the chapter’s Key Findings in the Traceable Accounts. Confidence is expressed qualitatively and ranges from low confidence (inconclusive evidence or disagreement among experts) to very high confidence (strong evidence and high consensus) (see below). Confidence levels are reported even where confidence is low. Confidence should not be interpreted probabilistically, as it is distinct from statistical likelihood.

A4.3 Evaluation of Likelihood of Risk

For the purposes of this assessment, likelihood is the chance of occurrence of an effect or impact based on measures of uncertainty expressed probabilistically (in other words, based on statistical analysis of observations or model results or on expert judgment). Authors came to a consensus using expert judgment, based on the synthesis of the literature assessed, to arrive at an estimation of the likelihood that a particular impact will occur within the range of possible outcomes. Where it is considered justified to report the likelihood of particular impacts within the range of possible outcomes, this report takes a plain-language approach to expressing the expert judgment of the chapter team, based on the best available evidence. For example, an outcome termed “likely” has at least a 66% chance of occurring; an outcome termed “very likely,” at least a 90% chance (see Figure 1).

A4.4 Uncertainty Language in Key Findings

All Key Findings include a description of confidence. Where it is considered scientifically justified to report the likelihood of particular impacts within the range of possible outcomes, Key Findings also include a likelihood designation. Where possible, levels of confidence and likelihood are provided for different steps along the exposure pathway to enable separate reporting of levels of uncertainty in understanding climate impacts, changes in exposure, the role of moderating or exacerbating factors, and observed or projected health outcomes.

Confidence and likelihood levels are based on the expert assessment and consensus of the chapter author teams. These teams determined the appropriate level of confidence or likelihood by assessing the available literature, determining the quality and quantity of available evidence, and evaluating the level of agreement across different studies. Often, the underlying studies provided their own estimates of uncertainty and confidence intervals. When available, these confidence intervals were assessed by the chapter authors in making their own expert judgments. For specific descriptions of the process by which each chapter author team came to consensus on the Key Findings and the assessment of confidence and likelihood, see the Traceable Accounts in the Supporting Evidence section of each chapter.