Lexicon

Walk-back analysis

The retrospective reconstruction of an intelligence failure or success to identify the points at which different decisions could have produced a different outcome

Walk-back analysis, in intelligence-service usage, is the retrospective analytical reconstruction of an intelligence failure or success — the deliberate working backward from an eventual outcome to identify the points at which different collection, analysis, dissemination, or decision-making could have produced a different outcome. The methodology is the principal institutional vehicle through which intelligence services attempt to learn from their record. The reconstruction is normally conducted as an internal institutional review (the post-event "lessons learned" process), as an external commission inquiry (the Jeremiah Commission on the 1998 Indian nuclear-test surprise; the Silberman-Robb Commission on the 2002 Iraq WMD National Intelligence Estimate; the 9/11 Commission on the September 2001 attacks), or as the published academic-and-policy reconstruction that the declassified institutional record subsequently supports.

The analytical methodology of walk-back analysis is distinctive in the intelligence cycle literature. A conventional intelligence assessment operates forward from the available collection material to a present judgment under uncertainty; a walk-back analysis operates backward from a known outcome through the chain of decisions and information availability that preceded it, identifying both the points at which the known outcome was knowable from the available material (the "missed indicators") and the points at which the known outcome was not knowable from the material (the "absence of warning") and the points at which the available material was misinterpreted (the "analytical failure" proper). The methodology's principal analytical risk is hindsight bias — the tendency to identify as missed indicators in retrospect material that, in the prospective collection environment of the time, was substantially less salient than the post-event reconstruction makes it.

The institutional purposes walk-back analysis serves are substantial. The internal-review purpose is the institutional learning function: the service identifies what it did wrong, what it did right, and what institutional changes are required to improve the next analytical cycle on a comparable subject. The external-commission purpose is the institutional accountability function: the political-and-policy environment outside the service holds the service to account for the failure and produces the political conditions under which institutional reform can be effected. The academic-and-policy purpose is the broader institutional learning function: the published record on intelligence failures becomes the institutional basis for the academic literature on intelligence and the institutional understanding under which subsequent intelligence services can be designed.

The principal methodological criticism of walk-back analysis is that it tends to identify pathological patterns in the analytical record (groupthink, mirror-imaging, layering of unsupported assumptions, the conflation of estimative confidence with collection confidence) that are also identifiable in successful analytical episodes — meaning that the criticism produces an analytical failure on retrospective examination of the same material rather than a learning insight that would have been operationally available at the time. The published methodological literature on walk-back analysis (the work of Richards Heuer at the CIA's Sherman Kent School, the post-2005 institutional development of structured-analytic-techniques training under the Office of the Director of National Intelligence) has produced substantial subsequent guidance on conducting walk-back analysis in ways that produce institutionally useful insights rather than retrospective indictments of decisions that would have been operationally defensible at the time.

See also