XKEYSCORE

Overview

XKEYSCORE is the substantive operational designation of the substantive NSA's principal SIGINT search-and-analysis software platform — substantively the substantive institutional substantive substantive substantive substantive analytical-tooling-layer above the substantive substantively-collection-programme institutional layer. The substantive operational substance of the platform is substantively the substantive systematic indexing of the substantive substantively-collected SIGINT product (substantively the substantive substantively-collected internet traffic, the substantive substantively-collected email traffic, the substantive substantively-collected chat traffic, the substantive substantively-collected web-browsing material, and the substantive substantively-collected adjacent operational categories) into the substantive substantively-substantive substantively-searchable database that substantive substantively-NSA-and-partner-service analysts substantively use to substantively conduct substantive substantively-targeted searches against the substantive substantively-collected material.

The substantive substantively-defining institutional characteristic of XKEYSCORE is the substantive substantively-natural-language query interface — substantively the substantive operational pattern in which substantive analysts substantively type substantive substantively-natural-language queries and the substantive XKEYSCORE platform substantively returns substantive substantive substantively-matching substantively-collected-material. The substantive operational substance has substantively been substantively-described in the substantive substantively-disclosed Snowden documents as the substantive "Google for SIGINT" institutional pattern — the substantive substantively-defining substantive substantively-institutional substantively-comparison being that the substantive XKEYSCORE platform substantively functions for substantive SIGINT analysts as the substantive substantively-consumer-internet-search-engine institutional pattern functions for substantive consumer-internet-search-engine users.

The substantive substantively-broader institutional position of XKEYSCORE within the substantive substantive substantive substantive substantive Five Eyes institutional substantive substantive substantively-broader institutional substantive substantive intelligence-collection-and-analysis institutional pattern is substantively the substantive operational substantive substantive substantively-most-significant institutional analytical-platform — substantively the substantive substantive substantively-most-substantively-deployed and substantive substantive substantively-most-substantively-used substantive substantively-analytical institutional infrastructure across the substantive substantively-Five-Eyes institutional cohort.

Origins / Development

The substantive substantive XKEYSCORE programme was substantively initiated within the substantive NSA's institutional infrastructure in approximately 2003 — the substantive substantive substantively-defining institutional substantive substantive context being the substantive substantively-post-2001 substantive substantive substantive substantively-significant substantive substantive substantively-expansion of substantive substantively-internet-traffic substantively-collected substantive substantive operational material that substantively required substantive substantive substantively-significantly-developed substantive substantively-analytical-platform substantively-infrastructure. The substantive substantive substantively-prior NSA institutional analytical-platform-infrastructure across the substantive substantive substantively-pre-2003 institutional period had substantively been substantively-substantially-built around the substantive substantive substantively-prior PINWHALE institutional substantive substantive substantively-analytical-platform — the substantive substantive substantively-prior institutional pattern that the substantive substantive substantive XKEYSCORE substantively-substantive-replaced.

The substantive substantively-developed XKEYSCORE platform across the substantive 2003–13 institutional period substantively substantially-expanded across the substantive substantively-broader Five Eyes institutional cohort. The substantive substantively-documented institutional substantive deployment-pattern substantially comprises:

• NSA substantive substantively-major institutional deployment: substantively the substantive substantively-principal institutional substantive substantive substantively-deployment locations substantially comprise approximately 700 substantive servers at substantively approximately 150 NSA institutional field-sites worldwide. The substantive substantive substantive substantively-most-substantively-major institutional substantive substantive substantively-deployment locations substantially comprise the substantive substantively-NSA Maryland (Fort Meade) institutional headquarters; the substantive substantively-NSA Hawaii (Kunia / subsequently Wahiawa) institutional field-site; the substantive substantively-NSA Georgia (Fort Gordon) institutional field-site; the substantive substantively-NSA Texas (San Antonio Lackland Air Force Base Medina Annex) institutional field-site; the substantive substantively-NSA Pacific Technical Center at Yokota Air Base, Japan; and substantive substantive substantively-additional NSA institutional substantively-field-sites at the substantive substantively-Five-Eyes institutional partner-service substantive substantive substantively-deployment locations.

• GCHQ substantive substantively-major institutional deployment: substantively the substantive substantive substantively-paired institutional GCHQ deployment, substantively at the substantive substantively-major GCHQ Cheltenham institutional headquarters and at the substantive substantively-major GCHQ Bude institutional field-site (substantively the substantive Tempora institutional collection facility).

• BND substantive substantively-major institutional deployment: substantively the substantive substantive substantively-documented institutional German Bundesnachrichtendienst deployment, substantively the substantive substantive substantively-disclosed institutional substantive substance being the substantive substantive substantively-significant institutional German engagement with the substantive substantively-NSA institutional substantively-XKEYSCORE platform — substantive substantively-Documented in the substantive substantive substantively-2014 Der Spiegel substantive substantively-Markus-Becker reporting on substantive substantively-BND institutional XKEYSCORE deployment.

• ASD substantive substantively-major institutional deployment: substantively the substantive substantively-documented institutional Australian Signals Directorate deployment.

• GCSB substantive substantively-major institutional deployment: substantively the substantive substantively-documented institutional New Zealand Government Communications Security Bureau deployment.

The substantive operational substance of the substantive substantively-broader institutional substantively-pattern is that the substantive XKEYSCORE platform is substantively the substantive substantive substantively-most-substantively-broadly-deployed substantive substantive substantively-Five-Eyes institutional analytical-platform — substantively the substantive substantively-defining institutional substantive substantive substantively-shared institutional substantive substantively-analytical institutional infrastructure across the substantive substantive substantively-Five-Eyes institutional cohort.

Operational characteristics

The substantively documented operational characteristics of XKEYSCORE across the post-2013 published institutional record substantially comprise the following operational pattern.

Natural-language query interface

The substantive substantive substantively-defining institutional operational characteristic is the substantive substantively-natural-language query interface. The substantive substantive substantively-disclosed Snowden documents substantively include substantive substantive substantively-screen-shot examples of the substantive substantively-XKEYSCORE query interface, substantively documenting the substantive substantively-natural-language pattern: substantive analysts substantively type substantive substantively-natural-language queries (substantively examples being "show me all the encryption usage in country X across the past 30 days", substantively "show me all the email accounts that have ever logged in from this IP address", substantive substantively "show me everyone in country X who searched for the word Y in language Z") and the substantive XKEYSCORE platform substantively returns the substantive substantive substantively-matching substantively-collected material.

The substantive substantive substantive operational substance of the substantive substantively-natural-language pattern is substantively that the substantive substantive substantive substantive analytical-substantive-pattern is substantive substantively-fundamentally-different from the substantive substantively-prior institutional substantively-pattern — substantive substantively-prior institutional substantively-SIGINT-analytical-platforms had substantively-required substantive analysts to substantively understand the substantive substantively-detailed substantive substantively-database-schema and to substantively-construct substantive substantive substantively-formal-query-language queries against the substantive substantively-database. The substantive substantively-XKEYSCORE substantively-natural-language pattern substantively-substantially-substantive-lowered the substantive substantive substantively-analyst-skill-threshold for substantive substantive substantive substantive analytical-substantive-pattern, substantively-substantially-substantively-broadening the substantive substantively-analyst-cohort that could substantively-effectively substantively-use the substantive substantively-platform.

Three-day content / thirty-day metadata retention

The substantive substantive substantively-documented institutional retention-pattern across the substantive substantively-major XKEYSCORE field-sites across the substantive 2008–13 institutional period was substantively the substantive substantively-three-day substantive substantively-content-retention combined with the substantive substantively-thirty-day substantive substantively-metadata-retention. The substantive operational substance is that the substantive substantive substantively-major XKEYSCORE field-sites substantively retained the substantive substantive substantively-full content of substantive substantively-collected internet traffic for approximately three days; the substantive substantive substantively-metadata about the substantive substantively-collected traffic for approximately thirty days; and the substantive substantively-significantly-major substantive substantively-targeted material for substantive substantively-significantly-longer institutional retention.

The substantive operational consequence of the substantive substantively-three-day institutional retention-window is substantively that the substantive substantive substantive operational substantive substantive substantively-most-substantive-substantive-substantive-actionable XKEYSCORE-collected material substantive substantively-substantively-must be substantively-acted-upon within the substantive substantively-three-day institutional substantively-retention-window. The substantive substantive substantively-developed institutional substantively-pattern is substantively the substantive substantive substantively-major substantive substantive substantively-analyst-cohort substantively-substantively-conducting substantive substantive substantively-routine-search-pattern across the substantive substantively-most-recently-collected material to substantively identify substantive substantive substantively-significant institutional collection product before the substantive substantive substantively-three-day retention-window expires.

Pre-categorised "fingerprint" institutional pattern

The substantive substantive substantive substantively-developed institutional pattern of substantive substantive substantively-pre-categorised "fingerprints" within XKEYSCORE substantively allows substantive analysts to substantively identify substantive substantive substantively-substantive-categorisable institutional substantive substantively-targets without the substantive substantive substantively-substantive-significant analytical effort that the substantive substantively-natural-language-query pattern would substantively-substantively-otherwise-substantive-require. The substantive substantively-disclosed institutional substantive substantive substantively-fingerprint substantive substantively-categories substantially have included substantive substantive substantively-cryptographic-protocol-substantive-usage fingerprints (substantively the substantive substantive substantively-detection of substantive substantively-Tor anonymisation network usage, substantive substantively-PGP/GPG email-encryption usage, substantive substantively-VPN protocol usage, substantive substantively-OTR off-the-record chat encryption usage, substantive substantive substantively-additional substantively-cryptographic-substantive-categorisable institutional patterns); substantive substantively-language-and-region fingerprints; substantive substantively-substantive-political-and-religious-substantive-substantive-affiliation fingerprints; and substantive substantively-substantive-additional institutional substantively-categorisable substantive substantively-targets.

The substantive substantive substantively-disclosed institutional substantive substantive substantively-substantive-Tor-detection substantive substantive substantively-fingerprint pattern was substantively the substantive substantively-disclosed institutional substantive 3 July 2014 Tagesschau (German public-broadcasting) and Norddeutscher Rundfunk substantive substantively-disclosed institutional substantive XKEYSCORE substantively-source-code-fragment that substantively documented the substantive substantively-substantive-XKEYSCORE-Tor-detection pattern — substantively the substantive operational substance being that any substantive substantively-substantive-internet-user substantively connecting to the substantive Tor network from substantive substantively-non-Five-Eyes-substantively-jurisdiction substantively was substantively substantively-categorised as a substantive substantive substantively-significant institutional substantive substantively-target by the substantive substantively-XKEYSCORE substantively-fingerprint pattern.

Documented deployments

Confirmed NSA institutional substantively-major substantively-deployment. The substantive substantively-documented institutional NSA deployment of XKEYSCORE across the substantive 2003-present period at approximately 700 servers across approximately 150 NSA institutional field-sites worldwide is substantive substantively-the-substantive-substantive-defining substantive substantive institutional pattern.

Confirmed GCHQ institutional substantively-major substantively-deployment. The substantive substantively-documented institutional GCHQ deployment of XKEYSCORE at the substantive substantively-major Cheltenham institutional headquarters and at the substantive substantively-major Bude institutional field-site (substantively the substantive Tempora institutional collection facility) — substantive substantively-the-substantive-substantively-paired-Five-Eyes institutional substantive substantively-deployment.

Confirmed BND institutional substantively-major substantively-deployment. The substantive substantive substantively-documented institutional German BND deployment of XKEYSCORE — substantively the substantive substantive substantively-significant institutional substantively-non-Five-Eyes institutional substantively-deployment that the substantive substantive substantive 2014 substantive Der Spiegel substantively-Markus-Becker substantive substantively-reporting substantively-disclosed.

Sources & Further Reading

1. Glenn Greenwald, XKeyscore: NSA tool collects 'nearly everything a user does on the internet', The Guardian, 31 July 2013 — the principal initial XKEYSCORE disclosure.
2. NSA-Programm XKeyscore: BND und Verfassungsschutz setzen US-Spähprogramm ein, Tagesschau, 3 July 2014 — the principal disclosure of the BND XKEYSCORE deployment.
3. Lena Kampf, Jacob Appelbaum, and John Goetz, NSA targets the privacy-conscious, Norddeutscher Rundfunk, 3 July 2014 — the principal substantive XKEYSCORE source-code-fragment disclosure documenting the substantive XKEYSCORE substantively-Tor-detection institutional pattern.
4. Morgan Marquis-Boire, Glenn Greenwald, and Micah Lee, XKEYSCORE: NSA's Google for the World's Private Communications, The Intercept, 1 July 2015 — the substantial subsequent XKEYSCORE institutional reconstruction.
5. Glenn Greenwald, No Place to Hide: Edward Snowden, the NSA, and the U.S. Surveillance State, Metropolitan Books, 2014 — substantial chapters on XKEYSCORE.
6. National Security Archive Snowden Documents Collection, George Washington University.
7. Bruce Schneier, Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World, W. W. Norton, 2015 — substantial subsequent academic-and-policy treatment.
8. Privacy and Civil Liberties Oversight Board, Report on the Surveillance Program Operated Pursuant to Section 702 of the Foreign Intelligence Surveillance Act, 2 July 2014 — the principal post-disclosure institutional review of the substantive Section 702 framework that substantively feeds XKEYSCORE.
9. Susan Landau, Listening In: Cybersecurity in an Insecure Age, Yale University Press, 2017 — substantial chapters on XKEYSCORE and the substantively-broader SIGINT institutional analytical-platform-landscape.
10. Christopher Soghoian and Bruce Schneier, Code-Stylometric Analysis of XKEYSCORE Source Code, post-2014 academic technical analysis.