The NSA's Skynet Is Sounding More and More Like the Skynet of the 'Terminator' Franchise
Skynet, the fictional malevolent artificial intelligence made famous by the Terminator franchise, had one overriding purpose: Wipe out humankind.
The real-life Skynet, run by the National Security Agency, has a slightly less extensive mandate than exterminating organic civilization. Instead, it sifts through huge troves of communications metadata to identify presumable terrorist suspects. Leaked documents published by the Intercept in 2015 show the NSA ran one Skynet trial involving 55 million cellphone records in Pakistan, where a U.S. drone bombing campaign has been raging for years.
Read More: The CIA Just Published Its Own 'X-Files,' Reminding Us That They Did It First
In short, the real-life Skynet is likely involved in identifying people to kill with flying robots. But according to a new analysis on Ars Technica U.K., the statistical methods the system uses virtually guarantee it is generating thousands of false leads, potentially exposing innocent people to Terminator-style liquidation.
Skynet uses the 55 million cellphone records from Pakistan as its data set, isolating 80 variables for each individual record. From there, its classification algorithm generates a "score" for each individual fed into the system. According to Ars Technica U.K., Skynet uses a technique known as a "random forest algorithm," which "uses random subsets of the training data to create a 'forest' of decision 'trees,' and then combines those by averaging the predictions from the individual trees."
From there, the system sets a 50% false negative rate, effectively eliminating half of the suspected terrorists.
But Ars Technica reports data scientists are skeptical of the way the NSA "trains the algorithm with ground truths" — the agency taught the system how to identify terrorists from subsets including 100,000 civilian records and seven from known terrorists.
"First, there are very few 'known terrorists' to use to train and test the model," Human Rights Data Analysis Group executive director Patrick Ball told the site. "If they are using the same records to train the model as they are using to test the model, their assessment of the fit is completely bullshit. The usual practice is to hold some of the data out of the training process so that the test includes records the model has never seen before. Without this step, their classification fit assessment is ridiculously optimistic."
University College London security professor George Danezis told the site NSA records indicate the supposedly low false positive rate of 0.18% would still classify thousands of innocent people in Pakistan as terrorists. As the Intercept previously reported, one of those likely innocents was a reporter at Al-Jazeera.
From there, what happens is unknown, including whether Skynet-generated results are monitored by a human or simply fed directly into drone kill lists.
"It's bad science, that's for damn sure, because classification is inherently probabilistic," Ball told Ars Technica U.K. "If you're going to condemn someone to death, usually we have a 'beyond a reasonable doubt' standard, which is not at all the case when you're talking about people with 'probable terrorist' scores anywhere near the threshold. And that's assuming that the classifier works in the first place ..."
What is known, however, is that the U.S. drone campaign in Pakistan has killed somewhere between 2,494 and 3,994 people and injured up to 1,738 others since 2003, according to the London-based Bureau of Investigative Journalism.
