If you have "no place to go," come here!

The Unit in My Pants Exploded Prematurely!

okanogen's picture

My partner was very disappointed, but after reloading we will try again in a half hour.

That model won't train.

Not without "the best educated guesses of smart human analysts".

Let me be blunt: the government's "predictive" surveillance program will never uncover or stop a single domestic terrorism event, EVER. If they ever say it has, they are LYING. Even dumb luck isn't lucky enough for that to happen. It will have either been some other detection or surveillance, or some prior knowledge (a "prior", prior probablity distribution or Bayesian statistical inference, Bayesian terms of art for which the above quote is a buzzword) that if followed up by any other method would be as effective.

This is based on elementary Bayesian probabalistic analysis. There just aren't enough terrorism events, compared to all other events, to overcome false positives (like the one above) or train a predictive model based on the enormous related flow of electronic communication. Disregarding that using simple coded language ("we are going to do a job on him/it"), screws the model up almost as much as my typing right here "I am going to bomb a train today". Even by typing that sentence just now, I damaged any future predictive model by injecting a false positive into the data, which must be filtered out using a specialized algorithm for the purpose. That is how fragile it is.

So how do you describe to a lay person that this "deep data" mining project is positioned to be the next great boondoggle in Military Industrial Complex history (after the arms race, "Star Wars", etc.)?

Here is an analogy. The supposed current big data mining process (sifting and storing all message and call senders and receivers and trying to find terrorist groups) isn't analogous to looking for a needle in a haystack. That would be far too high a probablility. The true analogy would be trying to find 2-3 needles that we think might be in haystacks, by first finding all haystacks all over the world, and knowing that 20-30 of them have small, thin bits of metal in them, find the haystacks with long thin bits of metal. Did we forget to mention that the small, thin bits of metal can teleport from one haystack to another? Did we also forget to mention they are hidden inside pieces of hay? And that they are painted to look like needles? More importantly, we didn't mention that the needles might not be metal at all, they could be plastic, or wood, or bone.

The issue with this (I was going to say "problem" but corrected myself) is that the fact this will never work, is not in my opinion a bug, it is a feature. The people tasked with doing this project are smart enough to know that it is impossible. Which means, permanent employment! Like nearly everything else in American government, this policy is not driven by logic, or its effectiveness, it is driven by interest groups, and fear. So who is getting the green in this one? Again, like everything else in America, follow the money, and you will find the reason for the policy.


transcriber's picture
Submitted by transcriber on

Let me be blunt: the government's "predictive" surveillance program will never uncover or stop a single domestic terrorism event, EVER.

Where is your willing suspension of disbelief? Of course they're going to stop domestic terrorism events, again and again and...

okanogen's picture
Submitted by okanogen on

That I hadn't read Arthur before posting this. His conclusion is the same, but based on his own research of business news:

They're just getting started. Note that the $219 million is net income. Earlier, the Times told us that "more than half its $5.8 billion in annual revenue [is] coming from the military and the intelligence agencies." The story also informs us that: "Booz Allen is one of many companies that make up the digital spine of the intelligence world, designing the software and hardware systems on which the N.S.A. and other military and intelligence agencies depend."

Anybody commenting on this (from either side) parroting their propoganda that it even CAN work, is helping them maintain their revenue stream.

People, this is a boondoggle, that CAN NOT work the way they claim, so it MUST have other purposes.