Mountains and Molehills: Why Data Mining Makes Sense

Mountains and Molehills:  Why Data Mining Makes Sense

Twice in the last ten years our credit card company asked if we had made purchases flagged by their software as deviations from our usual patterns.  Both times they were right; we had made no such transactions. Visa  promptly gave us a new account number. I was delighted with their data-mining , even though it "snooped" on my transaction patterns. What skin off my teeth was that "invasion of privacy"?

After 9-11 the F.B.I.,  which had noticed some of the unusual activity by the people planning the attack,  was criticized for failing to “connect the dots.”   If those dots had been connected federal authorities might have seen the pattern suggested by the connections and headed off the 9-11 disaster.  Critics of the recently revealed data-mining operations forget  that it is impossible to connect dots that have not been collected in the first place. 

In the present case the dots are individual calls from one telephone number to another.   Federal agencies have been collecting records of all these calls.  Responding to this,  the New York Times recently editorialized that Congress should enact  “legislation to limit the collection of call records and the monitoring of Internet traffic to that of people suspected of terrorism, ending the mass warehousing of everyone’s data.”     

This editorial misses the whole concept of data mining,  in which computers scan immense amounts of data (like which phone number phones which phone number, when, and for how long) and pick up patterns which suggest activity  meriting further investigation. Limiting collection of this information to calls associated with people already under suspicion would make it much less likely to detect people who are not under suspicion but ought to be.

Remember that the data about phone calls being swept up by government agencies does not include what people are saying in those calls.  After patterns have been detected,  authorities may place wiretaps on specific people,  but only after getting specific authorization by a court.  

Of course this data mining harms would-be terrorists,  but aside from that what harm does the so-called invasion of people’s privacy cause anybody?  What difference does it make in our lives?

“Big data” is a recent phenomenon made possible by modern computers,  which can scan immense amounts of information and detect patterns which could never be found by finite human investigators. Data mining techniques are already being used by astronomers,  traffic control people,  medical researchers, and in many other fields.  They are improving our ability to understand the universe,   make traffic flow more smoothly, and treat diseases.  There is no reason why we should not also exploit this technology
to improve the security of our people. 

The private groups which have declared war against the United States do not hesitate to use modern technology (cell phones,  the internet, explosives, etc.) to further their plans.  There is no sense in placing artificial limits on our own ability to use technology to limit the damage they can do.

In older wars people were conscripted, shipped off to fight, placed under wage controls, taxed more, and endured rationing. Surely the minimal "invasions of privacy"  caused by the programs recently revealed pale in comparison.

**************

This piece has run in the Oregonian and the (Adrian, Michigan) Daily Telegram.