Tinkering with the Nuts and Bolts of Electronic Surveillance
Three things amaze me most about the description, including the screen capture images, of the NSA data-mining software revealed by Edward Snowden to Glenn Greenwald, which you can read about in some detail in tonight’s story which I found linked at memeorandum. The most detailed description is of a search engine called XKeyscore.
First, the software looks pretty easy to use; indeed, it seems almost banal for someone of greater-than-average intelligence. XKeyscore seems driven by a fairly simple pull-down menu driven windowed interface, only one step up from something you could write yourself in Microsoft Access. And because the software is actually quite simple — a URL, IP address, e-mail address, or nearly anything else can quickly be cross-searched — it would seem that because of this, even information about who has accessed Deep Web locations, and from where, is available readily and quickly.
Second, the content of exchanged messages is reviewable, in most cases for at least 30 days after it has been made. I’d thought before this that metadata was the bulk of the information being mined, and that was a problematic enough notion. Moreover, there seems to be no way for this software to prevent the data mining data from capturing communications from one American citizen to another, particularly if the server routes through a non-US locus. Communications between American sites or American users and foreign sites or foreign users are stored in the same database as what are apparently foreign-to-foreign communications.
Third, the obtaining of a warrant, whether regular or FISA, appears relatively rare for the typical analyst. Oversight seems mostly internal, and mostly of the CYA variety:
Some searches conducted by NSA analysts are periodically reviewed by their supervisors within the NSA. “It’s very rare to be questioned on our searches,” Snowden told the Guardian in June, “and even when we are, it’s usually along the lines of: ‘let’s bulk up the justification’.”
In a letter this week to senator Ron Wyden, director of national intelligence James Clapper acknowledged that NSA analysts have exceeded even legal limits as interpreted by the NSA in domestic surveillance.
Acknowledging what he called “a number of compliance problems”, Clapper attributed them to “human error” or “highly sophisticated technology issues” rather than “bad faith”.
Given the apparent ease with which many searches by many analysts can be done daily, and the depth of information available with a few mouse clicks to drill down through the data, the potential for abuse is enormous.
The obvious oversight and cautionary restraint mechanism for this would be auditing the searches actually performed by the analysts, on both a random and an algorithmically targeted basis. To that point, the NSA issued a statement in its own defense, quoted by Greenwald:
Allegations of widespread, unchecked analyst access to NSA collection data are simply not true. Access to XKeyscore, as well as all of NSA’s analytic tools, is limited to only those personnel who require access for their assigned tasks … In addition, there are multiple technical, manual and supervisory checks and balances within the system to prevent deliberate misuse from occurring.
Every search by an NSA analyst is fully auditable, to ensure that they are proper and within the law.
“Auditable” does not mean “audited,” but in fact auditing appears to be actually occurring. That auditing, however, is internal to NSA, with little if any judicial oversight and based on Greenwald’s article, information given to Congressional overseers comes with miserly reluctance and liberal use of coy generalizations, and even that only after the actual reviews of actual data have been put through a process of post facto rationalizations.
We can either believe Director Clapper or not when he defends the good faith and professionalism of the people who work for the NSA and have access to this information. Such charity of belief is beside the point, though — what makes the American constitutional system work is various branches of government exerting checks and balances against one another, and that appears to be absent from this process. The typical check against a governmental (that is to say, executive) search of private information is the advance requirement of a search warrant — the executive must justify itself to the judicial branch before it conducts a search. That’s not happening, and that’s a problem.
Here’s my idea, then. Just an idea, not one I’m 100% sure I endorse yet, but I am warm to it right now as I write. In comments, feel free to workshop changes to my proposed process.
Bear in mind that we are talking about privacy and reasonable restraints to prevent overreaching and abuse of governmental power. The Fourth and Fifth Amendments are significant concerns, but they aren’t the exclusive ones. So whether particular information would be admissible as evidence in court is not the end of the examination.
Congress is good at setting up rules about whether and how something is supposed to happen in the future. Courts are good at deciding if rules have been followed in the past. The FISA system sets up a mostly reasonable regime that could allow for nimble executive action provided that there is prompt post facto review of a search. And in order for a court to work best, it needs evidence and opposing advocates arguing before a neutral finder of fact independent of pressure or influence from either side.
NSA’s searches must be performed on government-issued computers which track and record keystrokes, mouse movements, and screen images. This is simple enough technology; some of my employer clients use it already to monitor their own employees’ use of computers. The usage-capture data will record the working habits of all the NSA analysts, and randomly-sampled data will be sent to audit. Those audits will be done by a review board independent of the NSA, on a relatively constant basis.
That review board should consist of at least three auditors, designated by Congress, who hold the security clearances necessary to review the substantive searches done. The review board will look, on a daily basis, on sampled searches from NSA search engines, which may include anything up to a level of detail including the screen captures of the computers used by the analysts. That review board can issue three evaluations of the searches it reviews: “Justified,” “Questionable,” and “Problematic.” My guess is that most audits would result in unanimous findings that a particular search was justified; that is the result we would expect if we credit Director Clapper’s remarks about the professionalism of NSA analysts, remarks which we have no objective cause to doubt.
But there may be individual searches that do not produce such results. So, two votes for “Questionable” or one vote for “Problematic” will then refer the matter to the FISA court. In that body, a U.S. Attorney will argue in favor of the validity of the search and a public privacy advocate will argue against it. The FISA court’s rulings about the validity of the audited search would then be treated with substantially the same sort of criteria as would a warrant application under existing law — either approved or not, and apparently “not approved” searches include an opinion detailing why a particular dimension of the search is troublesome.
Analysts whose searches are found questionable or problematic too frequently, or worse, whose searches are found to have overstepped safeguards by the FISA court, will have escalating consequences; trends of the kinds of searches that give rise to problems will then inform further in-house training within the NSA. Over time, a body of laws and principles from accumulated decisions will grow and become familiar to the attorneys who practice in the surveillance system, either as public advocates or representatives of the government.
Greenwald’s article, based on Snowden’s information, is certainly scary, even if we assume that each and every person who has access to this software and the data that the software mines operates in 100% good faith all the time. It’s simply too much power to concentrate in one place without some kind of inter-branch check and balance. That’s what our system of government is all about. I hope my proposal would keep the NSA nimble enough to pursue information it needs to in order to stay on top of the bad guys; but I also hope that it would create a meaningful system by which the government’s power can be reined in according to Constitutional principles.
This is just a working proposal, and I’m not going to suffer a wound to my ego if some of you commenters come up with better ideas than this, or refine it, or poke big holes in it. All I ask is that you remember that our goal is to have both security and liberty — because as it stands, it looks to me like there isn’t enough liberty in the mix.
Burt Likko is the pseudonym of an attorney in Southern California. His interests include Constitutional law with a special interest in law relating to the concept of separation of church and state, cooking, good wine, and bad science fiction movies. Follow his sporadic Tweets at @burtlikko, and his Flipboard at Burt Likko.
I don’t like the NSA but I routinely use NSA’s SELinux code to secure my own systems and machinery. Some people don’t like SELinux and some badly architected applications become problem children when it’s enabled — because they want too much access.
Look, the problem is oversight. NSA is neither a good guy nor a bad guy in all this. They do what they’re allowed to do in the context of the applicable laws — and that’s the problem. When Clapper lied to Congress, he was lying to people who not only knew he was lying, they already had the facts of the matter, the very people who could change the laws which prevented him from telling the truth. Of course the NSA has been looking at phone records and email and damned near everything else you’ve been doing. Congress and the courts haven’t been doing their jobs, managing the NSA.
I have more problems with the credit reporting agencies than NSA. This will come as no surprise to any steady reader of Ordinary Times, I tend to repeat myself with irritating frequency. I have parsers for all the major credit reporting agency reports and some you haven’t heard of yet. Your information is being bought and sold, from that “customer loyalty” card you use at the grocery store to your health care information to your Google and Yahoo searches. It’s all out there. Do you think privacy laws are enforced in an era where the bureaucrats who might actually enforce them are being funded out of existence? Information is power and power is money and one can be traded for the other in the frictionless world of fibre optic cables and high speed routers.
You’re so easy to find and it’s so hard to hide, it’s actually more trouble than it’s worth to encrypt much any more. If I was a serious crook, I wouldn’t bother trying to hide my tracks. I’d start up a porn site and run my criminal enterprise using a master CA cert issued by any of a number of Highly Reputable Outfits, issuing my own sub-certs to all my criminal cronies. James Jesus Angleton, a man who would know, said if you’re trying to hide a leaf, put it on the floor of a forest.
Why a porn site? Lots of hits from all over the world, lots of credit card transactions and lots of bandwidth in use. A hurricane of leaves. Furthermore, the Do Gooders really don’t want to fish around in the cesspool thus created.
So Edward Snowden is hiding out in a Russian airport. Lots of crooks are hiding out in Russia and the Ukraine and Belarus and other such shitholes and dens of iniquity. In a well-run nation, such people would be detected and run to ground, using techniques very much akin to what we don’t like about how the NSA is doing its job now. Probably using the same tools. You can use them too, if you know how to configure and run tools like Aircrack and Wireshark.
NSA are not the enemy you ought to fear. NSA are jealously guarding their information, silly people that they are. They’re bureaucrats, their instinct is to stash this stuff and not to share it. Crooks, now, they do share information. They sell it. And so do these credit rating agencies, who I consider to be worse than the garden variety Nigerian/Russian credit card thief. Nobody’s saying a damn thing about them and their abuses of power. Rupert Murdoch’s turds hack into famous people’s voice mails — do you think Fox is alone in so doing?
The FISA court is asleep at the switch and Congress has done nothing to regulate them. SCOTUS won’t act: every time some Fourth Amendment case comes before it, it’s always come down on the side of the State and that goes back for many years. We’d like to think it’s still 1967 and Katz v. United States says the government has to obtain a warrant to do a wire tap. It’s not. Since 1979, Smith v. Maryland, 442 U.S. 735 has governed and NSA is doing nothing more than what Smith established way back then. The PATRIOT Act allows far more, of course: while it remains in effect, the Fourth Amendment is dead for all practical and legal purposes.
But the part which ought to trouble people — and seemingly doesn’t, outside of a small-ish community of technical people, is the threat posed by all those magcards in our pockets and the practically unregulated trade in information. NSA are not the bad guys in all this. The NSA are a bunch of big dogs whose owners can’t or won’t control them. Don’t blame the dog under such circumstances. That dog is loyal in a world without much of that commodity. They are at war with totally unscrupulous enemies. Beating the dog will not fix what’s wrong with his master.Report
Excellent comment, BlaiseP. I’d add that there’s a delicious irony in the outrage expressed by telcos and Silicon Valley over “forced” compliance in providing metadata to the NSA. Verizon uses the exact same data for its own commercial ends, and sells it to companies for marketing purposes. Google’s entire business model is based on the collection/filtering of customer data for targeted advertising. The corporate squawking about NSA strikes me as a classic case “the pot calling the kettle black.”
Hadn’t even stopped to think about credit rating agencies. You’re dead right on that.Report
“Look, the problem is oversight. NSA is neither a good guy nor a bad guy in all this. They do what they’re allowed to do in the context of the applicable laws — and that’s the problem. When Clapper lied to Congress, he was lying to people who not only knew he was lying, they already had the facts of the matter, the very people who could change the laws which prevented him from telling the truth. Of course the NSA has been looking at phone records and email and damned near everything else you’ve been doing. Congress and the courts haven’t been doing their jobs, managing the NSA.”
Bull f-ing sh*t. These people had verbal briefing they were barred by law from sharing, they had no way to confirm or rebut what they were being told, and (even if people were honest) they undoubtedly were getting highly redacted information.
As for abuses, you’ve pretty much torn up your right to question anybody, in the rest of government or in the private sector, because both of those groups have more transparency and face more consequences.Report
Shut up, Barry. And learn to construct an English sentence. I cannot work out the subject or object of anything you’ve written in this comment.Report
All that ability, & the NSA apparently can’t search it’s own email servers:
http://www.propublica.org/article/nsa-says-it-cant-search-own-emailsReport
This is a point in favor of what I said – even for unclassified things, the NSA faces no accountability.Report
It doesn’t face much accountability from anyone except a secret court and a Congress which conducts its oversight in secret. If those agencies rubber-stamp everything they do, we may also thank the highest court in the land which forged up that rubber stamp in the Smith decision.Report
And that is why I proposed as I did in the OP.Report
I wonder if merely suggesting a fix like this doesn’t grant the NSA too much rope. I’m still waiting to hear a compelling reason why government has the right to monitor private individual communications and meta-data as a matter of course. It seems to me that the mere fact that data is being collected by private parties and that it can be – and is – stored for a certain period of time doesn’t constitute an argument that government ought to be able to access that data in real time or review it without establishing probably cause or at least reasonable suspicion.
Of course, if your argument is that this shit is going to happen anyway so why not try to put a check on it, then I agree.Report
There are some foundational questions to consider.
Is what the NSA does with XKeyscore a ‘search’ at all? It seems obvious to me that it is, but maybe it isn’t quite as obvious as I think it is, or maybe if it is, it’s not a private person who is being ‘searched.’
Is there a reasonable expectation of privacy in one’s electronic communications? Again, it seems obvious to me that the answer is ‘yes,’ but if that turns out to be wrong, then strong criticism of interception and analysis of those communications may not be warranted.
Is it technologically possible to know that a communication is between two American citizens before that communication is intercepted? The answer here seems to be “no,” although I confess I am ignorant of the finer points of how this works. I know IP addresses can be traced back to specific geographic locations easily and that there are services that obscure the actual origination point of an IP address. Beyond that, I’m ignorant of almost anything other than what is superficially obvious on a technical level. But, if it is not possible, then a signal must be intercepted and analyzed before its content can really be known. The universe of things that can be done at all, without sacrificing signals intelligence altogether, is not so vast as we might wish it were.
Is signals intelligence necessary to preserve national security? I suspect that the answer is yes, although of necessity that is a religious rather than a scientific question for all but the most well-informed of analysts. I take it as a matter of faith that important security information is gathered this way. Having said that I want the government to find and kill those who would do me, my countrymen, and my country grievous harm, I must accept that the “find” part is going to involve a degree of “looking.”
And then, is the question of liberty and security, of privacy and effectiveness, necessarily a zero-sum game? The balance wire is suspended above a fall to the left of too little security or a fall to the right of too little freedom, and we ask a difficult thing of our public servant to not only keep their balance but actually move forwards while doing so.
That last point gets to yours most directly — there is going to be surveillance and interception of electronic communications. We’re going to abolish the NSA and we’re not going to stop using technology to try and find communication between the bad guys. So given that as a society we find this activity by the government to be indispensable, we must then conform that indispensable activity with our other principles, else collectively we fall off the high wire into the abyss. The falling might actually be pleasant, at least for a while — it’s when you hit the ground that it becomes unendurable to have made a mistake.Report