What is Infrastructure? The Colonial Pipeline Cyber Attack Tells Us
Well, here we are. A cyber attack (in this case ransom ware) was used to cripple the Colonial Pipeline. Quite possibly the attack originating overseas in a nation-state sponsored laboratory, if such environments can be equated to weapons development. Two weeks ago, I could have asked a hundred people if they could describe the Colonial Pipeline and wouldn’t have received even one knowledgeable response. Today, everyone east of the Mississippi River can likely draw a relatively accurate representation of its route on a map. Everyone is suddenly aware of the importance the Colonial Pipeline plays in the distribution of petroleum products up and down the East Coast. At this writing, citizens are rushing their cars, trucks, SUVs, motorcycles, all-terrain vehicles, lawn mowers, and surplus gas cans to stations looking to fill up before the supplies at the consumer point of sale dry up, so to speak.
This is what true national infrastructure looks like. This is not the poll-tested, happy face definition of infrastructure that has played a central role in the last two presidential elections. That version of infrastructure is presumed by voters to mean multiple billions of tax dollars that will ultimately affect their quality of life by finally repairing that perennial pothole on Walnut Street while providing a smiling green-energy government employee (a former coal miner) to install solar panels on their roof.
We should all be aware by now of the nature of poll-tested words and phrases in attempting to shape opinions and attract voters. Phrases such as common sense gun control, a living wage, and, of course, infrastructure investment have all been determined to garner positive responses among a large bloc of voters. Infrastructure investment conjures up thoughts of new, leafy bicycle paths, soaring suspension bridges, and armies of green energy workers recruited from the ranks of the under- and unemployed.
The Trump administration was rightly pilloried for not putting emphasis on infrastructure (and specifically cybersecurity) during his time in office. Trump sacked his Director of DHS’ Cybersecurity and Infrastructure Security Agency, Christopher Krebs, after the election. Mr. Krebs then took to the media and social media to lambaste the administration over accusations of tampering and malfeasance in the 2020 election results. In the wake of this kerfuffle, I would hope we could reprise Mr. Krebs to fill us in on other keys aspects of his oversight at DHS — infrastructure protection of the sixteen critical sectors. Let’ focus on energy, shall we?
The Biden administration touted infrastructure investment as a priority only to immediately cancel the Keystone XL Pipeline project – the Midwestern version of Colonial and a key infrastructure improvement. Apparently, his ardent supporters believe thousands of smoke-belching diesel trucks and gas and coal-burning trains carrying petroleum along highways and train beds is somehow better for the environment than a pipeline winding its way through the pastoral landscapes of Oklahoma, Kansas, and Nebraska.
This could quickly evolve into a sidebar about the vexing fixation so many people have with trains. I get it — train travel can be fun and relaxing. I have ridden trains through several European and South American countries. I often used the Acela corridor between DC and NYC while working as a consultant, and I once rode a bullet train between Tokyo and Kyoto. For better or worse, an expansive train network simply isn’t a viable option for the United States in the 21st century. To avoid the requisite discussion of demographics, topography, and national economics, let’s just move on.
Nearly two decades ago, the nascent Department of Homeland Security established the critical infrastructure security realm — now consisting of sixteen sectors including energy which is focused primarily on protection of the distribution networks for electricity and gas as well as nuclear generation facilities. Since that time, much of the ersatz work that has gone into the protection of these sectors has been accomplished by the development of standards, guidelines, frameworks, checklists, lexicons, proposals, indices, glossaries, addenda, and the ubiquitous public/private partnerships. These partnerships are established by government bureaucrats who are getting paid to draft all these documents and private companies who assign additional duties to some employee so a contractor is on hand to pick up the bar tab at all these partnership meetings.
The zeal for cybersecurity partnerships for protecting our infrastructure expanded dramatically after the Stuxnet virus incident of 2007 – now a decade and a half ago. In the intervening years, what hath our documents and partnerships wrought? The CEO of the private entity gets a photo-op with a senior government wonk as they shake hands over a piece of paper destined for long-term storage along the lines of the ending of Raiders of the Lost Ark. With all these documents and partnerships, one could be forgiven for wondering how a relatively basic ransom ware attack in 2021 was so successful against such a vital national target as a major oil pipeline.
The administration has now decided to toss off fifteen years of effort when the press secretary was asked about whether a ransom had been paid, Jen Psaki said it was up to the “private company” to determine if they should pay the DarkSide crooks — probably with nation-state entity involvement.
It seems the attacker had a private/public partnership.
We deserve a much more effective one ourselves.
Well, this is well timed, given the discussion starting on another post.
So, was this a technology hack, or social engineering (i.e. DarkSide got someone inside to execute their ransomware for them, as opposed to DarkSide finding a security exploit through a firewall and gaining execution privileges on workstation)?Report
Many of us wondered the same thing about Russian troll farms interfering in the 2016 election, but we were poo-poo’ed because the Russians conveniently didn’t need to actually manipulate any votes to achieve their objectives.
That aside, most public-private partnerships at the federal level still focus on what the private sector can gain at minimal cost to tax payers. And in the cyber realm, we really are not grappling nationally with threats as they are. There’s also the inconvenient issue that the federal government can’t pay as well as Google or Amazon Web Services, so we can’t recruit the best and brightest to work for the public good. That in turn means that the government isn’t well positioned to push back on the private sector when said private sector assures us “they’ve got this.” We only get the call when they don’t.Report
I could be wrong, but it always seems to me that the problem with public-private is that the public never holds the private accountable. When the private drops the ball, the public doesn’t go in for it’s pound of flesh, but instead kinda shrugs it off, and spends $$ and time writing up documentation on how to avoid it in the future, and those docs wind up in a doc vault in an old mine, never to be implemented.
And when I say these things, I’m told private companies would never enter into such agreements if they were truly held accountable, even though they enter into such agreements with other private companies all the time.Report
You are not wrong.Report
Damnit Philip, I was hoping you were going to tell me I was wrong!Report
even in a true public-private partnership agreement there are few enforcement mechanisms and most are so arcanely written that the federal officials in question can’t actually figure out what to do. Its an area of federal acquisition law and policy that needs serious work.Report
I have lots of insider knowledge here. It’s rarely one or the other. It’s simply incompatible systems being asked to work together.
True story: I met with a federal agency once who was working on such an agreement where private companies would send consultants out for ten days each to provide security consulting to smaller government entities. These private companies were charging up to $500/hr for their best consultants. Guess the quality of consultant the government received….
Government had to dial back to 10 days of consulting for each engagement with these interns and bottom-rung consultants – barely enough time to locate the restrooms at the client site. The system never. ever worked as desired by the government.Report
Let me guess – Firm fixed price contract with escalation clauses that the vendor expected to be exercised when it failed to deliver? And I’m going to guess the agency in question didn’t bother to do the paperwork to get these folks flagged in the federal contracting systems? Or for that matter never actually read the contract to see if the billed rate would actually result in the desired expertise? As a fed who does environmental science and personal services contracting periodically, I am always amazed at people not actually reading the bid package and then acting surprised when we don’t get what we wanted.Report
Apparently it was Boob Phishing.
Report
Oh wow that is just excellent. I sent it to my cousin who is a CSO. Definitely going to give him a heart attack. I guess we all better update our phishing simulations for accuracy. Not exactly a fake e-mail from Bank of America.
Also apparently every gas station anywhere near me is completely dry. I thought we were supposed to be leaving all of this behind in 2020.Report
Yes well blame the people buying gas in trash bags for that. If people had stuck to their normal patterns we would have been fine. There’s storage at the end of each pipeline node and every gas station gets delivery based on orders which are based on use. No one would have run out if there hadn’t been panic buying.Report
Why blame them when I can blame the CANSas City Chiefs?Report
Our IT security group occasionally sends out real a$$hole phishing emails. They really look like emails from internal accounts, and all the links look legit, often only off by one character.
I mean, internally, it would help if we didn’t have 5000 different internal domain names, which makes it real easy to make a link look legit.
But CANSas? Seriously? I’d fire someone for that one.Report
Is this the kinda thing where anyone in the cybersecurity industry with half a brain could identify a solution to this that we could be confident in for the foreseeable future?
Or is this the kinda thing where the bad guys are always going to be ahead of the good guys so unless we want to invest TONS in managing that, we just need to prepare for inevitable situations like this?
If it is neither of those, which one is it closer to?Report
“Is this the kinda thing where anyone in the cybersecurity industry with half a brain could identify a solution to this that we could be confident in for the foreseeable future?”
Yes, but it’s expensive and would annoy users, and what are the odds that their company will suffer a problem before upper management cashed out their options and moved elsewhere?
I mean some of it you can’t close — social engineering attacks. You can just train as best you can, do white hat attacks, and fire the dumbest people — or at least ensure the only thing they can open access to isn’t a problem for company.
Other stuff, like building from the get-go with security in mind, keeping machines patched and up to date, running regular white-hat attacks or at least scanning your own password DB’s with simple scripts against stolen password databases and dictionary attacks, it’s all time consuming and costly and seems to offer no ROE while making employees complain about having to keep track of a keycard or token.Report
Seven months ago we moved to a new city. The city has a municipal power utility. Our city and three other municipal utilities are the owner/customers of a power authority that generates and purchases electricity (also sells excess at times) and operates a transmission network. In 1999 the power authority recognized that they were going to have to put up a lot more sensors and control gear to handle things in the future. They bit the bullet and strung their own fiber so the control data network is physically separate. That doesn’t make it immune to outside hacking totally, but someone has do something wrong at the physical layer to open it up.
The power authority also said to the owner/customers, “You know, we’re basically building fiber rings around your cities. Labor is much more expensive than the fiber itself. Perhaps it would be good if we put in big bundles of fiber and you could rent back the excess for whatever.” Three of the cities have 144-pair fiber rings; the smaller other, which has come damned close to being washed away or burned out in the last decade, has a 72-pair ring. The city my son lives in has finished their municipal high-speed internet build out. Mine is about half through. When they are done, I’ll be able to get symmetric 100 Mbps service for $48/month.Report
That’s what my neighborhood did when it was laid out. Fiber to every house.Report