Thursday Throughput: The Hope of a COVID-19 Vaccine


Michael Siegel

Michael Siegel is an astronomer living in Pennsylvania. He is on Twitter, blogs at his own site, and has written a novel.

Related Post Roulette

71 Responses

  1. Avatar Oscar Gordon says:

    ThTh7: It’s not that complicated, you just have to understand what a standing shock is. Predicting when they will form isn’t even that complicated.

    Predicting WHERE they will form can be… complicated. And designing your aero surfaces to cause them to form in a specific location will keep your computing cluster humming along for days, even weeks.Report

  2. Avatar Michael Cain says:

    ThTh5: Yeah, if we ever get software intelligence, it will almost certainly run on different principles than a human brain.Report

    • Avatar Jaybird in reply to Michael Cain says:

      Have you been watching people play with GP-3?

      We’re on the cusp of something *HUGE*.Report

      • Avatar Jaybird in reply to Jaybird says:

        Check out this thread:

        I am not scared. Not really.

        But I think that that’s because I’m dumb and that if I were smart I would be scared.Report

      • Avatar Michael Siegel in reply to Jaybird says:

        For some reason, it’s reminding me of the Hithhiker’s scene where the philosopher’s ask what’s the point of staying up half the night arguing whether there may (or may not) be a God if GP-3 gives you his bleeding phone number in the morning.Report

        • Avatar Jaybird in reply to Michael Siegel says:

          This ain’t GPT-3, but GPT-2. But this is a bot that has been trained to make religious statements.

          Just check it out and scroll down for a while.

          It’s weird and creepy.Report

          • Avatar DensityDuck in reply to Jaybird says:

            eh. I’m pretty sure that we were doing that kind of thing with Markov chainers back in the 90s; we’ve learned to write grammar rules expansive enough (and gotten processors fast enough) that the thing can chain on syntax rather than just vocabulary.

            Which, that’s interesting, but I don’t think it’s a “we’ve given birth to aye-eye” moment.Report

            • Avatar Jaybird in reply to DensityDuck says:

              I don’t think it’s aye-eye yet either.

              But I’m noticing that we don’t know how we work.

              And that the Turing Test wasn’t about the computer but about how easily humans see patterns.

              And back to not knowing how we, ourselves, work again.Report

      • Avatar Brent F in reply to Jaybird says:

        GPT-3 seems to me to be a much more sophisticated ELIZA program. Its built to look smart to humans, not be actually all that smart.Report

      • Avatar veronica d in reply to Jaybird says:

        I’ll go on record and say that GP-3 type technologies will quickly come up against enormous diminishing returns and will have very limited commercial application.

        People suggest GP-3 type systems will replace software engineers. So far I’ve seen them produce very short CSS fragments. That’s certainly interesting.

        But consider this: the software I work on is 600k lines of Common Lisp and 400k lines of C++. The software encodes rules and procedures from various airline consortiums, along with exceptions requested by various individual airlines, often given to us in human readable language. Moreover, the software represents, in code, the tax policies related to air travel for every nation on earth, specified in various language with teams of attorneys to help us translated them into usable form. However, various airlines and customers request exceptions to our interpretation of tax policy, so we encode those exceptions.

        However, all of that acts as constraints on the solutions we generate. In other words, we generate travel itineraries that follow those rules. We calculate taxes according to law. But the main purpose of our software is generative. Out of all possible travel itineraries, and all possible sets of fares to price those itineraries, we generate an optimal, but diverse, set of solutions. We do this very fast given the scale of the problem. This involves a heavy amount of “micro optimization,” which requires that engineers understand the internal structures we’ve hand crafted to represent the various intermediate data structures, along with choosing various places in the search process to “prune away” partial solutions in the process of building full solutions. These “pruned” partial solutions are rejected for either not matching the airline policies, or else being too expensive or inconvenient for travelers. This is a balancing act. If we prune too soon, we lose good solutions. If we prune too late, we experience “exponential explosion” and our software will run slow.

        I hope this gives you some sense of the scale of the problem. However, it probably doesn’t. The software is very complex.

        Note, I’m only talking about the application itself. When deployed, it runs on hundreds of thousands of individual servers, so we have an auto deployment framework, along with various testing and fault tolerance frameworks. Moreover, the airlines regularly send us data updates, which have to be pushed out to those server farms. That also is automated. This amounts to many 100 locs of software.

        Also we track real time seat availability based on revenue management calculations, performed in cooperation with airlines. Different airlines have different systems with different constraints. The volume of data that needs to be shared is enormous.

        GP-3 currently can generate short CSS fragments. I’m not worried they’ll replace me or anyone on my team.


        I think a lot of people don’t really have a visceral understanding of complexity. They imagine we could feed GP-3 the corpus of Terence Tao’s work, and then somehow GP 3 will generate new math theorems that are both correct and interesting. It won’t.

        It does generate convincing absurdist fiction or poorly reasoned political screeds.Report

        • Avatar Stillwater in reply to veronica d says:

          I’ll just say this, anecdotally: Deep Blue was over twenty years ago.Report

          • MuZero was released last year. Starting from scratch it derived the rules for chess, shogi, Go, and 57 Atari games. For Go, it then trained by playing against itself. Beats every other piece of software, and humans. One of the best humans says it’s like watching an alien play Go, the style is very different. Of course, the follow-up question I always ask goes something like, “Great! When will it be able to to write a book that teaches a human to play Go?”

            The description sounds like the version that only plays Go would fit in one shelf in a rack and consume about a kW of power. (The learning version is much bigger.) As I recall, the chess-playing version of Deep Blue was many racks and drew upwards of a MW.

            On an always contentious topic, I still think my wife and I (middle-ish Boomers) will get to have a self-driving car that will at least take us on errands before physical deterioration makes it too dangerous to let us drive ourselves.Report

            • Avatar Oscar Gordon in reply to Michael Cain says:

              And the human brain uses something like 20W a day.Report

            • Avatar veronica d in reply to Michael Cain says:

              Go is a very complex problem, compared to chess, for example. However, it has the following properties:

              1. All possible game states are known in advance.

              2. For any game state, all information is visible.

              3. It is possible to be 100% certain about the game state.

              4. The game is turn-based, rather than real time.

              5. There is always a finite number of possible moves.

              6. These properties also apply to the opponent, and the agent knows what options exist for the opponent.

              None of these hold true for the problem of self-driving cars. It’s a different sort of problem.

              I do believe we’ll have self driving cars sometime in the near-ish future. However, we should be careful in generalizing from success at Go and general, real-time, real-world applications of AI.Report

              • Indeed. Although MuZero does learn to play Atari games, some of them quick-twitch games, against real time. The company behind MuZero is focusing on StarCraft for all of the reasons you list. My intuition is that there will be more in common in the low-level plumbing — eg, hierarchies of “games” and guided search — than many people think.

                Apparently their StarCraft II software has achieved grandmaster status playing against top humans under tournament conditions. One of the humans made the same sort of remark as the Go player I mentioned, that the software isn’t playing the game the same way a human does.Report

              • Avatar Stillwater in reply to Michael Cain says:

                How do computers do against high stakes poker players?Report

              • There’s now research software that does well at Texas Hold ‘Em at a table with the software and five professional players. Granted I’m looking for certain approaches, but in the couple of papers I’ve read there’s hierarchies of games, repeated subgames, guided search… Also, the software “discovers” some of the tactics humans use, like bluffing, and some tactics that humans don’t.Report

              • Avatar Stillwater in reply to Michael Cain says:

                Thanks Michael. Fascinating stuff. So fascinating that I watched a video of Kasparov’s sixth game against Deep Blue in ’97. Dude obviously shouldn’t have left his knight at risk like that. Rookie mistake. I saw it a mile away. 🙂

                In all seriousness though, the video I watched explained pretty well the “mistakes’ Kasparov made, which, given the opponent, was pretty interesting.

                I know nothing about chess.Report

              • Avatar veronica d in reply to Michael Cain says:

                It’s definitely impressive. I recall when AlphaZero first hit the scene. I was like, WTF?

                Anyway, I read their early papers. It’s definitely humbling.

                I guess my basic criticism is this. It seems very common for people to see impressive results such as these, and from that to just assume this is a solid step on the path to AGI. I don’t believe that. If you pry open these algorithms, they are basically using deep learning techniques to solve the exponential complexity of general reinforcement learning. That’s really cool, but it remains a limited thing. Deep learning system can be tricked. Note, you cannot trick them in Starcraft, because the data stream is still a “video game”. Certain inputs will not happen. They can’t.Report

              • Sometimes I think AGI is “just” a big enough, deep enough hierarchy of game-like things. “Me” is a heuristic goal-setting system shaped by a billion years of ancestors that didn’t make fatally bad choices too soon, plus the ability to set up my own deep learning processes, plus the occasional random number*. I don’t expect I’ll live long enough to see us assemble that much computation and communication in silicon form. But there will be a lot of stuff that we can build and combine limited specialized subsystems for if we want to. A hand may not be the ideal gripping element in any specific situation, but it’s damned good as a general-purpose device. And once that control problem is solved well once, in the AI world it’s solved everywhere.

                * The other day I was going through all of the “random” things that had to happen to keep me from spending most of my life in California. From my Dad’s teeth that kept him out of Navy OCS to the order in which I lusted after two different brunette women.Report

              • Avatar Jaybird in reply to veronica d says:

                This flitted across my timeline today: A self-driving motorcycle.


              • Avatar veronica d in reply to Jaybird says:

                Cool. Now let’s see it navigate the streets of Boston on a really busy day.Report

              • Avatar veronica d in reply to veronica d says:

                Let me add, I don’t want to diminish the cool engineering that goes into things like this. However, notice it’s on a empty track with visually distinct borders. It’s not raining. There are no other cars changing lanes. There are no pedestrians darting across the street. No one has spaypainted a weird pattern over a stop sign to mess with their visual system.

                Someday someone is going to have a terrific time putting a red octagon next to a freeway during rush hour to observe the mayhem.Report

              • Avatar Jaybird in reply to veronica d says:

                Pre-reqs to things keep showing up, though.

                It’s not the thing.

                But it’s a pre-req to it.Report

              • It’s been years since I drove in downtown Boston and I still get the nervous twitches thinking about it. Which is part of the reason I think the initial killer app for self-driving cars is chauffeuring elderly Boomers around the suburbs.Report

        • Avatar Jaybird in reply to veronica d says:

          I’m talking about stuff like this:

          I mean, when I got my first “real” job in IT (that wasn’t just data entry), there was a team of four guys who walked around the building. They were the Webmaster Team. They were the Webmasters.

          We’re probably talking more than half-a-mil in take-home salary walking around (and these were *CLINTON* dollars).

          They did a lot of with the data backup team, the account management team, the print spool team, and the 3rd Tier sysadmin core support team (they ignored the 2nd Tier sysadmin support team for the most part).

          All of the people on all of these teams together probably made up more than 80 people.

          Now, I understand, the teams are consolidated and culled and it’s fewer than a dozen people. Most of the jobs that were done by teams in the late 90’s are done by scripts now. (And I don’t want to say that this is a new development, indeed, the jobs were done by scripts by the mid-oughts.)

          I’m sure that your job will be fine. I’m sure that you’d be one of the dozen kept around.

          But the 80 people turned into a dozen people in less than a decade.

          We’re going to see something like that happen again.Report

          • Avatar veronica d in reply to Jaybird says:

            If we’re talking about general ML as automation, then sure. However, I doubt the specific GP-3 technologies will play a central role.

            They succeed at generating short CSS fragments because those sorts of fragments show up in the corpus it learned from. They are the kinds of toy examples that people post to StackOverflow and various tutorial sites. It can’t scale past that level because it doesn’t really “understand” meaning.

            Moreover, people post the successes of the algorithm, because they stand out. They don’t post the failures.Report

            • Avatar Jaybird in reply to veronica d says:

              Here’s another fine, fine example of one of the successes.

              Perhaps it’s not representative.


              • Avatar veronica d in reply to Jaybird says:

                Be skeptical of video demonstrations. They show success, not failure.

                I suggest you shouldn’t believe anything you see about GP-3 that you cannot personally interact with. If they want to demonstrate its power, give us a website where we can type in custom text to judge its success.

                Also note, this is using GP-3 to respond to queries, not generate new things. The point, it doesn’t generate logically consistent new things past a very limited scale.Report

              • Avatar Jaybird in reply to veronica d says:

                A good point. I’ve submitted my request to be added to the waitlist to play with this thing and my number has not yet come up.

                But assuming that the people who are showing these things are not fabricating stories, we’re talking about a technology capable of creating successes of this magnitude (assuming a level of competence that only people like Paras and Sharif happen to have).

                GPT-2 was first announced in February 2019.
                This is GPT-3. It’s not even August.Report

          • Avatar DensityDuck in reply to Jaybird says:

            There used to be entire departments of engineers whose entire career was “draw pictures of things”. That’s what they did, and they were very good at it, and I’m sure that if you asked them they’d have all kinds of reasons why you could never automate their job.

            And yet I’ve got a drafting module in my CAD software, and it draws the pictures of things. And it is up to me to design the things, and to identify what pictures of them are needed and what numbers to put on the pictures, but that entire department of people who turned “give me a picture of the part in thus-and-so orientation” into marks on paper has been replaced by a half-dozen mouse-clicks.Report

          • Avatar veronica d in reply to Jaybird says:

            An easy way to defeat GP-3 as a software engineer is to include code like this in the corpus:


            • Avatar Jaybird in reply to veronica d says:

              If GPT-3 only achieves the level of learning to code that we could reasonably expect from a laid-off factory worker, then…


              I forgot where I was going with that.Report

            • Avatar DensityDuck in reply to veronica d says:

              One of the big ways you can tell that a job is not as necessary as the worker thinks is when the worker starts doing their job shitty on purpose in order to stop a machine from replacing them.Report

              • Avatar veronica d in reply to DensityDuck says:

                You get that I’m not seriously suggesting people do this, right?

                The deeper point is, however, that GP-* can’t necessarily distinguish bad code from good code. All it can do is consume a large volume of text, find patterns, find associations between those patterns, and then spit out chunks of text. Thus, it is great at appearing to solve toy problems, becuase solutions to those problems appear in its corpus, which includes things such as Stack Overflow.

                However, sites like that tend to only have small, unrealistic sections of code. They’ve been stripped of context and complexity precisely because they are for learning, not actual engineering.

                You could have GP-* consume GitHub. That would have non-toy solutions. However, if you ask it to generate code to solve a complex problem dealing with, for example, matrices and eigenvectors, it will struggle. Certainly solutions to these sorts of problems exist on GitHub. Anyone could grab some code and copy it, just as GP-* could. We don’t need an AI to do that, just a good search function. What it won’t do is notice subtle mathematical relationships. If it could, it could generate new algorithms to answer newly posed problems, which are similar to existing solutions in the literature, but different in an important way.

                If you ask GP-* to solve a problem involving, for example, Lanczos methods, it will find code related to that. However, it doesn’t actually understand what Lanczos methods are in a mathematical sense. It just sees text and relationships between text. It sees the strength of the relationships, but not the deeper connections.

                Could it someday, as people improve the algorithms?

                Maybe, but that is quite a bit more complex than what the GP-* algorithms do now. It would be a huge advance.Report

            • Avatar Oscar Gordon in reply to veronica d says:

              Back when I was doing support instead of development, one of my duties was to help users with their macros (within reason). One of my more memorable calls was the guy whose macro wouldn’t run. Compiled fine, just wouldn’t run. So I had him send it to me. When I got it, it was (IIRC) something like a single method with 60K lines of code. The JVM took one look at that Noped right out of there. Upon further examination, it was the same 40 or 50 lines, over and over, with CAD part name assignments.

              Customer got a primer on the concept of ‘Loops’.Report

              • Avatar veronica d in reply to Oscar Gordon says:

                Ha! I can only imagine.

                I used to do field support for Word Perfect. Remember that? It had a macro system.

                The memories.Report

              • Word Perfect, reveal codes. When I worked for the legislature, we had to use Word Perfect. Responsibility for documents was passed on from one analyst to another, for years. Lots of years. One interim I spent a significant amount of time going through the documents I would be responsible for the next session with reveal codes turned on, stripping out all of the accumulated cruft. What a miserable task.Report

              • Avatar veronica d in reply to Michael Cain says:

                Oh gawd I forgot about “reveal codes”.

                The horror. The horror.Report

              • Yep. And people look at me funny when I say that all four-year degrees should require at least a bit of programming. (Also, writing for people’s consumption, but that’s an argument for a different day.) We have reached the point where almost all of us will be forced to “code” more than once in our lives.Report

              • Avatar Oscar Gordon in reply to Michael Cain says:

                Young engineer, too. Just hadn’t taken a programming class, ever. He had the software record a macro of what he wanted to do, and just copypasta’d that 1000 times.

                I felt bad asking him how long that took (days) and then showing him that the loop to do it took 15 minutes to write and test.Report

  3. Avatar Saul Degraw says:

    The big push for a vaccine is because I suspect almost everyone knows that the current way of life around most of the world is untenable for another 18-24 month. This is something I get into a lot of arguments with people on my side politically. We have been doing shelter in place for four months now in California. The current situation in the United States and maybe the rest of the world feels like a limbo of not quite being in quarantine but not being open either. From what I’ve read, there are flares of second waves going up in the rest of the world.

    There is a few I see on my leftier quarters of the internet that we should do shelter in place/stay at home until there is a vaccine. What is never said in these arguments is how long it needs to last? What if it takes 18-24 months to do a vaccine? 36-48 months? I am someone who takes the virus seriously but it seems psychologically unrealistic to expect life to continue like this for another year and a half to four years. Or maybe more.

    I wonder if part of it is the introvert’s revenge (I don’t think most people who spend time on internet comment boards are super extroverts) and politics (“down with the current way of life”).

    Basically, I think almost everyone thinks this sucks and wants life to get back to normal ASAP. No one wants to ask tough questions about COVID as being endemic.Report

    • Avatar Stillwater in reply to Saul Degraw says:

      I wonder if part of it is the introvert’s revenge

      Recycling the oldies but goodies I see.Report

    • Avatar Chip Daniels in reply to Saul Degraw says:

      Person: “I can’t live wearing a mask for two years!”
      Virus: “Your offer is accepted.”Report

      • Avatar LeeEsq in reply to Chip Daniels says:

        It’s more than just wearing a mask. It is the idea that we can basically prevent people from going on with their lives for an extended period of time. People are social animals and like doing things in the with other people, touching, going out, and having fun, etc. Saying no big weddings, concerts, or other social events for a long time is going to be a bit much without some real force behind it. Stories of what are basically Covid-19 speakeasies and people getting in trouble for having parties in their own home have become common.

        My hobby community has been negatively impacted by Covid-19 since partner dancing is not something you can socially distance for. All dance events have been cancelled for the rest of the year. People’s skills are going to decrease because of lack of practice. Even when things get back to normal, most of the studios are going to loose their spaces. The entire thing is going to have to be rebuilt from scratch or something to it. That might not be possible. So its more than waiting for the virus to pass because of a vaccine. There is going to be months of work that needs to be done to rebuild the community before even a social dance can start again. I’m not sure that people are going to stand for this.Report

        • Avatar greginak in reply to LeeEsq says:

          This virus is deadly and contagious enough to be terrible but not quite enough to make everybody really afraid. It’s still distant or plot for some people. Yeah it sucks, we all know and feel that. The force should be beating the virus and having fewer people die. We need to have compassion for the people who are suffering now, all of us, but not at the price of extending this. It’s not like a famous person and former virus mocker has shuffled off this mortal coil today. We’ll be rebuilding for years. That is for sure.Report

          • Avatar LeeEsq in reply to greginak says:

            My point was that eventually you are going to need to use some real force to get everybody to go along with beating the virus. I also think Saul is right that you can’t keep things going until the vaccine appears if that time period is over two years away from March 2020. The global population isn’t going to put their lives on hold for that long.Report

            • Avatar Chip Daniels in reply to LeeEsq says:

              I agree with you and Saul. People are already starting to break quarantine and best practices.

              Which indicates to me that it is increasingly likely that we will develop a culture similar to pre-modern cultures where we just accept high rates of death and suffering.Report

              • Avatar LeeEsq in reply to Chip Daniels says:

                Infectious disease pandemics disappeared from the Western world with the polio vaccine. The last pandemic like Covid-19 was the 1918 flu pandemic. The worlds of the early and mid-20th century might not be as advanced as the current one but it was a modern culture rather than a pre-modern one.Report

              • Avatar Stillwater in reply to Chip Daniels says:

                People are already starting to break quarantine and best practices.

                A story told in two headlines:

                June 19: Colorado bars can reopen this weekend for up to 50 people

                June 30: Gov. Jared Polis closes Colorado bars again after coronavirus cases increase

                11 days apart!Report

        • Avatar Stillwater in reply to LeeEsq says:

          My hobby community has been negatively impacted by Covid-19

          More than 40% of US renters are at risk of eviction

          And I bet some of those people are introverts too.Report

      • Avatar Saul Degraw in reply to Chip Daniels says:

        The mask is fine. Lots of stuff is fine for now. The question is how long does it continue to be fine for. Whole sectors of the economy are essentially shut down for the foreseeable future. We may see a tidal wave of evictions, foreclosures, and bankruptcies. School should not reopen in the fall but there are going to be huge issues with children getting subpar instruction probably.

        Not to mention all the social effects of delayed and canceled weddings, birthday parties, vacations, etc.

        All the stuff that makes life enjoyable seems to be locked in a box right now. Maybe it will come back sooner than I think but the idea of all the things making life enjoyable being locked in a box have not been seen since what? WWII? Even then, there was a lot done to keep morale up as much as possible along with the sense of community and sacrifice. The thing about a pandemic is that the best message is “Life is going to suck for the next 1.5-4 years. Sorry.”Report

    • fillyjonk fillyjonk in reply to Saul Degraw says:

      well, one thing this has taught me is I am definitely not a full on introvert by that definition. I am hanging on by mere fingernails to my sanity right now.

      I want to hope a vaccine is coming soon, but I’ve had my hopes dashed so much in this that I’m reaching the point of acceptance, that what remains of my life will be a locked-down misery where I go to my office, to the grocery store, home, never see anyone, never get close to anyone, ultimately, die alone.

      I want to be proven wrong in that but “a vaccine before the elections!” seems a little too facile. Deus ex Machina really only works in Greek plays and cartoons.Report

    • Avatar LeeEsq in reply to Saul Degraw says:

      Part of the reason why I expect nobody wants to deal with Covid-19 being endemic is that medical science and public health believes it not only has tools to lower the intensity of the pandemic, they have an obligation to. The idea of asking what happens if Covid-19 becomes an endemic is simply a sign of defeat. It means going back to when infectious diseases where a part of life that people learned to deal with rather than something rare. I suspect that ordering a total change until when a vaccine appears seems more humane to people than having the world learn to live with Covid-19.Report

  4. ThTh4: The Kuiper Belt occurred on August 29, 1977.Report

Leave a Reply

Your email address will not be published. Required fields are marked *