Coronavirus news #4. An update to the last post about the IHME model. Bottom line: there's still good reason to be sceptical of its predictions.
1) I had expressed scepticism about a graph on the IHME website. It confused some epidemiologists too, but one of them finally figured out what was going on. It was driven by a combination of the nature of the model (atypical and not ideal, but not wrong), poor visualisation choices combined with a lack of explanation (uncertainty in the 'fitted' part of the graph was hidden without explanation), and bad luck (the date of max uncertainty was not simply the next day; it had remained stable from the previous week). Too long to explain here, but you can read more here if you're interested: https://twitter.com/CT_Bergstrom/status/1249135767613554690. I partially withdraw this particular criticism; it's definitely bad but not as egregious as it seemed. The other points still stand, and unfortunately some more have cropped up:
2) People with actual expertise in this figured out that the model has a strange quirk: it assumes that once the peak is reached, the epidemic will decline at the same speed at which it rose (once social distancing is in place). There is absolutely no reason to expect this to be true, and the patterns in China and a few other countries do not match this; the decline is generally quite a bit slower. This is not important for the model's main purpose (predicting peak hospital usage) but does matter because people & governments are making plans based on these predictions of numbers of deaths. A more thorough explanation of this is here: https://twitter.com/CT_Bergstrom/status/1250304069119275009
3) A group of statisticians and epidemiologists tested how well the model predicted deaths on the following day and it did badly. The model was 'fitted' with data till March 29 and made predictions for deaths in all US states on March 30. Every prediction had a '95% prediction interval'; these represent a range of possibilities for the number of expected deaths. If the model worked well, 95% of the predictions should fall within those intervals. In reality, less than 30% did. In other words, the model was very overconfident in its expected range of possibilities - and this is just for predictions on the next day. The model has since been updated and this group will test the new predictions soon. Details in the full report here.
----------------
In response to a question about herd immunity:
Herd immunity is great, the question is how we get there. There are two paths: either we vaccinate most people or most people get infected. Most people getting infected likely means that many millions will die, especially if it happens quickly enough for medical facilities to get swamped. A vaccine will apparently take 1-2 years to develop. Lockdowns for that kind of time frame are basically impossible because people will revolt. If it were possible, it would still lead to a lot of deaths (I have no idea how many) through the damage to incomes and economies & the consequent drop in medical care and degradation of supply chains.
So we're buying ourselves time to prepare right now with shorter lockdowns, working to understand the virus and disease better, develop treatments and testing capabilities. We are probably going to reopen before the public health experts and doctors think is ideal because the public will demand it. I expert partial reopenings in many countries in 2-4 weeks. Switzerland and a few countries actually have a declining total number of infections, so a slight reduction of the lockdown is justifiable. How we manage the reopening will be crucial: Have we put in place the infrastructure to do testing and tracing on a large scale (necessary to avoid large loss of life, despite the privacy concerns)? How small do we start? How do we manage the inevitable jockeying between different entities that want their industry/business/hobby to be allowed? What is the sequence of reopening steps to follow? What planning do we put in place to trigger a second lockdown if the inevitable second wave of cases starts to threaten again?
----------------
In response to a question about reinfection:
Reinfection seems to be way down the list of most experts' concerns right now, and with good reason. Partly there's not enough data, though it has been looked at recently on a small scale. The general consensus seems to be that it would be extremely unusual for this virus to NOT lead to at least some post-infection immunity, because that's what happens with most comparable viruses. So it's a reasonable (but definitely important) assumption till we know more. The reports of re-infections have so far been rare and in some cases don't seem very reliable because of issues with testing.
But if we make the pessimistic assumption that immunity declines with time till it is eventually useless, whether herd immunity takes hold will I think depend on the rates of disease spread and the rate at which immunity declines. And it will be temporary anyway, since this scenario assumes everyone will lose their immunity.
Basically, most realistic paths end with us having herd immunity through vaccination.
If you want to read more about immunity to viruses, this seems like a good place to start (kind of technical, though):
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3899649/
----------------
In response to a question about the Swedish strategy and the modelling it was based on:
This led me down a rabbit hole! First, I discovered that Worldometer, my usual source of data on deaths and cases, shows a bizarre weekly pattern in its data for Sweden, with very few deaths on weekends. And it's not a reporting problem, since there's no spike on Mondays. Digging into it further, the original data source does NOT show this weekly pattern, so I don't know what is going on [EDIT: this was because there were reporting lags and the original source would update its numbers when the data came in. But Worldometer would take only the latest day's number and ignore updates to previous days].
But back to the question: I'm not familiar with the Swedish modelling work because the only link I found explaining it was a while paper in Swedish. But I don't think that anyone is really trying to achieve herd immunity in the way that this is understood. Even the weird initial UK plan was not exactly about trying to achieve herd immunity; it was in some sense a side effect. Like with Sweden now, the plan to not lock down was based on the assumption that people will not tolerate the strict and long lockdown that would be needed to completely stop the disease spread. Or that the economic and consequently human costs of such a long lockdown would be unbearable. So given that a long, total lockdown is not possible, how best do we manage the situation? A short, strict lockdown would suppress the disease for now but lead to the same spike once the lockdown is lifted.
The Swedes argue that allowing a slow spread right now through weak measures is the best option available. Assuming they have no shortage of medical supplies (the best argument for a lockdown is preventing medical facilities from being overwhelmed), their plan may be wrong but it's not totally ridiculous. Sweden has a low density of population, so the risks are probably lower than most countries (Stockholm and perhaps some other cities will be more at risk). Switzerland has enforced a relatively weak lockdown (though still much stronger than Sweden) but had a lot of success containing the spread, so this is one point in favour of weak measures. And Sweden retains the ability to enforce a lockdown when things get worse (which they will) and have the population being willing to go along with it.
In the meantime, Michigan just had masses descend on the state capital to block roads (and hospitals!) to protest the short lockdown they have already experienced. What happens when things get worse and stronger measures are needed? I don't envy politicians and administrators having to take these decisions.
Ultimately, short-term lockdown followed by intense testing and tracing still seems the best strategy. But there's so much unexplained variation between countries that it's hard to claim with any confidence that the Swedish approach will fail.
1) I had expressed scepticism about a graph on the IHME website. It confused some epidemiologists too, but one of them finally figured out what was going on. It was driven by a combination of the nature of the model (atypical and not ideal, but not wrong), poor visualisation choices combined with a lack of explanation (uncertainty in the 'fitted' part of the graph was hidden without explanation), and bad luck (the date of max uncertainty was not simply the next day; it had remained stable from the previous week). Too long to explain here, but you can read more here if you're interested: https://twitter.com/CT_Bergstrom/status/1249135767613554690. I partially withdraw this particular criticism; it's definitely bad but not as egregious as it seemed. The other points still stand, and unfortunately some more have cropped up:
2) People with actual expertise in this figured out that the model has a strange quirk: it assumes that once the peak is reached, the epidemic will decline at the same speed at which it rose (once social distancing is in place). There is absolutely no reason to expect this to be true, and the patterns in China and a few other countries do not match this; the decline is generally quite a bit slower. This is not important for the model's main purpose (predicting peak hospital usage) but does matter because people & governments are making plans based on these predictions of numbers of deaths. A more thorough explanation of this is here: https://twitter.com/CT_Bergstrom/status/1250304069119275009
3) A group of statisticians and epidemiologists tested how well the model predicted deaths on the following day and it did badly. The model was 'fitted' with data till March 29 and made predictions for deaths in all US states on March 30. Every prediction had a '95% prediction interval'; these represent a range of possibilities for the number of expected deaths. If the model worked well, 95% of the predictions should fall within those intervals. In reality, less than 30% did. In other words, the model was very overconfident in its expected range of possibilities - and this is just for predictions on the next day. The model has since been updated and this group will test the new predictions soon. Details in the full report here.
----------------
In response to a question about herd immunity:
Herd immunity is great, the question is how we get there. There are two paths: either we vaccinate most people or most people get infected. Most people getting infected likely means that many millions will die, especially if it happens quickly enough for medical facilities to get swamped. A vaccine will apparently take 1-2 years to develop. Lockdowns for that kind of time frame are basically impossible because people will revolt. If it were possible, it would still lead to a lot of deaths (I have no idea how many) through the damage to incomes and economies & the consequent drop in medical care and degradation of supply chains.
So we're buying ourselves time to prepare right now with shorter lockdowns, working to understand the virus and disease better, develop treatments and testing capabilities. We are probably going to reopen before the public health experts and doctors think is ideal because the public will demand it. I expert partial reopenings in many countries in 2-4 weeks. Switzerland and a few countries actually have a declining total number of infections, so a slight reduction of the lockdown is justifiable. How we manage the reopening will be crucial: Have we put in place the infrastructure to do testing and tracing on a large scale (necessary to avoid large loss of life, despite the privacy concerns)? How small do we start? How do we manage the inevitable jockeying between different entities that want their industry/business/hobby to be allowed? What is the sequence of reopening steps to follow? What planning do we put in place to trigger a second lockdown if the inevitable second wave of cases starts to threaten again?
----------------
In response to a question about reinfection:
Reinfection seems to be way down the list of most experts' concerns right now, and with good reason. Partly there's not enough data, though it has been looked at recently on a small scale. The general consensus seems to be that it would be extremely unusual for this virus to NOT lead to at least some post-infection immunity, because that's what happens with most comparable viruses. So it's a reasonable (but definitely important) assumption till we know more. The reports of re-infections have so far been rare and in some cases don't seem very reliable because of issues with testing.
But if we make the pessimistic assumption that immunity declines with time till it is eventually useless, whether herd immunity takes hold will I think depend on the rates of disease spread and the rate at which immunity declines. And it will be temporary anyway, since this scenario assumes everyone will lose their immunity.
Basically, most realistic paths end with us having herd immunity through vaccination.
If you want to read more about immunity to viruses, this seems like a good place to start (kind of technical, though):
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3899649/
----------------
In response to a question about the Swedish strategy and the modelling it was based on:
This led me down a rabbit hole! First, I discovered that Worldometer, my usual source of data on deaths and cases, shows a bizarre weekly pattern in its data for Sweden, with very few deaths on weekends. And it's not a reporting problem, since there's no spike on Mondays. Digging into it further, the original data source does NOT show this weekly pattern, so I don't know what is going on [EDIT: this was because there were reporting lags and the original source would update its numbers when the data came in. But Worldometer would take only the latest day's number and ignore updates to previous days].
But back to the question: I'm not familiar with the Swedish modelling work because the only link I found explaining it was a while paper in Swedish. But I don't think that anyone is really trying to achieve herd immunity in the way that this is understood. Even the weird initial UK plan was not exactly about trying to achieve herd immunity; it was in some sense a side effect. Like with Sweden now, the plan to not lock down was based on the assumption that people will not tolerate the strict and long lockdown that would be needed to completely stop the disease spread. Or that the economic and consequently human costs of such a long lockdown would be unbearable. So given that a long, total lockdown is not possible, how best do we manage the situation? A short, strict lockdown would suppress the disease for now but lead to the same spike once the lockdown is lifted.
The Swedes argue that allowing a slow spread right now through weak measures is the best option available. Assuming they have no shortage of medical supplies (the best argument for a lockdown is preventing medical facilities from being overwhelmed), their plan may be wrong but it's not totally ridiculous. Sweden has a low density of population, so the risks are probably lower than most countries (Stockholm and perhaps some other cities will be more at risk). Switzerland has enforced a relatively weak lockdown (though still much stronger than Sweden) but had a lot of success containing the spread, so this is one point in favour of weak measures. And Sweden retains the ability to enforce a lockdown when things get worse (which they will) and have the population being willing to go along with it.
In the meantime, Michigan just had masses descend on the state capital to block roads (and hospitals!) to protest the short lockdown they have already experienced. What happens when things get worse and stronger measures are needed? I don't envy politicians and administrators having to take these decisions.
Ultimately, short-term lockdown followed by intense testing and tracing still seems the best strategy. But there's so much unexplained variation between countries that it's hard to claim with any confidence that the Swedish approach will fail.
