In August 2011, Kristin Cobble, Rebecca Petzel, and I had a planning meeting for Groupaya, the consulting firm we would start several months later. As part of that, Rebecca led us through some initial scenario thinking, which consisted of brainstorming certainties (trends we thought were almost certainly going to happen by 2016) and uncertainties (trends we thought were possibilities).
Here were the initial lists we brainstormed:
Economy really crappy in 2015
Africa will be online
Design firms flooding into the business (good design the price of entry)
Communication and Advertising Firms coming into the business
There’s a backlash against “collaboration”?
There’s a backlash against “social”?
Earthquake in San Francisco
Skilled, cheaper consultants coming here from developing countries
Knowledge work in the US in the decline
Knowledge work undervalued in the US
Trust in Internet services? Things like Wikipedia, AirBnB, eBay rely on trust
Institutional clamp down or continued democratization
Middle East political situation
U.S. “Arab Spring” coming?
Backlash against rationalism; rise of fundamentalism
Large factory consulting firms hijacking our business
Our “Certainties” list wasn’t very good. The economy was not “crappy” by conventional metrics in 2015, although we were continuing to feel the impacts of widening inequality. And we didn’t really see communications firms come into the business.
Our “Uncertainties” list was far more interesting. We no longer have net neutrality, at least at the federal level. Trust in several social media (Facebook and Twitter in particular) is down, and deservedly so. And reading the bullet point, “Backlash against rationalism; rise of fundamentalism,” now makes me want to cry.
I review these notes every few years out of curiosity and sentimentality, and I pulled them up again last month as COVID-19 was wreaking havoc on our lives. A few things come up for me when I look at these:
It’s possible to have an interesting scenarios conversation without a lot of prep. We were clearly already connected to a lot of interesting people and perspectives, which was how stuff like “backlash against rationalism” made it onto our list. (Kristin contributed that one based on conversations she had had with her friend and former colleague at Global Business Network, Eamonn Kelly.)
Prep would have helped broaden our perspectives and address some blind spots.
Pandemic wasn’t on the list of uncertainties.
The biggest thing that comes up for me is that we never truly benefited from the power of scenario thinking, because we treated it as a one-off. Imagine if we had returned to this list once a year, even without any additional prep, and talked through the possibilities. What might have come up? How might this have changed our thinking? What might we have done differently as a result?
This is a regret I often have about my own past work, and it’s something I find with consulting work in general: We barely benefit from the work (which is often time- and resource-intensive), because we never revisit it. There are lots of reasons we never revisit it, but the most common one is that we’re going too fast. I’ve been able to correct this with my own work (although it took several years and lots of focus and failure), and I continue to try to help others do the same. It’s been really, really hard, which is sad, because it’s so beneficial.
On any other day, I would have barely paid attention. But these are not ordinary times. So I stopped, and I watched, and I wondered, as the sandpipers played by the crashing waves, unaware of the strange things occupying my mind in these strange times.
TL;DR I’m now using this dashboard as a way to make sense of what’s happening with COVID-19. It’s still too soon to draw any conclusions about how well the U.S.’s interventions overall are working.
I started trying to make sense of the COVID-19 growth rate data myself on March 13, 16 days ago. I learned a lot along the way, and my daily ritual of looking up numbers and updating my spreadsheet has been strangely calming. Here’s my latest graph:
Italy’s growth rate seems to be flattening, which is a positive sign
U.S.’s growth curve continues to rise at a steady rate; more on this below
Even though China and Korea’s growth rates have been steady for a while now, it’s not zero. They have this under control (for now), but it’s far from over, and it won’t be until we have a vaccine, which folks keep saying is at least 12-18 months away.
My friend, Scott Foehner, chided me last week for saying that the results are lagging by about a week. He’s right. Based on Tomas Pueyo’s analysis (which I cited in my original blog post), the lag is more like 12 days. This is why the Bay Area shelter-in-place ordinance was for three weeks — that’s how much time you need to see if you’re containing your growth rate.
Shelter-in-place in the Bay Area started on March 17, exactly 12 days ago and four days after I started tracking. California’s order started on March 20. Other states followed after that, but not all.
It’s hard to make sense of all this when aggregated as a country. I’ve been wanting regional data for a while now, but have felt too overwhelmed to parse it out myself. Fortunately, other people have been doing this.
One of the positive outcomes of me doing this for myself for the past few weeks is that it’s given me a better sense of how to interpret other people’s graphs, and it’s helped me separate the wheat from the chaff. It’s also made me realize how poor data literacy seems to be for many media outlets, including major ones. They’re contributing to the problem by overwhelming people with graphs that are either not relevant or are not contextualized.
Wade’s dashboard is pretty configurable overall, although you have limited control over which region’s data you’re showing. Here’s the closest equivalent to what I’ve been tracking:
And here’s what I’ve really wanted to see: the state-by-state data:
What does this tell us about the interventions so far? Again, not much. It’s too soon. Check back in another week.
I’ve seen some articles floating around with graphs comparing California to New York, crowing that sheltering-in-place is already working here. That may be the case, but it’s still too early for us to know that, and it’s irresponsible to point to a chart and suggest that this is the case. There are lots of reasons why New York might be doing so poorly compared to California that have nothing to do with interventions, density being the obvious one. Regardless, history has proven that even a few days can make a huge difference when it comes to containing epidemics, and I feel incredibly grateful that our local leaders acted as quickly as they did.
I think there are two questions that are on people’s minds. One is about hospital capacity. I’ve seen various attempts to model this, including the Covid Act Now site I mentioned last week. The one I find easiest to browse is this dashboard from the Institute for Health Metrics and Evaluation. They publish their model, which I haven’t even attempted to parse yet. (I doubt that I have the expertise to evaluate it anyway.) It suggests that, even if our current measures have flattened the curve in California, we’ll still exceed our capacity of ICU beds needed in about two weeks, although we should be okay in terms of general hospital capacity.
The second question is how much longer we’ll need to shelter-in-place (or worse). Even if we flatten the curve, lifting shelter-in-place will just get that curve going again unless we have an alternative way of managing it (e.g. test-and-trace). I haven’t seen any indications of when that will happen, so we’ll just have to continue to be patient. I feel like every day is a grind, and I’m one of the lucky ones. I can’t imagine how folks on the frontlines and those far less fortunate than me are dealing right now.
My morning ritual for the past week has been to update my COVID-19 spreadsheet and ponder my chart. Here’s the latest:
On the one hand, if you compare it to last week’s chart, it’s not a happy result for those of us in the U.S. (Italy’s curve might be flattening. We’ll see next week.) On the other hand, remember that this is a lagging indicator. This past week’s line was essentially pre-determined by what happened the previous week. Earlier this week, the Bay Area institutedshelter-in-place. Shortly thereafter, California made it state-wide, and New York and Illinois followed suit after that. We’ll see if this has any noticeable impact next week.
I made one slight tweak to the graph (adding labels to the axes; thanks to Kate Wing for the gentle scolding). I’d like to change the gridlines on the x-axis to every seven days, but can’t do that in Google Sheets. Not a huge deal. I’d also like to experiment with a log 2 graph (versus log 10) on the y-axis to more easily show how many days it takes for new cases to double, but again, can’t do that from Google Sheets. Again, not a big deal. I’d also like to do a region-by-region analysis, as suggested by many others and made possible by David Janes’ data, but haven’t gotten around to that yet.
I started doing all of this as an exercise in self-care. I wanted to understand what was happening, and I found what I was reading to be not just largely unhelpful, but actually debilitating. This has helped a lot. There is something very calming about looking up numbers, plugging them into a spreadsheet, and pondering the results, even if the results aren’t very good. (Come to think of it, this also played a huge role in helping me achieve better work-life balance, so it might be a pattern.) I haven’t been able to avoid the media as much as I hoped, but it’s helped me make sense of what I’m seeing and ignore articles and missives that are generally unhelpful or worse. It’s also validating when folks who understand this stuff far better than me are coming to similar conclusions.
I’ve loved seeing friends and others play with the data as well. One of the best websites I’ve seen is Covid Act Now, which shows state-by-state projections based on hospital capacity and what we understand about different interventions. They’ve also shared their model openly, and they’re posting the right disclaimers. (Good rule of thumb: Be skeptical of anyone who claims certainty about their conclusions unless they’re an epidemiologist, and even then, take everything with a grain of salt.)
I’m also inspired by everyone working on the front lines — from health care workers to domestic workers — and to those who are doing their part to support those who are. (Hat tip to Jon Stahl for sharing the amazing work that Carl Coryell-Martin instigated, for example.) Stay safe everyone, stay at home if you can, and be well.
Many thanks to all of the feedback about my latest attempt to make sense of the Coronavirus pandemic. I listened, and I played, and I learned, and I now have a new graph that I think better represents how the U.S. is doing:
This is a semi-logarithmic graph of daily new cases over time. I’m comparing the U.S. (blue and bold), China (red), South Korea (yellow), and Italy (green). I’ll explain the changes I made from last time below, but first, three quick takeaways:
First, don’t trust my analysis. I’m an amateur at this, my math is incredibly rusty, and it turns out that my statistics (which were always suspect) are even more suspect than I thought. Critiques, corrections, and constructive discussion encouraged!
Second, don’t passively consume what you read. This started off as a quick exercise to try to make sense of the craziness. It’s led to lots of encouragement, but also lots of (welcome) critiques, which has helped me sharpen my analysis, correct some assumptions, and feel like I have a better grasp of what’s going on and how to assess other things I read. I feel a lot better than I did a few days ago
Third, the U.S. isn’t doing great right now. Our line is more or less tracking China, Korea, and Italy’s initial growth rate, but both China and Korea had started slowing their growth rate about a week before we did. Our curve is looking a lot more like Italy’s, which does not speak well of the weeks ahead here.
Here are the changes I made from last time:
First, I switched over to a semi-logarithmic graph. Hat tip to Ken Chase for encouraging me to do this and to Matt Bruce for this pointer as to why this is important.
When I originally started mapping this out, I didn’t think that the semi-logarithmic graph would tell me much more than the linear graph did. Italy was off the chart, which was all I felt like I needed to know, and I felt like I could make sense of the rest of the curves on the chart. Still, after receiving this feedback, I decided to try the semi-logarithmic version to see what I would learn. As you can see above, my conclusion changed quite dramatically. We in the U.S. are not doing well right now. You can see that the slope of our graph tracks quite closely with Italy.
The other (more math-y) benefit is that I can measure the slope of the line (0.15), which gives me the rough power law for how viral Coronavirus has been in both the U.S. and Italy (y = 100.15x). China and Korea’s containment slope (-0.1) provides the rough power law for the potential impact of containment (y = 10-0.1x). You can use these equations to model out different scenarios (which my friend, Charlie Graham, has been doing).
Second, I’m no longer normalizing by population. Two people questioned whether or not this was useful. (Thank you, Majken Longlade and Corey O’Hara.) Their argument was that normalizing by population doesn’t tell you much in the case of epidemics, because transmission and virality are more a function of closeness and density. The U.S. is a huge country geographically compared to Korea or Italy. The better approach would be to try to normalize based on population of regional outbreaks.
I agree, and I’d like to try to do this. The data is a lot harder to come by, but I think it would be possible to manually pull together with a little bit of elbow grease, especially if we’re constraining the countries where we’re trying to do this. (Leave a comment below if you’d be interested in trying this.)
(Side note: I am extremely grateful for the wide availability of data and for all the people doing an incredible job of analyzing and sharing. This is no accident. A lot of folks have invested an incredible amount of time and energy over the past two decades advocating for the open web and open data, all of which is required to make this work. This becomes even more clear when realizing what’s missing. Martin Cleaver asked me to include Canada in my graph. Easy enough, I thought. Turns out it’s not. Canada doesn’t make this data available. Majken shared this article that explains Canada’s situation.)
Nevertheless, I thought it was still useful to look at data normalized by population. My reasoning was that, at worst, it wouldn’t make the data worse, and, at best, it might make it better. I decided I would try to test this assumption by comparing the graphs of the normalized data with the non-normalized data above:
Again, I think switching to a semi-logarithmic graph made a difference, because if you compare these to graphs, the slopes (which I’m most concerned about) are largely the same. Normalizing the data doesn’t impact the slopes. On the one hand, my assumption was correct — normalizing didn’t seem to hurt the data. On the other hand, it also didn’t tell me anything new, either. So, I decided to stop normalizing and stick with the data as is. (I’d still like to try normalizing by outbreak region, though.)
One point that came up often was that these graphs don’t take into account the underreporting in the U.S. due to lack of testing. I tried to take this into account in my very first sketch, as I mentioned in my original blog post. However, I decided to move away from this for a few reasons. First, every country is underreporting. It didn’t feel useful to add in hand-wavy multipliers. Second — and this is where the semi-logarithmic graph again comes to the rescue — adding a multiplier won’t change the slope, which is what I’m really interested in. It just moves the curve up or down.
Remember, these are all lagging indicators anyway. In all likelihood, any changes we make today won’t be reflected for at least a week. What’s done is done. The best thing we can do right now is to be as proactive as possible, given the circumstances. If we’re going to implement policies like Italy has, it’s better that it happens today than a week from now. Public policy aside, there is one thing we all can do that will absolutely make a difference: STAY HOME!
Many thanks to Martin Cleaver and Matt Bruce for sharing my previous blog post, which led to a lot of the discussion that shaped this latest iteration. And many thanks to all who have engaged with this so far. Stay home, wash your hands, and take care!