Why the Pandemic Statistics are Wrong

tl;dr:

The absolute numbers in most official and media reports are not useful; we should use proportions instead.
Lockdown is not safe; people die from isolation.
We don’t have enough information to make reliable predictions; we do have enough to make decisions.

Just as we should be aware that new cases are not new infections, so we should be aware that the ‘new death’ figures published by PHE have little to do with actual deaths caused by COVID-19 across the UK.

Like anthrax, COVID-19 is now a notifiable disease. This means suspect fatalities are tested for it and positive results recorded on the death certificate. The PHE reports fatalities in hospitals with COVID-19 on their death certificate, not fatalities who are assessed as dying of it – and as they test more they will discover more. The higher the extent of infection in the population, the more these figures will over-represent deaths by the virus. On the other hand most home deaths of the virus are not included so these figures will under-represent deaths by the virus. These two effects do not cancel each other out! (more detail in this article by ONS’s Sarah Caul)

This means we can’t tell – from these PHE numbers – whether we are seeing large numbers of people that are being killed by the virus, or discovering that a large proportion of the general population is already infected. Or somewhere inbetween.

More people are definitely dying. The UK Office of National Statistics collates deaths in the UK by week:

And the European Morbidity Monitor collates deaths across Europe. The earlier annual spikes are the effects of seasonal flu across Europe (ignore the final ‘down’ tick as not all the data is in yet):

We can see many more people are dying than ‘normal’, although it’s not entirely unprecedented. Many of these will be from COVID-19, but some will be dying due to side-effects of the lock-down (see below).

How then do we estimate the impact of COVID-19 in particular? Around 16,000 deaths in the UK were attributed to the flu over the 2017-2018 flu season. Are we in a similar place, or a nightmarish worse one? Are people dying of COVID-19 instead of the flu? Dying of COVID-19 now instead of the flu tomorrow? Next week? Next month? This year?

We won’t be able to tell until all the data is in, and even then there will be some data gaps and uncertainties.

Sometimes that’s fine; we can wait. But in these circumstances waiting until it’s all over will be too late. We have a question now, and we don’t have enough information to be sure of the answer.

The Urgent Question

Being locked down, if you’ll excuse the pun, is not free. Those of us on linkedin are connected online, and many have professional salaried jobs and a house with a garden. We continue to work and engage and ‘socialise’. We may even be happy to home-school our children; making sure they are doing their zoom-assigned homework, engaging online tutors and trawling the web for teaching materials. We may feel safe, and looked after.

Our lot is nothing like those isolated, alone in a small flat with only their mobile to interact with others, and free TV channels to amuse themselves. It’s nothing like those watching their income dry up and their businesses, their future, their livelihoods, their children’s education, dribble away. It’s nothing like those trapped in a house with abusive partners and parents. It’s nothing like those who are ill or frail and have just lost their support networks.

Worse, people are dying because of the lockdown.

Some people with severe illnesses – perhaps nobly reducing the load on the NHS, perhaps just frightened – are not going to hospital or are not calling the ambulance in time and so are dying unnecessarily

Some residential home carers – perhaps called away to look after children, perhaps just frightened – are leaving. Even if nightmare individual cases are rare – and we don’t know that they are – the extra load on the remaining staff will affect care quality. No matter how heroically the remaining staff work in the face of infection, social isolation, difficult patients and thankless relatives, reduced care will kill more frail individuals more quickly.

Indirectly we expect thousands more suicides as the economic downturn hits, and we expect other hard-to-quantify harms as people’s support networks disintegrate. As the effects cascade through the economy more and more businesses will have to downscale and shed all levels of employees.

As a nation we might cope with a few weeks of lockdown, but every week – every day – visits more harm on many. The government’s attempt to mitigate the pain by spending is just moving it around – borrowing from the future to pay for the present. There is still no money tree, no matter how much we wish for it. And money can’t fix some problems; some children don’t have the equipment, attitudes, aptitudes, home space or parental support for remote learning and the longer they remain un-schooIed, missing out on the building blocks required for the next term, the more likely they will face difficulties later.

In the meantime the government has popular support; all eyes are on them and they have a cause, a crisis to manage. Few people are protesting – but then we cannot even protest about our inability to protest.

The government may well be tempted to put off the inevitable questioning and backlash for another day, another week, another month… or more.

Many of us too can comfortably, for now, keep pressing that snooze button. Some of us may be dreaming that everything has changed and we can radically alter society, but I’m afraid the mundane real world awaits us all – along with all the harmful consequences of our four-week suppression. We too may be tempted to put off the inevitable, but the longer we delay the harsher the consequences – and the more people will forcibly object to being forcibly imprisoned.

In any case, even if we were selfish we should think about those we depend on to function as a society and how the effects on them will affect us.

We need to think about – to discuss and maybe even agree – the lockdown exit strategy.

This not a simple question of ‘lives’ vs ‘the economy’. Not only do people also die under lockdown but the economy is not separate from life; our livelihoods are our lives and we require wealth for health. Quantity of life is being traded for both quality and quantity. The harms of lockdown are not easily compared with the harms of freedom in a pandemic, but in any case locking other people up ‘for their own good’ should feel … uncomfortable.

What, then, are the right circumstances to consider a return to freedom? In the future – and this will happen again in our interconnected world – what are the right circumstances to self-imprison? Should we, for example, have locked down our societies for three months during the flu epidemic of 2017-2018 in order to protect the NHS and save lives?

Answering this is partly moral and partly political. But to inform the answer we want the right, useful, timely data.

What do we want

To understand most risks we need a sense of proportion, rather than a taste for absolutes. Knowing that fifteen people died in car crashes somewhere tells you very little if you don’t also know, at least, the number of people who didn’t die.

Knowing that we have 2,000 new cases of a virus tells us nothing if we don’t know how those new cases came to be. Who was tested and why? Are they people who have just got the symptoms and reported them? Who else has the symptoms but hasn’t reported them? Who doesn’t have symptoms but is infected? Are they cases discovered by sampling the population?

Similarly, knowing that we have 500 new deaths with a virus tells us nothing if we don’t know how those fatalities were recorded, or how many were tested for the virus in the first place.

Use Proportions

Not Absolutes

For viral pandemics we typically want to know:

Infection extent history: at any one time what proportion of the population have the virus?
Health service load history: what proportion of the relevant health-service capacity is in use?
Deaths history: what proportion of deaths are due to the virus? What proportions of them had comorbidities? What proportions of life expectancy remained?

As opposed to the useless absolute ‘new cases’, ‘new deaths’ and ‘crude death rates’ figures, with these proportions we can understand the risks associated with various vulnerabilities and estimate how quickly infection is spreading. Infection extent can tell us:

If it is very low or localised then infected individuals can isolate to eliminate the spread (as in South Korea, or when dealing with Ebola)
If it is widely spread but low then we can ‘flatten the curve’ by widespread lockdown,or at least particularly vulnerable populations can isolate.
If it is high then it is too late and we will just have to cope. Lockdowns will have no effect

When do we want it?

Wanting data is all very well, but we can’t always have what we want. In practice it is hard to measure anything when the priority is to reduce harm in a rapidly escalating crisis. We will not normally, initially, have the right specialist equipment or chemicals for widespread testing of a new virus, nor the right training or processes in place to measure, record, collect and collate the relevant data.

We have to make do with what we have now, not what we would like to have later. An answer too late is no answer at all.

What do we have now?

Let’s see what publically-available data we can use to estimate our three sets of ideal data: infection extent, health service load and related deaths.

Infection Extent

We can look at the extent of an infection in a population by testing the whole population (which only works for small ones) or sampling it (which can give us skewed results depending on who we sample).

Full Population Tests

I only know of two significant full population tests: a cruise ship and an Italian village.

On the Diamond Princess cruise ship a close-living population was put in quarantine two weeks after an infectious individual joined them, although even after that internal isolation measures may not have been sound. The whole population of 3,700 people were tested: in that short period about 700 (20%) had been infected; of those about 60% showed no symptoms and 1% (all with pre-existing conditions) died.

The Italian village of Vo similarly tested its full local population of about 3,500 soon after the first death in order to identify the infected and isolate them. They found 3% had been infected; of those over 50% showed no symptoms.

Broad Sampling

Countries such as Germany and Korea have tested broadly. Germany found nearly 10% were infected at the end of March. Both countries started testing early after the first cases so the spread is likely limited because people isolated when tested positive.

These populations, like that of the village of Vo, are likely to be still susceptible to this virus if and when they open their houses and borders.

Limited Sampling

Where we can’t test widely we have to rely on sampling. This normally requires diverting test efforts from what may seem to be more important people, such as patients and their medical staff. However there are sometimes other options: Danish researchers for example took a small sample of 1,500 blood donations and tested them, finding 22 (1.5%) were positive. Extrapolating this to the total Danish population and comparing it to their COVID-19 related deaths, they found a death rate of around 0.16%, but this is a small sample.

In the UK, now that more tests are available, PHE are testing more than inpatients. Tests of key workers and their households (“Pillar 2”) are finding about 30% are infected – lower but not dissimilar to the inpatient rate.

These are not new infections; they are discoveries of existing infections. If a third of the population is infected, we can expect a third of the 5,000 daily tests to provide 1,500 new cases every day. If more testing kits arrive and we test 10,000 people in a day, we can expect 3,000 new cases. That is not an increase in infections, it is an increase in discovered infections. To measure increases (or decreases) in infections we need to use the proportion of tests that had positive results.

All the same our proportions will be skewed by who we test. Since people with symptoms are more likely to be tested than those without, and key workers have been in more contact with more people than those in isolation, the background population infection extent is likely to be lower than the Pillar 2 results show. We can conclude then that the recent extent of infection in the UK is probably less than 30%, but significantly more than Germany’s 10% which has around a quarter of the deaths of the UK but a slightly larger population.

From my notes, the proportion for this Pillar 2 population was similarly about 30% on the 11th & 12th April when the figures were first published. Those that were infected in early April will have largely either succumbed or recovered by now and so will not appear in the Pillar 2 figures but will be mostly immune from the current forms of the virus.

Overall then we have a rough estimate, as we did a month ago, of an infected proportion of the UK population not dissimilar to that of the Diamond Princess, ie 20%. Add a similar proportion of those infected a month ago who have recovered, and we can expect between a third and a half of the UK population have already been infected, be immune or be otherwise resistant.

To confirm this we need to test more widely and we need to test for antibodies. “Pillar 4” of the programme will sample the population for antibodies to find people who have had the virus and recovered. Pillar 2 and Pillar 4 together should tell us what proportion of the population has already been infected and so whether continuing lockdown is useful.

Pillar 4 antibody tests will not be available until May. We have a rough but well-bounded estimate to act on – or we can wait. As long as we understand what harms we do to others as we wait.

An alternative to infection extent: 'reproductive number'

We can track a pandemic’s progress using its ‘reproductive number’, which is how many people each infected person will infect, over time.

At first in a pandemic the reproductive number is greater than one: on average each infected person will infect more than one other person. Infections rise exponentially.

As time goes on, each infected person will meet fewer people who are not already infected, have become immune, or are otherwise resistant.

Eventually this reaches the ‘magical’ point where each newly infected person will, on average, infect less than one other person (the reproductive number is less than one). The infection will now eventually die away as people recover.

In Switzerland researchers have seen that the reproductive number dropped below one in mid March. Lockdown measures were put in place between 13th March and 20th March, but the reproductive number was already dropping and the measures (marked here as dotted lines from the 13th March) had little apparent extra effect :

We have to make do with what we have now, not what we would like to have later. An answer too late is no answer at all.

In Germany, RKI have been back-casting from case reports to estimate the actual dates of infection rather than the reported dates of the new cases. Using this they too estimated the reproductive number, which was already dropping before the partial lockdown on the 16th and fell below one a few days later before the strong lockdown on the 23rd:

This too strongly suggests that by the mid-late March the infection had already reached its peak extent.

Limited Sampling

The Intensive Care National Audit and Research Council produce a weekly report of COVID-19 patients in critical care. Up to the week ending 12 April, ICU use looked like this:

The curve has clearly ‘been flattened’, with arguably the turning point (where an increasing rate of increase became a decreasing rate of increase, ie the reproductive number dropped below one) around the end of March for the UK.

The number of available beds is not clear, which is not surprising given the awesome effort being made by so many different people and organisations to supply more of them. However apart from particular hotspots there are more empty critical and acute care beds available overall than normal.

Death Rates

While we have the numbers of people who died in hospital with Coronavirus from PHE this is not very useful without understanding who is tested and why. Nearly 1,000 people die per day in the UK either in hospital or within 30 days of leaving it. If a third of the population is infected and all inpatients are tested, we would expect a baseline of around 300 deaths ‘with’ CVID-19, no matter what they actually died from. It may be that all inpatients are indeed tested so that they can be isolated and the staff wear suitable PPE, but it is more likely that there is some discretion in testing so the baseline will be lower.

In any case we see a dramatic rise in the overall number of weekly deaths from ONS, and it seems likely that a major part of these are due to viral deaths outside hospitals. Here are the weekly figures, broken down by age, of the last few weeks and the first two weeks of the year during the flu season:

Comparing the last two weeks with the seasonal-flu height in the week ending 10th January (comparing with the averages over the year so far shows a similar pattern), we find:

There were no excess deaths under the age of 35; in fact there were a handful fewer possibly due to less travel. This does not mean that there were no deaths from COVID-19, but that they are not significant in the overall risks.
There were just over 120 excess deaths in the range 35-50 (although 40-44 year olds have had three worse weeks this year).
The rest of the excess deaths, as we might expect from all we have seen so far, were all in older people, peaking at nearly 900 this last week in the 80-84 bracket.

Previous highs include January 2018 which saw 15,000 deaths in one week, and January 2015 when over 16,000 died (7,387 in the over 85s) in one week.

Be aware that age here is probably a proxy for underlying causes. As we get older we tend to accumulate medical conditions and it is some of these that are likely the comorbidity factors.

Wrapping up, at last:

Weekly morbidity figures show a real, significant effect on our population, especially the old and frail. Intensive care audits show the strain on our health services. People who die with it die an unpleasant death. It is clear that this is not a minor illness or ‘like any flu’.

It’s also clear that it is not the looming monster it is sometimes portrayed to be. Many people are infected and don’t even notice, the risk to young and healthy people is negligible, and a large part of the UK population is already infected.

We can start to compare the harms of future virus infection with the harms of further ‘lockdown’. We can start to see what factors need to be compared with each other, and where to go looking for them – and what we need to ask our institutions to do better next time.

Your own take on this – the decisions and actions that you make – will depend on how you evaluate those harms, and how much you care about those who are harmed.

Some data analysis points to take away:

New cases are not new infections; they are discoveries of existing infections.
Reported deaths-with-COVID-19 are not actual deaths due to the virus.
Use proportions not absolute numbers.
Safety is not ‘on’ or ‘off’; Locking people up does not keep people safe. We need to identify and compare the relevant harms if we want to make informed decisions about safety.

Sources

Related Articles