Inside Bloomberg’s Covid Resilience Ranking

(Bloomberg) -- Everyone is fighting the same coronavirus, but about a year into the pandemic, quality of life and control of the pathogen’s spread look vastly different across the world.

Bloomberg’s Covid Resilience Ranking scores the largest 53 economies on their success at containing the virus with the least amount of social and economic disruption.

We considered many datasets, indicators and indexes produced by organizations around the globe and applied three fundamental criteria in whittling the list down to the 11 components that make up our Ranking:

• How complete is the data?
  • Many relevant indexes and databases -- for instance, measures on trust in government -- cover only a small number of places. We focused on indicators that cover the vast majority of the 53 economies in the Ranking, filling in the gaps with substitutions where reasonable.
• How current is the data?
  • All datasets have a lag, some of up to a few years. Because of the pandemic’s pace and transformational impact, we chose to use the most up-to-date datasets where possible, with the maximum lag one year. Nine of our indicators are refreshed daily, while two are annual figures.
• Who collects the data?
  • We decided to only use data compiled by Bloomberg or indicators from reputable third-party organizations with a track record of collation and analysis.

In the January edition of the Ranking, a new indicator -- Vaccine Doses Given Per 100 People -- was added to reflect the progress of inoculation drives across the world.

Why only rank 53 economies?

We decided for brevity and relevance to limit the Ranking to economies valued at more than $200 billion prior to the pandemic.

How is the Ranking aggregated?

Each of the 11 data indicators are aggregated through the “max-min” method, which is used to convert metrics expressed in different scales into a common one, while maintaining the relative distance between values.

All the indicators are scored on a 0-100 scale, with 100 (blue) indicating the best performance and zero (orange) the worst. The rest fall in between, scaled by their distance from one another. The final Bloomberg Resilience Score is the average of a place’s performance across the 11 indicators, equally weighted.

The final score given to each place is a relative measurement on a given date. That score shouldn’t be compared in isolation to the economy’s previous scores as the max-min ceiling and floor values change in every update.

The Ranking is updated once a month.

Can Covid-19 data be trusted?

Under-detection, under-reporting and manipulation of virus data on cases and deaths have been recurring issues across many economies during the pandemic.

In most places, gaps in the data are largely due to the chaotic and fast-moving nature of the crisis: the supply of test kits has been inadequate, leading to under-detection of cases. Official death tolls are also likely to be under-reported due to people dying at home before being diagnosed, lags in reporting from overwhelmed hospitals and Covid-19 deaths being recorded in some places as due to other causes.

Countries like Spain, the U.S., China and Turkey have adjusted their numbers throughout the pandemic, and credible reports have emerged that some nations, including Iran, Brazil and Russia, have intentionally concealed or downplayed data. Beyond that, excess mortality in some economies this year -- the additional number of deaths overall compared to previous years -- has outstripped official Covid-19 death tolls.

Still, given that it’s hard to distinguish between slow reporting, inadequate resourcing and the intentional concealment of data, we’re using cases and deaths data compiled by Johns Hopkins University, which draws largely from government sources, with the awareness that this is likely not the full picture.

A note on China’s numbers, which have been called into question by the U.S. and others. Throughout the pandemic, the country has repeatedly adjusted its virus data, adding nearly 15,000 new cases in one day in mid-February and raising its overall death toll by 40% to 4,632 in April. These revisions arguably make its latest numbers more reliable. Researchers have used fraud detection techniques to conclude that, while China did manipulate its data in the early stages of the pandemic, its numbers have been accurate since.

Why do you have two datapoints on virus deaths?

The trailing one-month case-fatality rate is a good indicator of whether a place is effectively treating infected people and preventing Covid-19 deaths, a fundamental aspect of containing the virus. Over time, this ratio has improved among mostly developed economies as doctors and hospitals learned how to better fight the coronavirus.

The rate also captures the point economies are at on their individual pandemic curves. Australia had a high case-fatality rate during a brutal southern hemisphere winter wave that triggered a three-month lockdown in the country’s Victoria state. As of December, with that outbreak largely quelled, the rate has fallen to a relatively low level. Correspondingly, the case-fatality ratios for nations in the northern hemisphere have deteriorated after they entered the cold season.

The one-month case-fatality rate doesn’t, however, capture the scar that the pandemic leaves on an economy as a whole, especially in places like Belgium, Sweden, Italy, the U.K. and the U.S., where Covid-19 cut a swathe through elderly populations in its initial phase in the spring. A higher percentage of Sweden’s population died from the virus than Vietnam’s, even if Sweden’s ability to save the lives of Covid-19 patients has since improved. This is why we’ve also included an indicator that reflects cumulative Covid-19 deaths as a share of the total population: Total Deaths Per 1 Million.

Economies with older demographics generally rank lower on this measure, given the way the virus ripped through aged-care homes in its first wave. In Belgium, officials decided to classify every nursing home death as being from Covid-19, even if the patient wasn’t officially diagnosed before they died. This approach has made Belgium the bottom-ranked economy on overall mortality though it’s not bottom-ranked in the one-month case-fatality ratio indicator.

What does the Positive Test Rate show?

The positive test rate is considered by experts as the most reliable way to determine if a place is testing enough.

A high rate of virus tests coming back positive indicates authorities are probably only testing the sickest patients who seek out medical attention and are not casting a wide enough net. It’s a sign there is likely to be undetected infections in the community. World Health Organization guidance is that governments should wait for the positive test rate to fall below 5% for at least 14 days before relaxing social distancing measures.

For economies that don’t report daily positive test rates, we derived their rates by dividing the number of cases over total tests administered on the last date that data was disclosed. We did this for China, Hong Kong and Vietnam. The correlation between an economy’s daily positive test rates and this measure is high, giving us confidence to substitute these values.

China’s latest-reported overall testing number is from August and officials have not disclosed a more recent positive test rate figure. However, its latest flareups confirm that the figure continues to hover close to zero in China: in the northern city of Shijiazhuang, the population of about 11 million was tested twice with over 600 cases found.

Our World in Data doesn’t collate figures for Egypt and Brazil. These two nations, therefore, don’t have scores for this indicator in Bloomberg’s Ranking.

One caveat on this dataset is that some places report the total number of people tested, while some report the total number of tests. The possibility of multiple tests being administered to the same person in a single day could mean they’re not exactly apples-to-apples, but these differences are not significant in scenarios where millions of tests are being conducted daily.

What are you measuring with the Access to Covid Vaccines indicator?

With an effective vaccine seen as the silver bullet that could end the pandemic, securing enough supply is an important component of an economy’s Covid-19 response.

Some places, like the U.S., have stated openly that their strategy is not to necessarily contain the virus’s spread, but to use vaccines and treatments to eventually quash it. Given that widespread access to a viable vaccine could lead to a quick neutralization of cases and drop in deaths, future supply is an important component of the Ranking. Consequently, this indicator has bolstered the ranking of some places where the virus is surging -- i.e., the U.K. and Canada.

In the first edition of the Bloomberg Covid Resilience Ranking published on Nov. 24, the Access to Covid Vaccines indicator tracked the number of supply agreements each place had signed based on information compiled by Duke Global Health Innovation Center and our own research.

As governments disclosed more information on their negotiations and order sizes, this has been replaced since December with a Bloomberg tracker that collates how many vaccine doses each place has ordered and what percentage of their populations is covered by those deals. Orders of vaccines that require two doses per person are halved in the calculation.

The tracker captures supply and manufacturing contracts where the number of doses has been disclosed. It counts only confirmed orders, but not optional top-ups that many places have also announced. For orders made by entities representing multiple economies like the European Union, the Carlos Slim Foundation, or the global, WHO-backed alliance Covax, we assigned the doses to every place within the group equally as a share of population.

Read more: More Than 56.6 Million Shots Given: Covid-19 Vaccine Tracker

There’s no drastic difference in the Ranking due to the shift from tracking number of agreements to dosage adequacy as the underlying indicator: the same economies, like Canada and the U.K., top both measures. Yet it does reveal the greater challenge facing the most populous countries like China, India, the U.S. and Indonesia. While China and India have orders or production capacity of over two billion doses annually, they still cover less of their populations than many smaller places.

As economies strive to inoculate their populations as quickly as possible and competition grows over limited supplies, this emphasis on dosage adequacy more accurately reflects how well places are positioned in their vaccination efforts.

Why is the vaccine access data different for some places?

Bloomberg’s vaccine tracker counts only publicly-disclosed, commercial orders made between governments and companies. However, this leaves out places where governments are reliant on domestic, state-controlled production -- namely China and Russia -- or where they have refused to disclose order size.

We have adjusted for this where credible public information is available.

China has five vaccine candidates in Phase III clinical trials. We’ve tallied the annual production capacity of the furthest-along vaccines and deducted the overseas supply deals they’ve signed to reach an approximate level of coverage for the domestic population.

Similarly for Russia, we’ve used domestic production capacity as disclosed by the government to approximate its level of vaccine coverage. We did not deduct overseas supply deals from domestic production capacity, like with China, as Russia has said that other countries will be supplied from its production hubs abroad, and most of domestic capacity will be for local needs.

The Singapore government says it will make vaccines available to all adult citizens and long-term residents in 2021, with shots procured from companies including Sinovac Biotech Inc., Moderna Inc. and Pfizer Inc. But officials refused to disclose the dosage size of Singapore’s orders. Therefore, we assigned it a dosage adequacy value equal to a gauge of its adult population as a share of total population.

Why did you add a measure tracking vaccines administered?

Starting in 2021, we have added an 11th indicator named Doses Given Per 100 People to reflect the progress of inoculation drives across the world. This represents a different aspect of the vaccine race from Access to Covid Vaccines, as places that have inked big deals may still face logistical challenges and myriad delays in actually getting the shots to people.

Israel and the UAE, the two economies that have been the fastest at administering doses, were not the places with the biggest orders. Yet supply adequacy remains important, as some governments are already facing shortages after quickly administering the doses they have.

It’s noteworthy that Russia only reports the number of people who have received at least one dose; it doesn’t disclose the total number of doses administered, which would be higher. Meanwhile, Austria reports a calculated number of vaccinated persons based on the deliveries of doses to vaccination facilities.

What does Lockdown Severity measure?

This indicator is based on an index produced by the University of Oxford, which assesses the number and strictness of government policies that limit people’s movements as a way of containing spiraling outbreaks.

We’re interpreting restrictive government policies as a negative in the Ranking as the stricter the lockdown, the more disruption people are experiencing. Taiwan is among the top-ranked for this indicator as the population is subject to almost no restrictions, while European countries, many of which have had to impose repetitive lockdowns, rank lower. A higher score on Lockdown Severity denotes a less advantageous performance.

Some may argue that strict lockdowns should be viewed positively, a sign a government is acting aggressively to control the virus. But nearly a year, and multiple waves, into this pandemic, the need to impose lockdowns reflects a failure to contain and manage Covid-19, and so we’re scoring it accordingly.

Stringent restrictions also correlate with the mental and economic toll of the virus on a population. Social disruption and isolation have been linked to higher suicide rates in some places, while school closures are raising concerns over child development and increases in hunger and drop-out rates among disadvantaged families.

Does Lockdown Severity reflect conditions across an entire economy?

The Oxford University index reflects the most stringent conditions in place in a given economy, regardless if those curbs are just being imposed in a specific region. That means a lockdown in one city or area will be the basis for an economy’s overall score.

This naturally penalizes large, expansive countries like China, India and even the U.S., where conditions can vary greatly from region to region and city to city. China’s conditions are scored as very stringent on the Oxford index because some cities and districts are in strict lockdowns as local officials fight a resurgence in the north.

But the majority of China’s 1.4 billion population face barely any restrictions, and government officials expect 1.7 billion trips to be taken over the upcoming Chinese New Year holidays.

Still, while imperfect, we view Oxford’s economy-wide approach as a proxy for how aggressively governments are likely to react if an outbreak emerges. China’s playbook this year has been to impose some of the most oppressive measures in the world, from bans on people in specific areas from leaving their homes to mass, obligatory testing. So while life is relatively relaxed for many in China right now, that could shift abruptly if even one case is identified where they live or work.

What does Community Mobility show?

Given the broadness of the Lockdown Severity indicator, and the fact it reflects government policy and not its impact, we sought out another datapoint to better capture this picture. Google’s Covid-19 Community Mobility Reports, which underpins our Community Mobility measure, tracks people’s real-time movements and helps round out our understanding of how they are responding to virus restrictions in their everyday lives.

We track movement to and from retail, recreational and work places, taking a 30-day average to smooth out the effects of holiday periods. The closer movement levels are to the economy’s pre-pandemic baseline, the higher the score on this datapoint as part of Bloomberg’s Ranking.

Places where Stringency and Mobility scores are at odds reflects a lack of official ability to enforce restrictions, and a lack of compliance among the population.

As Google data is not available for mainland China, we used an estimate from an activity tracker created by Bloomberg Economics -- used in a weekly analysis of activity in 26 major economies -- to derive China’s corresponding score. The correlation between Google’s data and Bloomberg’s daily activity tracker for the other 25 economies is high, giving us confidence in using it as a substitute for China.

Google doesn’t track mobility data for Iran, either, and we couldn’t find a viable alternative source, so Iran doesn’t have a score for this indicator.

What does the GDP measure track?

When we first launched the Ranking in November, this indicator was based on the International Monetary Fund’s annual gross domestic product forecasts for 2020. The greater the expected contraction, the more challenging the economic reality is for people in these places, and therefore the weaker their performance on this measure.

China is an example of how largely eliminating the virus locally can lead to a rebound in economic activity. But its vast domestic market of more than 1 billion consumers means it’s an outlier. For almost every other place, containment and economic growth are in a complex trade-off relationship.

Places like Singapore and New Zealand have nearly eliminated virus transmission among their populations, but at a massive cost to their economies, which are reliant on tourism and travelers from abroad.

In 2021, we refreshed this indicator to projections for the year ahead. We also swapped out the IMF’s forecasts for a consensus figure compiled by Bloomberg from economist surveys, as this is more up-to-date than the IMF’s quarterly-updated number. For places with less than five economist forecasts, we revert to the IMF’s latest-available estimates instead. This was done for Bangladesh, Egypt, Iran and Iraq.

Our GDP measure reflects the percentage contraction or expansion projected this year in the 53 economies. We chose to use this indicator, despite its bias towards emerging, high-growth economies, instead of the “swing” between the current forecast compared to the pre-pandemic figures. This is because the Ranking aims to show the best places to be right now, not just how well they’ve deflected the economic blow from Covid-19.

For India, the forecast used refers to the year ending March 31, 2022, in alignment with the government’s fiscal cycle.

Why is the state of pre-pandemic health care relevant to Covid-19?

The Universal Healthcare Coverage indicator draws from a 2019 dataset published by the Institute for Health Metrics and Evaluation published in the Lancet in August. It maps the effectiveness of 23 preventive and treatment measures, ranging from access to basic non-Covid vaccines to cancer care.

While the dataset was produced prior to the onset of the pandemic, it reflects an economy’s ability to effectively prevent, detect and treat illness across a population. Bloomberg’s deduction is that places with higher scores on this measure are in a better position to prevent deterioration and death in Covid-19 patients, and more able to maintain the provision of non-Covid related healthcare during the pandemic. Hong Kong isn’t tracked, so it doesn’t have a score for this indicator.

There are various measures and datapoints out there tracking the strength of health-care systems, including from the WHO. We chose this indicator due to its completeness: 204 countries and territories are tracked. It’s also the most up-to-date of those we surveyed.

What’s the point of including the Human Development Index?

This indicator, produced annually by the United Nations Development Programme, has three parts: life expectancy at birth, years of schooling, and wealth per capita. The three components were chosen to represent the overall well-being of a society.

Scores since the December Ranking reflect the recently-released 2020 report.

Like the Universal Healthcare Coverage datapoint, this measure captures an economy’s pre-pandemic performance. Still, the Human Development Index reflects a society’s ability to withstand the Covid-19 blow, and can be a proxy for how populations have reacted to the crisis:

• Years of schooling reflects access to education and acts as a proxy for a population’s trust in science, which experts say is a key determinant in whether people follow public health guidance on social distancing and mask-wearing. We considered other measures of trust in science, such as the proportion of STEM degrees among degree-holders, but didn’t find a suitably comprehensive and up-to-date alternative dataset.
• Wealth per capita reflects people’s income, adjusted for purchasing power.
• Life expectancy is a proxy for whether access to health-care is equitable across a population.

The UNDP doesn’t include Taiwan in this indicator, but the government there calculates its own Human Development score every year using the same methodology. We use Taiwan’s self-published score for this measure.

Was the Ranking vetted?

The Covid Resilience Ranking is the result of months of sifting through various information sources by Bloomberg reporters and data specialists. It was developed in consultation with experts in the data collation, economic and scientific fields. While the Ranking is naturally a subjective measure, we think it captures fairly and comprehensively the best and worst places to live and work right now as we continue to ride out this pandemic.

It will morph and be updated as circumstances change -- stay tuned.

(Updates throughout, including the addition of the Vaccine Doses Given indicator and the refreshing of the GDP growth forecast metric for 2021.)