Trust the Algorithm — Don’t Think

9 min readOct 4, 2020

A Covid-19 Lesson For A Future We Can’t Avoid

“At the highest level of authority, we will probably retain human figureheads, who will give the illusion that the algorithms are only advisors and the ultimate authority is still in human hands… Within a couple of decades, politicians might find themselves choosing from a menu written by AI” — Yuval Noah Harari (2018)
“one of the most comprehensive modelling exercises that we have ever seen in the state, arguably nationally, and a piece of work that has international significance as well…You can’t argue with this sort of data, you can’t argue with science, you can’t do anything but follow the best health advice” -Daniel Andrews, Premier Of Victoria (2020)

Melbourne, the capital of Victoria, Australia is the victim of the longest COVID-19 lockdown on earth (150 days and counting of stay at home orders)¹. Initially a model of coronavirus excellence, Victoria has lapsed into a health and economic disaster. Given sweeping emergency powers to create and enforce new laws and a near-infinite budget, the premier Daniel Andrews was limited only by his imagination and the bureaucracy he inhabited.

In daily briefings, Andrews, a science adoring leader, emphasised that the laws and steps to recovery would be driven by data and by the science. Being in Melbourne, a hub of Biotechnical and health research knowledge, Andrews had access to the sharpest minds, world-class modelling and seemingly the best advice. Where did it go wrong?
One can point (and most of the media do), to the long list of mistakes and scandals that have plagued his administration. Lying, lack of accountability, lack of competence have been endemic as his small team have bulldozed everyday life with harsh and enduring lockdown laws.

Modern, narrative-driven journalism has focused on the story of incompetence and corruption. The Australian, Murdoch’s right-leaning newspaper is full of reasons not to trust the government and provides a shopping list of examples and a coherent story that is repeatable in a pub. Asa consequence it misses the story of the day — science.

Modelling and Science has been used to justify political decisions since the beginning of the pandemic

The focus upon data and science has been at the centre of the virtue signalling perpetrated by politicians and media alike. The issue lies not with the models but how journalists and politicians with no scientific expertise misrepresent and simplify to generate a coherent narrative.
Time and time again throughout the pandemic, common sense has outperformed these complicated and intricate models. We are right to see the hypocrisy and doubt the validity of these models.

The Prime Minister, Scott Morrison famously said “What I can’t help but be struck by, is that under the thresholds that have been set in that plan Sydney would be under curfew now. Sydney doesn’t need to be under curfew now. They have a tracing capability that can deal with outbreaks.”²
Additionally, those for an eye for detail would have noticed that during the period April 27th, until May 2nd, the 14-day rolling average was below 5, the limit which the modelling predicts there is only 3 per cent chance of case that the state would have to go back into lockdown³. The entire state went into lockdown for three months just months later for this threshold was reached.

“All models are wrong, some are useful “ — George E. P. Box

This quote is at the centre of the issues regarding the modelling in question. When predicting complex and chaotic phenomena as is the case here, it is more important to know the confidence of the prediction than the prediction itself.

If your mate says, I think this horse will run second and I’m 95% sure it will finish between 1st and 13th, you wouldn’t bet your house on their latest “hot tip”.
Confidence intervals provide a range of outcome which your model is 90% certain will contain the result. In an agent-based, stochastic model, confidence intervals can be easily calculated. These confidence intervals are misleading because they illustrate the likelihood of an outcome occurring within the model, not in real life.

Example how an “Agent” navigates social distancing rules.

Within these models, people are reduced to a series of if-else statements governed by random actions. It is absolutely clear to anyone that this is NOT an accurate representation of reality. It completely neglects how people interact, how social norms propagate and shift, how viral ideas spread online and how fed up people become when they are locked in their rooms for 6 months at a time. Hundreds of approximations and assumptions are made, and they all add up.

Small List Of Assumptions that were required to be taken

The researchers justify these assumptions by assuring us the model is optimised based on the data from the previous lockdown. But common sense tells us that you can’t just look at what happened last time and assume it will play out the same way. Sports fans know this is the height of stupidity. The technical term for this is an “ergodic process”, or somethings whose properties are visible to an observer and do not change over time. A pandemic is not this. It’s an absolute clusterfuck.

The only way we can gauge the true accuracy of the model is by comparing its predictions to real life. When scientists fail to compare these predictions to reality, they are not doing science, they are practising a religion.

“It doesn’t matter how beautiful your theory is, it doesn’t matter how smart you are. If it doesn’t agree with experiment, it’s wrong” — Richard Feynman

The only prediction completed with the current model that was published ahead of time predicted with 95% confidence that in two weeks the rolling average of infections would be between (52 and 73). Not only did the model overestimated the result by 250 cases (30%), but also predicted that the observed outcome was so unlikely it was virtually impossible.

A glance at the black line reveals almost immediately that what happened was not a freak accident. It seems that the only thing a PhD in virology, a supercomputer and the worlds most sophisticated models are good for, is to turn smart people into slaves of what they don’t fully understand.

“A prophet is not someone with special visions, just someone who is blind to most of what others see” — Nassim Nicholas Taleb

In standard scientific journal p<0.05 (5% chance the hypothesis is true given the evidence) is used to gauge whether the results of an experiment is meaningful. This is the exact opposite scenario. The first experiment seems to indicate that this modelling cannot even satisfy p<0.95.
This is not rigorous, proven science.

Despite the fact the model seems to be no better than a dart-throwing monkey and predicting outcomes two weeks in advance, it is being used by politicians to predict outcomes months in advance.

IN 1961, Edward Lorenz discovered that tiny deviations in the initial conditions would eventually cause wildly different weather predictions. People are much more unpredictable than the weather. A single security guard, tasked with guarding infected travellers was the cause of the proceeding outbreak that infected thousands!

Butterfly Effect: Just as Edward Lorenz you cannot predict the weather indefinitely into the future, the same applies to pandemics.

Fortune cookie science is not the key to guiding society towards a successful and prosperous recovery as we wait up to a year for a vaccine to arrive. Politicians are the masters of obfuscation, and many have not had the responsibility to make significant decisions and bear the consequences.

Unlike politicians, any small business or entrepreneur knows not to engage in fortune-telling when you can engage in fortune shaping. The first step to opening a successful restaurant is not to predict profits 5 years advance or scope out another location for expansion. The first step is to perfect your recipe.

Instead of investing in and improving the tangible and long-lasting infrastructure that will enable a strong economy and society during the pandemic, the government has spent most of the time trying to engage in fortune-telling.

While the boffins had their heads buried in the models, it took until 5 million people were an entire month in their second lockdown to mandate mask-wearing indoors⁴. It took until late august, and months of lobbying by the Australian Medical Association, for the state contact tracing system to replace paper note-taking and fax machines with 21st-century technology⁵. It wasn’t until the 22nd of July that Victorians were reminded to be cautious after surveys revealed: “9 in 10 — or 3,400 cases — did not isolate between when they first felt sick and when they went to get a test.”⁶ It wasn’t until 10 days later that the government thought it a good idea to check on positive cases. They found that 25% were not at self-isolating at home when the police popped by to check. Most worrying of all, only 3/4 were confirmed to have been referred to police⁷.

It was only after completing these very simple steps that the number of cases started to decline. The billions spent to prop up the economy during lockdown could have been invested in infrastructure and technology that creates jobs, increases innovation and reduces the economic and health burden of the state.
Instead, the government is throwing billions down the drain while waiting for the case levels to drop below a level decided by a supercomputer that can’t predict the future.

This is not to say that these models do not have a role to play. They are incredibly useful tools that have the potential to guide policy. They are not the panacea.

Algorithms, supercomputers and super-human data crunching are now a part of everyday life. Whether it be predicting peoples behaviour on Facebook, building self-driving cars or tackling climate change, these algorithms are here to stay. More than half of all trades on the stock market are completed without human supervision⁸. Politicians around the world are now required to be expert data-analysts when evaluating economic dependencies, climate change and tackling the next global emergency.

Currently, humanity exists in an uncanny valley, where we can see both the power and promise of technology but do not have the proper experience in understanding the risks. History is full of examples of naive scientists ruined by false assumptions and a belief that they understood what was fundamentally unknowable. When dealing with complex systems, Lorenz taught us that there is only one way two expose the truth. It must be exposed to the winds of history and observed.

Without the false comfort of omnipotent models, policy-makers will be forced to stop talking and start building. Concrete actions, common sense and transparency will always outperform complexity, opacity and other such bullshit.