Tony Blair Institute comes under scrutiny for AI study

23 July 2024 Consultancy.uk

The research group of former Prime Minister Tony Blair has come in for ridicule from technology commentators. The report presented arguments claiming AI could reduce the public sector workforce, but the analysis was heavily based on content generated by ChatGPT-4 itself.

Founded by former UK Prime Minister and Labour Party leader Tony Blair, the Tony Blair Institute for Global Change is a London-based think tank and consultancy, which has 60 staff in more than 30 countries. Established in 2016, the organisation focuses on political strategy, public policy, and technology expertise, according to the firm’s website.

The organisation has regularly appointed heavyweight political figures into its ranks, seeking to lean on their credibility. However, the Tony Blair Institute’s latest research seems to have chipped away at that veneer. The so-called think-tank outsourced the very work which should be its bread and butter, and tasked an artificial intelligence model to come up with answers for the hottest questions relating to itself.

Numerous media organisations have called the institute’s methodology into question, since the release of the study – presented initially by the organisation’s sainted founder himself. Rather than conduct an exhaustive study with workers and employers to determine how automation would affect each profession within a given market, they used an O*Net data set to identify 20,000 tasks performed by workers, and then fed the data to ChatGPT.

Contending that having human experts to go over each task – something research organisations have had to do without digital support for centuries – would be “intractable” (difficult to perform), the researchers prompted the AI to determine what tasks were suitable for automation and what tools could be used to automate them. One of the issues with this – beyond the institute shirking its one job – the report also noted that ChatGPT is infamously prone to hallucinations, but did not mention how “intractable” evaluating each of the technology’s outputs would be.

The team asked the AI system to categorise nearly 20,000 tasks. Assuming that there were mistakes within that data set – which is likely, given that OpenAI’s own website admits its models regularly make mistakes – then the research will contain faulty information, for which a peer review would have also been “intractable” for. 

But just as problematically, asking ChatGPT, an active player in the market openly looking for investments, whether it could be a useful tool would not stand up in any other context. It would be akin to asking a management consultant whether or not they could help a company find efficiencies. If there’s money at play, the answer will almost always be “yes”. 

When ChatGPT helps to formulate a case that technology like it could perform 40% of public-sector work in the future, that is not an impartial argument. It also comes with a conspicuous caveat that before that point where ‘massive savings’ will begin to manifest, AI will require huge injections of funding from the taxpayer. 

According to the Tony Blair Institute, AI benefits would require the government “to invest in AI technology, upgrade its data systems, train its workforce to use the new tools and cover any redundancy costs associated with early exits from the workforce.” For the next five years that would cost approximately $4 billion per year, while that would escalate to $7 billion per year after that. With reports from every business and news source in the UK and US now talking about the potential of a “bursting AI bubble” at the moment, such a sudden injection would be hugely fortuitous for owners of a technology which has so far delivered little in terms of profit, and depended on endless investment for survival.

The research has subsequently been the recipient of a growing amount of scrutiny, and mockery. Novara Media’s Aaron Bastani also took to X to point out that the Tony Blair Institute is a long way from being impartial either. According to Bastani, the largest doner of the institute is Larry Ellison, executive chair of Oracle – a leading provider of data centres which would likely benefit from the huge level of infrastructural investment needed to implement the proposed ‘AI-revolution’ in the public sector. Ellison is said to have already donated more than $100 million to the organisation.  

Elsewhere, Oxford University’s Mohammad Amir Anwar took to X that the Tony Blair Institute was “making shit up”; while, the University of Washington’s Emily Bender told 404 Media that researchers “might as well be shaking at Magic 8 ball and writing down the answers it displays.”

It has also been noted how the institute’s founder and namesake doesn’t have a fantastic track-record when it comes to tech-bubbles. Two years ago, amid the hype of the crypto-bubble, Blair was sitting on a stage in the Bahamas with Bill Clinton and the apparent cryptocurrency wunderkind Sam Bankman-Fried. Months later, Bankman-Fried had been arrested and extradited after he appeared to hedge risks with the FTX coin that he issued himself. News coverage also suggested he had used about $4 billion from investors to absorb losses at his trading house, Alameda.