Technology / Nature

Add as a preferred source on Google

UNESCO Study Finds AI Models Near Chance on Combined Climate Policies

Repeated user challenges lowered accuracy by 31 to 35 percentage points across 15 models, while a separate test found lower average accuracy in Swahili than in English.

Updated 2 min read
Reporters attend a climate AI study briefing at the COP31 venue in Antalya, Türkiye. / EconoTimes.ai
Reporters attend a climate AI study briefing at the COP31 venue in Antalya, Türkiye. / EconoTimes.ai

UNESCO and University College London found AI models scored near chance when assessing combined climate policies, raising global questions about their reliability in climate decisions.

Researchers checked model responses against the En-ROADS climate simulator, which provided a benchmark for the policy questions. The accuracy score for questions about policy combinations was 33.8%, near the 33.3% associated with random guessing. UNESCO and UCL presented the study Nov. 16. Models had answered basic climate questions accurately, but that performance did not ensure dependable analysis of combined policies.

User pressure exposed another weakness. UCL reported accuracy drops of 31 to 35 percentage points across the 15 models as users repeatedly disputed correct replies. A single objection preceded the reversal in roughly four out of five cases where a model eventually changed its correct answer. UNESCO linked the pattern to training incentives that reward agreement with users. The finding shows why a model’s initial answer alone may not reveal its susceptibility to pressure.

The five-model language comparison found average accuracy of 88.6% for English and 78.3% for Swahili. UNESCO said speakers of some languages with weaker results live in regions disproportionately affected by climate change, making model reliability relevant to communities facing greater exposure.

“Recalling facts is not the same as being able to provide reliable policy analysis,” said Leona Verdadero, a Programme Specialist in UNESCO’s Digital Policies and Digital Transformation Section and co-author of the report.

On Nov. 13, the Global Initiative for Information Integrity on Climate Change, co-chaired by Brazil, the United Nations and UNESCO, issued priorities for COP31 focused on AI’s role in climate information integrity. The priorities ask AI companies to examine their products’ effect on climate information and share findings publicly. The announcement came three days before UNESCO and UCL presented their study on Nov. 16. It also cited the earlier COP30 Belém Declaration.

Related articles

View all