An arXiv study by Google-affiliated researcher Nghi D. Q. Bui found that a checklist sharply limited mistaken edits to correct Gemini code, while an on-demand approach performed worse in pooled coding tests.
Across the five Gemini models, checklist agents later broke 0.02% of code requirements they had already satisfied after an incorrect objection, compared with 12.2% for standard agents. The counts were one reversal in 6,130 requirements for the checklist group and 744 in 6,074 for the standard group.
Thinking-in-Bets takes its name from Annie Duke’s 2018 book. Before feedback, agents note their claims, supporting checks and confidence, then refer to that record after an objection to decide whether to stand by their work, investigate or revisit an uncertain point. The preprint says the approach targets a tendency to discard earlier checks or agree too readily with criticism.
Standard agents led On-Demand Betting in pooled mid-session tests of four Flash models: 35 to 29 task passes out of 80, and 25 to 18 fixes among 45 and 39 valid reports, respectively. On-Demand Betting used 85% fewer checklist tokens than Always-On Betting.
A separate comparison ran Gemini 3.8 Flash twice: On-Demand Betting repaired 17 of 23 valid reports, while the standard agent repaired 12. For task completion, the on-demand method reached 21 of 40; the standard agent reached 23.
Always-On Betting produced 89 passing deliverables out of 100 in Agents’ Last Exam’s review of reports and calculations; standard agents produced 58. Standard agents repaired 24 of 25 flawed drafts, compared with 23 of 25 under Always-On Betting.
Scores on the closed-book Vals Finance Agent Benchmark for financial analysis reached 34.2% for Gemini 3.8 Flash, up from 29.0%, and 52.5% for Gemini 4.0 Argon, up from 44.6%. Submitted to arXiv Oct. 26, 2026, the preprint assessed eight public benchmarks spanning coding, academic reasoning and command-line work.
The tests used simulated users who generally made brief objections without providing a failing example. Bui chose thresholds manually; 25 of 85 headline comparisons remained statistically significant after a Holm adjustment. The training-free prompting experiment aimed to preserve verified work, the preprint says, but may also make agents less likely to fix genuine bugs in unfinished code.
