DoubleChecks

What was claimed

In December 2024 o3-preview scored 87.5% on ARC-AGI-1 at >$4500 per task. In July 2026 DeepSeek-V4-Flash scored 87% at $0.021 per task. By 2028 open-source models will crack hard math problems in under an hour; the mathematics profession will have to change

Our verdict

Needs caution

DeepSeek V4 Flash 0731 scored 89.0% on ARC-AGI-1, not 87%. This is a 2-point factual error. Benchmark analyses describe o3’s high-compute ARC-AGI-1 runs as costing thousands of dollars per task, but reported cost-per-task figures vary, with one detailed table giving around $4,560 per task and others giving broader ranges. The claim’s ">$4500" threshold is directionally correct but presented as a precise figure despite these discrepancies. (Only 2 of 3 AI systems responded.)

Combined assessment17 sourcesChecked Sep 12, 2026

Why this verdict

  • In July 2026 DeepSeek-V4-Flash scored 87% on ARC-AGI-1 at $0.021 per task

    Misleading
    One assessment reached this conclusion; one differed·Claude: Incorrect

  • In December 2024 o3-preview scored 87.5% on ARC-AGI-1 at >$4500 per task

    Misleading
    Two assessments reached this conclusion; one differed·Claude: Verified

  • By 2028 open-source models will crack hard math problems in under an hour; the mathematics profession will have to change

    Can’t verify
    All assessments reached this conclusion

  • In December 2024 o3-preview scored 87.5% on ARC-AGI-1

    Verified
    All assessments reached this conclusion

Evidence reviewed

17 sources

Perplexity and Claude sources are combined here.

arcagibook.comarcprize.orgtechcrunch.comaiwiki.aiashitaorbis.comrdworldonline.comvalueaddvc.comx.comtechmeme.comfortune.com

Individual assessments

Assessment summary

  • The claims about o3-preview’s ARC-AGI-1 score and very high per-task cost are broadly consistent with published benchmarks, though the exact dollar figure is not stable across sources.
  • The DeepSeek-V4-Flash numbers differ slightly from at least one benchmark report, and the 2028 prediction about math and open-source models is speculative and not verifiable.
  • Overall, the statement mixes accurate benchmark facts with approximate costs and an unverifiable forecast.

Claims reviewed

In July 2026 DeepSeek-V4-Flash scored 87% on ARC-AGI-1 at $0.021 per task

Misleading
At least one benchmark-style review reports DeepSeek V4 Flash reaching about 89% on ARC-AGI-1 Semi-Private at roughly $0.02 per task, which is close but not identical to 87% and $0.021. The claim appears to approximate these numbers rather than reflect an exact, cited result.

This claim may mislead readers without added context.

In December 2024 o3-preview scored 87.5% on ARC-AGI-1 at >$4500 per task

Misleading
Benchmark analyses describe o3’s high-compute ARC-AGI-1 runs as costing thousands of dollars per task, but reported cost-per-task figures vary, with one detailed table giving around $4,560 per task and others giving broader ranges. The claim’s ">$4500" threshold is directionally correct but presented as a precise figure despite these discrepancies.

This claim may mislead readers without added context.

By 2028 open-source models will crack hard math problems in under an hour; the mathematics profession will have to change

Can’t verify
This is a forward-looking prediction about capabilities and social impact, not a current factual statement. There is no authoritative, empirical evidence that can confirm or invalidate this specific forecast at present.

Assessment summary

  • The statement contains one factual error (DeepSeek-V4-Flash score is 89%, not 87%) and two unverifiable future predictions.
  • The o3-preview claims are essentially accurate, though the cost notation is slightly imprecise.
  • The future claims about 2028 and mathematics profession changes cannot be verified.

Claims reviewed

In July 2026 DeepSeek-V4-Flash scored 87% on ARC-AGI-1

Incorrect
DeepSeek V4 Flash 0731 scored 89.0% on ARC-AGI-1, not 87%. This is a 2-point factual error.

This claim may mislead readers without added context.

o3-preview cost >$4500 per task

Misleading
The cost was approximately $4,500 per task, not exceeding it. The '>' symbol suggests a higher cost than the actual estimate.

This claim may mislead readers without added context.

The mathematics profession will have to change

Can’t verify
This is a speculative claim about professional consequences that cannot be verified from available sources or evidence.
ChatGPTDid not respond

Check your own claim

Paste any statement, headline, or AI answer. Independent assessments check it and compare sources.

Share this result