We ran the same 40 prompts through 11 AI assistants across coding, writing, research, and math — and priced every one against what you actually get on the paid tier.
We graded each assistant on a 40-prompt test set across coding, writing, reasoning, factuality, and agentic tool use — blind-reviewed by three editors against a rubric.
Correctness on 10 multi-step reasoning prompts with verifiable answers.
Fraction of the 10 prompts where the final answer is exactly right.
Editor score for whether the model broke the problem into the right intermediate steps.
Whether the model carried intermediate results forward without forgetting or drifting.
Compilation + test pass rate on 10 coding tasks across TS/Python/Rust.
Fraction of generated snippets that compile/parse without edits.
Fraction that pass the hidden test suite on the first attempt.
Editor score for whether the code looks like something a senior engineer would write.
Editorial score on 10 long-form prompts, blind-reviewed.
Hallucination rate on prompts with verifiable facts.
Fraction of claims that are verifiably wrong when fact-checked against primary sources.
When the model cites, does the citation actually support the claim?
$/useful-response on the plan most users buy.
Reliability on multi-step tasks that require tool calls.
We pay for every AI subscription we test and refuse referral-linked pricing in our tables.
The most reliable for code review and long-form writing. Refuses to fabricate when it doesn't know.
Still the most versatile all-rounder. GPT-5 image generation and voice mode are genuinely ahead of the field.
Underrated for research tasks. Google Search grounding means fewer hallucinations on current events.
Microsoft's GPT-5-powered assistant. The only reason to pick it is the Microsoft 365 bundle.
Free across WhatsApp, Instagram, and Messenger — but Llama 4 still trails the frontier on reasoning.
Le Chat Pro is the best EU-hosted assistant and genuinely competitive on code and reasoning.
DeepSeek R2 is the price/performance leader — genuinely competitive reasoning at a tenth of the Opus price.
Grok 3 on X Premium gives you a capable assistant with real-time social signal baked in — if you're on X anyway.
Research-first search wrapper. Not really a generalist, but unmatched on cited answers.
Kagi bolted frontier models onto their ad-free search engine. Expensive but fast and private.
Early multi-model router that's been outpaced by Perplexity on UX and Kagi on privacy. Still a legitimate niche pick for model-switchers.
Reasoning and code scores landed within 4 points of Claude Opus at roughly a tenth of the effective price — the clearest budget pick in the category.
Reasoning accuracy up 3 points vs. Sonnet 4.5 test run; code quality now narrowly ahead of GPT-5.
EU-hosted, open-weights option, and competitive coding scores earned it a full 'recommended' verdict.
The shift from chat assistants to agents is real; we now score tool-use reliability explicitly.
| Service | Productivity tier price (Pro / Plus / Advanced), before any annual discount. | % correct on 10 multi-step prompts with verifiable answers. | Combined compile + test pass rate on the 10-task code benchmark. | Effective context window — what the model can actually use without quality dropping off, not just what the spec sheet says. | ModalitiesText / image / voice / video |
|---|---|---|---|---|---|
Claude Pro | 20 | 92 | 88 | 200 | Text + image input |
ChatGPT Plus | 20 | 89 | 85 | 128 | Text + image + voice |
Gemini Advanced | 20 | 86 | 80 | 1000 | Text + image + video |
DeepSeek R2 | 2 | 88 | 84 | 128 | Text |
Mistral Le Chat Pro | 15 | 84 | 82 | 128 | Text + image |
Perplexity Pro | 20 | 82 | 72 | 32 | Text |
Microsoft Copilot Pro | 20 | 88 | 84 | 128 | Text + image + voice |
Grok 3 (X Premium+) | 16 | 83 | 76 | 128 | Text + image |
Kagi Assistant | 25 | 90 | 85 | 200 | Text |
You.com Pro | 20 | 80 | 74 | 32 | Text + image |
Meta AI (Llama 4) | 0 | 74 | 68 | 128 | Text + image |
Reasoning and code scores are blind-reviewed against a 40-prompt evaluation set. Context is the effective working window, not the advertised max. DeepSeek price reflects the consumer app tier, not per-token API billing.
If you pick one and only one: Claude for writing/code-heavy work, ChatGPT for everything else. At $20/mo each, many people we know subscribe to both. Budget-constrained? DeepSeek R2 gets you within 4 points of frontier reasoning for roughly a tenth of the price.
For occasional use, yes. Every provider throttles the free tier below the paid model's capability — you'll notice the difference on anything nontrivial. Meta AI is the exception: it's free and the paid tier doesn't exist.
Blind evaluation against a fixed prompt set. We don't re-score every model every time a new version ships — we re-score the entire field once a quarter to make comparisons honest.
GPT-5 is currently the best integrated image generation. Standalone image tools (Midjourney, Flux) still beat it on aesthetic quality, but integration in a chat assistant matters for most users.
Depends on your threat model. EU users who need a data-residency story should look at Mistral Le Chat Pro. Chinese jurisdiction rules out DeepSeek for some enterprises. Kagi Assistant has the strongest privacy posture in the category.
Usually no — you're paying for the hosted experience, not the weights. It matters if you need to self-host for compliance (healthcare, defense, regulated industries) or if you want insurance that the model can outlive the vendor. Mistral and DeepSeek both offer open-weights options.
Every correction is logged publicly. Response within 10 business days.