All Runs · ScoreboardModel Performance History
| Date | Config | Win rate | Recovery | GPQA acc | MMLU acc | Debug acc | Questions | Cost |
|---|---|---|---|---|---|---|---|---|
| Loading scoreboard… | ||||||||
Complete Analysis · SuperGPQA Hard (35 q, seed=42)All Model Pair Configurations
Same 35 questions across every model pairing. Key insight: orchestration value scales inversely with the lead model's solo accuracy — the weaker the lead, the more room for challengers to recover mistakes.
| Configuration (Lead → Challenger(s) → Lead) | Lead solo | Challenger solo | Orch accuracy | Lift vs lead | Win rate | Recovery |
|---|---|---|---|---|---|---|
| GPT-5.2 → Claude 4.5 → GPT-5.2 | 14.3% | 20.0% | 48.6% | +34pp | 87.5% | 46.7% |
| GPT-5.2 → [Claude 4.5 + Grok 4.1] → GPT-5.2 3-model | 14.3% | 17.1% / 57.1% | 60.0% | +46pp | 86.4% | 63.3% |
| Grok 4.1 → GPT-5.2 → Grok 4.1 | 57.1% | 17.1% | 57.1% | 0pp | 50.0% | 20.0% |
| Grok 4.1 → Claude 4.5 → Grok 4.1 † | 68.6%† | 22.9% | 65.7% | −3pp | 33.3% | 9.1% |
† Grok 4.1 showed 68.6% solo in this specific run vs ~57% in other runs (API-level non-determinism at temp=0). Win rate = Connect Next wins ÷ (wins + losses), ties excluded. Recovery = % of lead's wrong answers that orchestration corrected.
Finding: Orchestration lift scales with the lead model's error rate
GPT-5.2 as lead (14% solo) → Connect Next recovers 63% of errors, +46pp accuracy lift. Grok 4.1 as lead (57%+) → fewer errors to recover, challenger critique has minimal impact. In production, Connect Next is most valuable when pairing a fast/cost-effective lead with a specialized challenger for quality review.
Latest Run · Overall PerformanceWin Rate
Visual SummaryPerformance Charts
Win Distribution
How often multi-model orchestration wins vs single model
Accuracy by Category
Multi-model vs single model accuracy per benchmark type
Contextvs Published GPT-4o Scores
Connect Next vs published GPT-4o scores on official academic benchmarks
GPQA Diamond is the headline benchmark used by OpenAI and Anthropic for frontier model comparisons. Published GPT-4o scores ~50% on it — meaning there is real room for orchestration to add value. Scoring is fully automatic (letter extraction vs official ground truth). Zero human judgment.
Single-call baseline = same model as lead. The comparison isolates orchestration gains from raw model quality.
By CategoryAccuracy Breakdown
Side-by-Side ComparisonsWhere Orchestration Made the Difference
ReproducibilityHow to Run This Benchmark
Anyone can verify these results
You need OpenAI and Anthropic API keys. The questions, answers, and scoring logic are fully open.
Results are saved to benchmark_results.json and displayed on this page automatically.
TransparencyMethodology
How This Benchmark Works
Every question is answered twice using the same models, ensuring a fair comparison:
Scoring — GPQA / MMLU / GSM8K: Fully automatic — the final letter (A/B/C/D) or number is extracted and compared to the official ground truth. This is the identical scoring method used in the original papers. Zero human judgment.
Scoring — Open-ended: Claude (independent judge, not involved in answering) rates both answers 1–10 on accuracy, depth, and clarity. The judge sees both labeled "Answer A" and "Answer B" — the identity of which is multi vs single is hidden.
Temperature: All answers generated at temperature=0.0 for full reproducibility.
SourcesOfficial Benchmark Datasets
See it for yourself
5 free questions every week — no credit card, no setup. Just add your question and watch multiple models collaborate on it in real time.
Pro plan ($4.99/mo) unlocks unlimited queries with your own API keys + agentic mode