By Model VS · Guide text updated . Tables below are calculated from the site's saved Arena snapshot when the page is built. They are interpretations of a published preference dataset, not battles Model VS ran. The public board can move after our snapshot. Report a correction with the model id and the source row you expected.
A vote is a choice between two answers.
LM Arena asks people to compare responses and say which one they prefer. The rating is a summary of those choices. It is not the fraction of math problems a model solved, not the fraction of repository issues a coding agent closed, and not a promise about your customers. A model can be preferred in casual chat and still fail a test suite you care about. The reverse is also common: a model that looks strict and dull in a side-by-side vote can be the one that follows a schema.
That distinction is why Model VS keeps Arena on its own surface. The homepage leaderboard can sort the saved overall slice by the source's raw rank. Sorting is a display choice. It does not turn neighboring ranks into a proven order of ability. If you need a task score, leave this page and open the LiveBench category guide. If you need a coding-agent submission, open the SWE-bench flag guide. Do not average those numbers with an Arena rating to manufacture a winner.
Preference data still helps. It is one of the few public signals about which reply people pick when they see two of them. Use it to narrow a conversational shortlist after you have already rejected models that violate a hard constraint: data residency, context, tool support, price ceiling, or a license you cannot accept. A leaderboard cannot check those constraints. Provider documentation can, and it changes.
The source already refuses a false tie-break.
We have always reported confidence intervals alongside Arena scores and surfaced any ties in the rankings that those intervals imply.
The raw rank is simply the rank of the model's Arena score. There are no ties in the rank column: each model is assigned a unique rank.
Our saved file stores a point rating plus a lower and upper bound for each overall row. The interactive table labels that bound as the source 95% interval and shows the raw rank beside it. We do not recompute Arena's rank spread, and we do not collapse overlapping models into a single printed rank. If you need the official spread, use Arena's own leaderboard. What you can do here, without any extra statistics, is the overlap check: if the higher raw rank's lower bound is still less than or equal to the next model's upper bound, the intervals overlap.
Overlap is a warning, not a hypothesis test between that pair. The votes are not independent draws from your users, the prompts are not your prompts, and the covariance between two models is not in the two intervals. A model with few votes often has a wide interval. A wide interval can sit on top of a crowded table and look like a lead. Arena has written about that display problem directly: uncertain new models used to inherit an optimistic rank. Read the vote count before you treat a fresh name as the new default.
The dataset we copy is lmarena-ai/leaderboard-dataset, noted on the file as CC BY 4.0. Model VS rounds the prose tables to two decimal places. The JSON keeps the source precision. If a decision depends on a tenth of a point, you are already past what this page will support. Open the file, or open Arena, and then run your own prompts.
Look at one separated pair and one overlapping pair.
The build inserts two consecutive pairs from the top of the saved overall list when it can find them: the first pair whose intervals do not overlap, and the first pair whose intervals do. Consecutive means neighboring raw ranks in our file, not a claim that Arena considers them the only relevant comparison. A model can be separated from the next row and still overlap a row further down. Use the table as a worked reading, then inspect the ids you actually might buy.
| Record | Source model id | Raw rank | Rating | Published interval | Votes |
|---|---|---|---|---|---|
| Higher raw rank | gemini-4-argon-high | 1 | 1525.22 | 1516.39–1534.04 | 4,932 |
| Next raw rank | claude-opus-4-6-high | 2 | 1504.73 | 1501.31–1508.15 | 77,636 |
The first rating minus the second, before display rounding, is 20.49 points. The published intervals do not overlap. Separation still describes this snapshot's preference ratings, not the probability that one model will complete your task.
| Record | Source model id | Raw rank | Rating | Published interval | Votes |
|---|---|---|---|---|---|
| Higher raw rank | claude-opus-4-6-high | 2 | 1504.73 | 1501.31–1508.15 | 77,636 |
| Next raw rank | claude-fable-5-high | 3 | 1504.27 | 1499.88–1508.66 | 38,387 |
The first rating minus the second, before display rounding, is 0.46 points. The published intervals overlap. Overlap is not a pairwise significance test, and it is not a reason to average the two ratings.
A separated pair is still only preference evidence. People may be rewarding tone, length, formatting, or a language mix that your product will not see. A overlapping pair is a reason to keep both names on the shortlist if both pass your constraints. It is not a reason to split the difference and call the midpoint a better model. There is no model at the midpoint.
Vote count belongs in the same glance. A large vote count means more people participated in the comparisons that produced the rating. It does not mean those people match your users, your language, or your risk tolerance. A small vote count means the interval should be wide and the rank should be easy to move on the next snapshot. Our October snapshot note shows how a fixed-length list moves when new ids enter. Check the date before you paste a rank into a memo.
High, max, and thinking are not nicknames.
Source ids often end in a suffix such as -high, -xhigh, -max, or -thinking. Those strings are part of the record. Dropping them and comparing “the Claude” or “the Grok” mixes two published configurations. One may use a larger thinking budget, a different sampling setup, or a different product default. The bill, the latency, and the failure mode can change with the suffix even when the marketing name does not.
The next tables are chosen by the build from ids that share a stem: one row ends in a known suffix, and another row is the same stem without it. One example prefers a pair whose intervals do not overlap, and one prefers a pair whose intervals do, when both exist. The lesson is the same either way. You cannot inherit the suffixed rating for the unsuffixed id. If the intervals overlap, you also cannot use the raw-rank gap between them as a settled preference win.
| Record | Source model id | Raw rank | Rating | Published interval | Votes |
|---|---|---|---|---|---|
| With -high | claude-opus-4-6-high | 2 | 1504.73 | 1501.31–1508.15 | 77,636 |
| Without that suffix | claude-opus-4-6 | 7 | 1497.27 | 1493.95–1500.60 | 82,189 |
The first rating minus the second, before display rounding, is 7.46 points. The published intervals do not overlap. Separation still describes this snapshot's preference ratings, not the probability that one model will complete your task. Keep the full source identifier, including the effort or thinking suffix, in any shortlist.
| Record | Source model id | Raw rank | Rating | Published interval | Votes |
|---|---|---|---|---|---|
| With -high | claude-opus-4-7-high | 5 | 1501.30 | 1497.53–1505.07 | 64,946 |
| Without that suffix | claude-opus-4-7 | 10 | 1494.10 | 1490.33–1497.87 | 66,058 |
The first rating minus the second, before display rounding, is 7.20 points. The published intervals overlap. Overlap is not a pairwise significance test, and it is not a reason to average the two ratings. Keep the full source identifier, including the effort or thinking suffix, in any shortlist.
When you write a shortlist, copy the id from the snapshot, not from memory. Put the date of the snapshot next to it. If procurement will call a provider, ask which endpoint corresponds to that id. A homepage card that says only the family name is incomplete. Our rankings table keeps the source id under the display name for that reason.
Do not add an Arena point to a LiveBench point.
An Arena point is a preference rating. A LiveBench point is a task score on a named release, usually a percentage of a task the authors defined. A SWE-bench point in our specialist file is a reported percent of issues resolved by a submission that may include tools and a harness. These units do not add, and they do not cancel. A model can lead the preference list and sit far down a coding category. That disagreement is a feature of the evidence, not a bug in the page. Record it, then decide which question you are actually asking.
The same rule applies inside Arena. An overall row is not a math rank, a coding rank, or a multilingual rank unless you opened that category on Arena's site. Our saved comparison file used for this article is the overall slice. Do not describe it as “best at code” because the overall number is large.
Overall winner = Arena rank + LiveBench overall + SWE-bench percent. That expression throws away the unit, the date, the configuration, and the submission scaffold. Model VS will not print it.
Use the interval to refuse a false precision.
- Write the job in one sentence, including what would make an answer unacceptable.
- Remove models that fail a hard constraint before you look at rank.
- If the job is “which reply do people pick,” read Arena with the interval and the full id. If the intervals overlap, keep both.
- If the job is a scored task, switch sources. Start with the category that matches the task, not with Overall.
- Run a small bake-off on your own cases. How we rate is the protocol: freeze the setup, keep the failures, and allow an inconclusive result.
The homepage block at /#leaderboard is the live table for the current snapshot. This essay is the reading rule for that table. When the snapshot changes, the injected numbers change with the build. The reading rule does not: overlap means you are not entitled to a one-name conclusion, and a suffix means you are looking at a different record.