InsightsPreference evidence

A larger vote count is not a tighter verdict.

Vote count and interval width answer different questions. A high-vote model can still overlap the next rank. Votes are not a win probability.

Two columnsNot one verdict
VotesHow many comparisons the row stores
WidthUpper bound minus lower bound
StopNeither number is a win probability

By Model VS · Guide text updated . The tables below are filled from the saved Arena snapshot when the page is built. Model VS did not collect the votes and does not recompute the rating. Report a correction with the source id and the file you expected.

Vote count

What does a vote count actually count?

A vote count on an overall Arena row is the number of comparisons the source stored for that saved record. People saw two replies and picked one. The file then keeps a point rating, a lower bound, an upper bound, a raw rank, and that count. The count is about participation in the published vote. It is not a headcount of your users, not a sample of your language, and not the number of times the model will be right on a task you wrote.

Time on a public board is one reason a count grows. A configuration that has been in the vote for a long stretch can show a large total while a newer configuration of a related name shows a small one. Those are different records when the source ids differ. A suffix such as -high, -xhigh, -max, or -thinking is part of the id. Adding the vote cells of two suffixes produces a total the source did not publish. How to read LM Arena intervals explains why those suffixes are not nicknames, and why an overlapping band is not a settled order. This page assumes that rule and asks a narrower question: what the count does not tell you about the width of the band, or about whether the next raw rank is decided.

The homepage can sort the saved overall slice by raw rank. Sorting is a display. It does not turn the vote column into a probability, and it does not make the row with more votes the one you should ship. If the job is a scored task, leave this page for the LiveBench category guide. If the job is a coding-agent submission, open the SWE-bench flag guide. A vote total does not confirm either of those surfaces. Methodology keeps them apart so nobody adds a preference count to a task score.

A family nickname is a second way to misuse the column. The count belongs to the exact source id in the row. If procurement says only the family name, you do not yet have a vote total to cite. Copy the id from the snapshot, then read the count that sits on that string. A cousin with a different suffix can have a much larger count and a completely different band. Inheriting one for the other is how a shortlist starts quoting a model the buyer will not call.

Width

What does published interval width measure?

Width, in the tables on this page, is arithmetic the build performs on two stored fields: the published upper bound minus the published lower bound. The subtraction uses the source values. The page then rounds the display to two decimal places, the same way it rounds a rating. Width is not a new leaderboard, and it is not a quality score. A narrow width means those two bounds sit close together on that one row. A wide width means they sit further apart. That is the whole claim the column can support.

Decisive is a different question, and it needs two rows. The reading we use is the overlap check from the interval essay: if the higher raw rank's lower bound is still less than or equal to the next row's upper bound, the published intervals overlap. Overlap is a warning about this snapshot's preference ratings. It is not a hypothesis test between that pair. The prompts are not your prompts, and the covariance between the two models is not hiding inside the two intervals. A narrow band can overlap the next rank when the point ratings are close. A wide band can stay clear of the next rank when the point gap is large. Tight and decisive are not synonyms. Treating them as synonyms is how a small lead gets written into a memo as a finished order.

We do not recompute Arena's rank spread, and we do not collapse an overlap into a single printed rank. Arena already separates the unique raw rank from the spread implied by its intervals. The interval essay quotes that distinction. What this page adds is the vote count and the derived width on the same row, so a reader can see that the busy cell and the narrow cell answer different questions. The dataset we copy is lmarena-ai/leaderboard-dataset, noted on the file as CC BY 4.0. If a decision depends on a tenth of a point of width, you are past what this page will support. Open the JSON, or open Arena, and then run your own prompts.

What each saved Arena column can answer on this page. This is a reading rule. It does not name a model, and it does not add a score.
ColumnQuestion it answersQuestion it does not answer
Vote countHow many comparisons the source stored for that exact idWhether the next raw rank is settled, or the chance your users prefer the row
Interval widthHow far apart the published bounds sit on that one rowWhether those bounds clear the next row
Overlap of two rowsWhether the published bands meet on this snapshotA winner, a probability, or a result on a task you did not run

Width is also not a sample size you can compare across sources. A LiveBench category has no vote count, and a specialist percent has no Arena interval. Putting “more precise” on a row because its Arena band is short says nothing about a task column. When Arena and LiveBench disagree is the page for an exact id that reverses order across those units. This page stays inside the Arena overall slice.

This snapshot

Where does a high vote count still overlap the next rank?

The build starts with the overall row that has the most votes and a usable published interval. If that row's interval overlaps the next row in the saved slice, the table prints those two ids, higher raw rank first. If it does not, and the interval overlaps the row just above, the table prints that pair instead. If the busiest row overlaps neither neighbor, the build does not pretend that it does. It prints the consecutive overlapping pair whose larger vote count is the biggest on the file, and the caption says the most-voted row was not that pair. The vote total is only a selection key. It is not a score.

  • Overall rows with a published interval and a vote count: 100
  • Consecutive pairs with both fields: 99
  • Pairs whose published intervals overlap: 98
  • Pairs where the higher vote count also has the wider interval: 5

The most votes on this slice: gemini-2.5-pro, raw rank 86, 124,834 votes, interval width 4.93. Ordered from narrowest to widest, that width is place 1 of 100. On this snapshot the most-voted overall row is also the narrowest. That association does not settle the next row.

The overall row with the most votes, then the next row in the saved overall slice. Chosen because those published intervals overlap. The vote total only identifies the row. It is not a model score and not a win probability. Source publish date 2026-10-02; snapshot saved 2026-10-06.
Role on this pairSource model idRaw rankRatingPublished intervalInterval widthVotes
Higher raw rankgemini-2.5-pro861445.571443.10–1448.034.93124,834
Next raw rankgpt-4.5-preview-2025-02-27871444.691438.98–1450.4011.4214,547

Reading the pair in the order printed above, higher raw rank first. The first rating minus the second, before display rounding, is 0.88 points. The published intervals overlap. Overlap is not a pairwise significance test, and it is not a reason to average the two ratings. Interval width is the published upper bound minus the published lower bound, before display rounding. Width describes one row. Overlap describes two rows. Neither is a win probability, and the vote count is not a conversion of one.

Read the width column beside the votes before you read the raw rank as a verdict. The paragraph above the table says whether the most-voted row is also the narrowest row on this slice, and it counts how many neighboring pairs still overlap. The extremes can be the same id. Sameness would mean that, on this file, the busiest record is also the tightest band. It would not mean the next row has been beaten. Overlap survives a narrow band whenever the next rating sits inside it. A separated pair elsewhere on the list, including a pair in which the higher rank has fewer votes and a wider band, is a different fact. Use the interval essay for that rank-gap reading. Use this page when the temptation is to let the vote column finish the argument.

Do not turn the vote cell into a percent. Dividing one row's votes by the sum of the two rows invents a share the source did not define. The people in the vote did not each cast a ballot for a champion. They compared replies, often against many opponents, under a rating method this site does not reimplement. A large count can mean the rating has been seen many times. It can also mean the id has been public longer. Neither reading is a percent chance that one model beats the next one. The file does not store that probability. How we rate tells you to freeze a setup and run your own cases when you need a decision the interval will not give you.

Absence belongs in the same glance. An id with a small count is not a silent loss. The band is often wide, and the next snapshot can move the raw rank. An id that is missing from the file has no vote count to cite. Do not borrow the family name's total. The October changelog shows how a fixed list moves when ids enter or leave. Check the publish date in the caption before you paste a rank or a vote total into a document that will outlive the week.

The other way

When does the busier row have the wider band?

Votes and width often move together, and the extremes may line up. That association is not an identity. The next table looks only at consecutive overall rows. It keeps a pair when one row has both more votes and a strictly wider interval than its neighbor. Among those pairs it prints the one with the largest vote-count gap. The gap chooses the rows. It is not a quality score. A later sync can choose different ids, or it can report that no such pair remains. If every neighbor on a future file has the wider band on the quieter row, the page will say that, rather than searching past the neighbors until a slogan comes true.

Consecutive saved rows where one side has both more votes and a strictly wider interval. Chosen as the pair with the largest vote-count gap under that rule. The gap only selects the rows. It is not a model score and not a win probability. Source publish date 2026-10-02; snapshot saved 2026-10-06.
Role on this pairSource model idRaw rankRatingPublished intervalInterval widthVotes
Higher raw rank — more votes, wider intervalminimax-m3961440.061436.11–1444.007.8958,757
Next raw rank — fewer votes, narrower intervalgpt-5.1971438.511434.91–1442.107.2044,298

Reading the pair in the order printed above, higher raw rank first. The first rating minus the second, before display rounding, is 1.55 points. The published intervals overlap. Overlap is not a pairwise significance test, and it is not a reason to average the two ratings. Interval width is the published upper bound minus the published lower bound, before display rounding. Width describes one row. Overlap describes two rows. Neither is a win probability, and the vote count is not a conversion of one.

The useful reading of that pair is modest. The busier row is not automatically the tighter row, even one rank away. Someone who sorts by votes to find the precise model is sorting by a different question. Precision, in the narrow sense of a short published band, is the width column. A long public vote is the vote column. A shortlist that needs one of them should name which one. A shortlist that needs a separated preference should look at overlap, then keep both names when the bands meet. The gap printed by the selection rule stays in the caption. It does not belong in a procurement slide as if it measured quality.

Hard constraints still come first. Data residency, a license you can accept, a context length, a tool API, and a price ceiling can remove the id with more votes. The board does not know your contract. Provider documentation does, and it changes. If a constraint removes the busier id, you do not inherit its vote count for the id you kept, and you do not inherit its width. Copy the full source id from the row you are actually citing, including any effort or thinking suffix, and put the snapshot date beside it. The thirty-minute shortlist is the place to stop after three ids. This page is the reason the vote column is not the tie-break inside those three.

What you can write, cleanly, is a sentence that keeps the units apart. Shape: on the Arena overall file named in the caption, one id has the larger vote count and the printed width, and the next saved row is the other id. If the published intervals overlap, the sentence stops there. It does not treat the vote count as a win. The job you are hiring for stays in your own words. That sentence survives a sync better than a blended rank does. There is no model at the midpoint of the band, and there is no model whose quality equals its vote total.

Then leave the board. If the bands overlap, keep both ids until your own prompts separate them. If the bands are separated, you still have only preference evidence. People may be rewarding tone, length, formatting, or a language mix your product will not see. Three cases will not estimate a company-wide rate. They will tell you whether the vote column was a distraction. Keeping both names, and shipping neither, are both allowed results. The count has done its job when a reader stops mistaking it for the interval.

FAQ

Questions about votes and interval width

Does a larger vote count mean a tighter interval?

Not as a rule. The vote count is how many comparisons the source stored for that row. Interval width is the published upper bound minus the published lower bound. On some neighbors the busier row is the wider one. This page prints that pair from the saved Arena file when the file contains one.

Can a high-vote model still overlap the next rank?

Yes. Overlap compares two published bands. A large vote count can sit on a narrow band that still meets the next row. Overlap is a reason to keep both source ids. It is not a loss for the busier row, and it is not a win for it either.

Is the vote count a win probability?

No. The file does not store the probability that one model beats the next one for your users. Do not divide the vote cells, and do not read a vote total as a percent chance.

Should the vote column break an overlapping pair?

No. If the bands overlap, keep both ids until a constraint or a prompt of yours separates them. How we rate is the protocol for that check. A longer public vote history is not one of your requirements unless you wrote that requirement down.