By Model VS · Guide text updated . This is the sister sheet to the 30-minute shortlist. That page ends with at most three ids. This page adds a price column the JSON files do not contain. Any quality number below is copied or computed from a saved snapshot. No dollar amount on this page was taken from a provider. Write to us if a future file adds a price key and this page still says it is absent.
Which prices are missing from the snapshots?
A cost triage needs two inputs that share an id: a quality measurement you are willing to cite, and a price you are willing to pay. We can supply the first input only when it is already in a saved file. We cannot supply the second. arena-latest.json, livebench-latest.json, and specialists-latest.json are the benchmark snapshots this site ships. None of them is a price list. The build checks the keys and says so in the table below. A missing key is not a zero price. Zero would be a price. Absence means the cell stays blank until a person fills it from a page they opened.
Public list price means the figure on a provider’s pricing page, or on a model card, that you actually saw. The unit might be a currency amount per million input tokens, per million output tokens, per request, or whatever unit that page prints. Copy the unit with the figure. Copy the URL. Copy the date. Those three are part of the price citation, the way a release string is part of a LiveBench citation. A number remembered from a slide, with no page and no date, is not a list price. This page will not turn that memory into a table.
The half-hour sheet already has a hard stop for a price ceiling finance has signed. That ceiling is your constraint. It is not a market quote, and we do not fill it in. If the ceiling is missing, the triage output is “blocked on a price,” which is a legitimate stop. Do not pick a quality leader while you wait and then staple a guessed discount onto it. Methodology already treats price, latency, and data-handling terms as separate from a public measurement.
Which quality field is the triage allowed to copy?
One field. The job picks it, which is the same rule as the half-hour sheet. A preference claim may copy an Arena overall id, raw rank, rating, published interval, and vote count, with the publish date and the save time. Copying the vote count does not turn it into a weight for the price. Vote count versus interval width is why that column is not a precision score and not a win. This page uses the vote count only as a field you may transcribe, then leaves it alone. It does not reprint the overlap pairs from that essay.
A scored task may copy one LiveBench category average for the acceptance sentence, plus the full id, the release, and the save time. Overall is allowed only when the sentence is the unweighted mean of every category. The category guide matches tasks to columns. This page does not restate that map. Once the column is chosen, that column is the quality cell. The other columns do not enter a ratio.
A submission claim may copy one specialist metric, the reported value, the verification flag, the dataset id, and the card link, with both timestamps. The flag essay and the card essay explain those fields. This page does not reprint their tables. The percent is a quality cell, not a price, and the flag stays attached to the percent when the percent moves onto a cost sheet.
If Arena and LiveBench order your three ids differently, you still do not average them into one quality number and then divide by money. When Arena and LiveBench disagree shows exact-id reversals from the current files. The triage keeps the quality cell named in the acceptance sentence and leaves the other board in a note. Two qualities and one price are not a single efficiency. The worked row below is the first complete LiveBench record in file order. It shows the shape of a quality cell next to an empty price cell. It is not the id you should buy.
| Saved file | Quality fields you may copy | Price stored |
|---|---|---|
| arena-latest.json | Overall rows: 100. Paths: category, leaderboard_publish_date, license, model_name, organization, rank, rating, rating_lower, rating_upper, variance, vote_count. Publish date 2026-10-02. Saved 2026-10-06. | No price key |
| livebench-latest.json | Release 2026-06-25. Saved 2026-10-06. 66 table rows, 66 with a complete Overall. Category names: Reasoning, Coding, Agentic Coding, Mathematics, Data Analysis, Language, IF. | No price key |
| specialists-latest.json | Saved 2026-10-06. Benchmarks: swe-bench-verified (% resolved); gpqa (accuracy). Citation paths: lowerIsBetter, modelId, parameters, rank, source.isExternal, source.name, source.url, value, verified. | No price key |
| Sheet line | Quality side from this file | Price side |
|---|---|---|
| Source model id | claude-opus-4-5-20251101-thinking-64k-high-effort | Not in file |
| Overall | 72.58 | Not in file |
| Public list price | Not in file | Reader fills the public list price, the unit, the URL, and the date they opened the page |
Use that Overall cell only if your sentence really is the unweighted mean. If the sentence names a category, go back to the file and copy that category instead. The caption will still be true: the price side is not in the file. A later sync can change which id is first in source order. Re-read the table. Do not quote the id from memory after the file has moved.
Where does a public list price get written down?
On your sheet, not in our JSON. The checklist is the method. Print it. The quality column names the file. The price column is blank on purpose. You can type in the last column in the browser. The page does not save or send what you type. Paste the result into your own document before you close the tab. The example column is a procedure. It is not a quote, and it contains no price and no benchmark result.
| Line | What belongs there | Example — procedure only | Your notes |
|---|---|---|---|
| Candidate id | The full source id, including any effort or thinking suffix. | Example. Copy the id from the half-hour sheet. A family nickname is not an id, and a cheaper setting is not this id. | |
| Quality file and clock | One file path, plus the publish date or release, plus the save time. | Example. Open one of the three files. Copy the clock printed on that file. Do not mix a LiveBench release into an Arena row. | |
| Quality field and value | The one field the acceptance sentence named, copied from that file. | Example. A scored task uses one LiveBench category. Overall stays blank unless the sentence is the unweighted mean. Keep the unit beside the value. | |
| Price URL | The public page you opened. Our files do not store it. | Example. Paste the provider pricing URL or the card URL where the figure appeared. A recollection with no page leaves this cell blank. | |
| Price figure and unit | The figure and the unit printed on that page. | Example. Write the figure you saw and the unit the page used. Do not convert it into a per-quality score on this site’s behalf. | |
| Date seen | The date you opened the price page. | Example. Write the calendar date. A missing date means the price citation is unfinished, even if the figure looks familiar. | |
| Configuration match | The same id on the price page, or an explicit mismatch. | Example. If the price page drops a high, max, or thinking suffix, write mismatch. Do not use the score with that price. | |
| Refusal | The ratio you will not attribute to Model VS. | Example. Leave the quotient uncomputed here. If you compute one later, in your own sheet, label it as yours and name both units. |
Hand the sheet to the person who will call the endpoint. They should be able to see the id, the quality cell, the price page, and whether the configuration matched, without reopening a leaderboard to invent a discount. If the price URL is blank, the triage is not finished. Add the URL or write “blocked on a price.” Do not add a fourth model to make the arithmetic look complete. The half-hour limit of three ids still applies.
Why is there no cost-per-quality score on this page?
Dividing a rating by a currency figure produces a new metric. We did not define that metric, and we do not have the currency figure. Arena points, a LiveBench average, and a percent resolved are different units from each other and from money. A quotient would hide the unit, hide the date of the price, and hide which configuration was billed. Publishing one would be a new score this site did not measure. How we rate is explicit that public measurements are not a substitute for your own check, and they are not a bill.
You may compute a ratio in your own spreadsheet. If you do, the label has to be yours. Name the quality unit, the currency unit, the id, both dates, and the fact that Model VS did not produce the ratio. Do not put our name on it. Do not rank vendors by it and describe the ranking as ours. A lower ratio is not automatically a better purchase. The quality field may be the wrong exam. The price may omit retries and the minutes a person spends correcting the output. A cheap setting may be a different model id from the one that posted the score.
Retries, tool calls, and a thinking budget change the bill without changing the saved score. The score’s suffix is the configuration the measurement belongs to. If you will call a cheaper setting, you do not own that score. Leave the quality cell on the id you will actually call, or mark the pair as unmatched and run the three cases from the half-hour sheet on the setting you will pay for. Three cases will not estimate a company-wide cost. They will tell you whether the list price and the failure mode belong in the same meeting.
Hard constraints still come before any ratio you privately compute. A license you cannot accept, a region you cannot use, or a context length that cannot hold the transcript removes the id even when the arithmetic looks favorable. The board does not know the contract. The card or the vendor terms might. Provider card versus the leaderboard shortlist is how you cite that page without pretending it is the leaderboard. A license string stored on an Arena row is not a substitute for the paragraph you read.
After the sheet has one quality cell and one price cell for each surviving id, stop. You are not owed a winner. You are owed a record that a later reader can audit. If the price page moves next week, your figure remains the figure you saw, tied to the date you wrote. Start a new line if you refresh the quote. Do not edit the old quality value to make a new price look consistent. The October changelog shows how a saved list moves. A price page can move even when our file did not. The triage has done its job when nobody on the thread can mistake a blank price cell for a number we published.
Questions about triaging cost and quality
Does Model VS publish a price?
No. The saved Arena, LiveBench, and specialist files have no price field. A public list price has to be copied by the reader from the provider page, with the URL and the date.
Can a score be divided by a dollar price?
Not as a Model VS result. The units do not match, and the price was not in the file. If you compute a ratio yourself, label it as yours and name both units.
Which quality number should the triage copy?
One field, chosen by the job. Arena fields for a preference claim, one LiveBench category for a scored task, or a specialist metric and flag when the job is that benchmark. This page does not average those boards.
What if the price page is a different configuration from the score?
Write the mismatch and stop. A suffix on the scored id is part of the product. Do not attach a cheaper price to a more expensive id, and do not attach that id's score to the cheaper price.