By Model VS · Guide text updated . The inventory below is built from the saved Arena snapshot, the saved LiveBench snapshot, and the saved specialist snapshot. We do not download the card body. Report a correction if a key list does not match its file.
When is a provider card enough to cite?
A provider card, in this essay, is a page a lab or a host publishes about one model. It might be a Hugging Face model card, a vendor model page, or a README at a URL you opened yourself. The card is enough when the sentence is about that page. The sentence you can support sounds like this: on this URL, on this date, the page showed this license paragraph, this context length, this intended-use line, or this price. You name the URL. You name the date you opened it. You quote the words that were there. Model VS did not save those words. The citation is yours, and it expires when the page changes.
That job is different from proving a leaderboard result. A card can mention a benchmark. Mentioning a benchmark does not make the mention our row. The card can omit the split, the harness, the sample count, or the date. It can show a percent from a paper that predates the repository. Those are reasons to keep the card citation in its own sentence. They are not reasons to throw the card away. A license question, a context-length question, and a price question are card questions, or vendor-terms questions, because the snapshots on this site do not store those paragraphs.
A family nickname is not a card citation. If the page is about one configuration and the sentence is about another, you have the wrong page. Suffixes such as -high, -xhigh, -max, and -thinking are part of a source id on our leaderboards. A card that drops the suffix is a card about a different product until the page says otherwise. Write the id you mean in the same note as the URL. Reading specialist and HF cards is the field-by-field reading of a link we did store. This page does not repeat that inventory. It answers the prior choice: which surface is allowed to carry the sentence you are about to publish.
| Sentence you are publishing | Surface that can carry it | Do not cite this in its place |
|---|---|---|
| A license paragraph, context length, intended-use line, or price you read on a provider page | That page’s URL and the date you opened it | A leaderboard rank, or a license string copied from a different file |
| An Arena rating, interval, vote count, or raw rank | The saved Arena overall row, with the publish date and the save time | A card sentence that does not show those fields |
| A LiveBench category average, or Overall | The saved LiveBench row, the release, and the save time | A README table that has no release string |
| A specialist reported value | The saved row: model id, metric, flag, dataset id, card link, and both timestamps | The card link alone |
When does the sentence need the saved shortlist?
The saved shortlist is the row in a file we actually publish. Use it when the sentence contains a rank, a preference rating, an interval, a vote count, a LiveBench average, or a specialist reported value. Those numbers live in the three JSON files named above. They do not live in a card we did not copy. If you cannot point at the file, the release or publish date, and the save time, you are not citing the shortlist. You are remembering a screenshot.
The shortlist is also the right surface when you are choosing among ids we have already saved. A card is a document about one model. A shortlist is a list, taken with a method, on one date. “This id has this raw rank on the saved Arena overall slice” is a shortlist sentence. “Readers will prefer this model” is not, even when the rank is real. How to read LM Arena intervals is the rule for a preference claim. LiveBench categories for shortlisting is the rule for which column matches a task. A 30-minute model shortlist is the rule for stopping at three ids. This page sits in front of those essays. It stops a card paragraph from being used as a rank, and it stops a rank from being used as a license.
Open the shortlist when the card and the job disagree about what is being measured. A vendor page can call a model its coding model. The LiveBench file can still require you to cite one category, or to leave LiveBench closed because the job is a SWE-bench submission. The specialist file, not a marketing adjective, holds that submission’s metric and flag. If you do not need a public number, you do not need either surface. Write the acceptance sentence and move to your own cases. How we rate is the protocol for that check: same setup, failures kept, inconclusive allowed.
Our saved Arena file is the overall slice. A sentence about a different Arena board, such as a coding or a language board on the upstream site, is not supported by the category values listed in the inventory below. Open that board and cite it, or do not cite it. Do not describe the overall slice as that other board because the model name looks related.
What do the saved files actually contain?
The build lists keys. It does not infer a card from a model name, and it does not fetch a README to fill a hole. If a later sync adds a URL key or a price key, the table should change with the file. Until it does, absence is the finding. Absence is not a zero, and it is not permission to paste a number from memory.
| Saved file | A shortlist citation can name | Card URL stored | Price stored |
|---|---|---|---|
| arena-latest.json | 100 overall rows. Keys: category, leaderboard_publish_date, license, model_name, organization, rank, rating, rating_lower, rating_upper, variance, vote_count. Category values: overall. License field values: Apache 2.0, Kimi K3 license, MiniMax Community License, MIT, Modified MIT, Proprietary. | No card URL key | No price key |
| livebench-latest.json | 66 rows in the table, 66 with a complete Overall. The header is the model id plus 23 task columns. Category names: Reasoning, Coding, Agentic Coding, Mathematics, Data Analysis, Language, IF. Organization is not a header key. | No card URL key | No price key |
| specialists-latest.json | 167 rows across 2 benchmarks (swe-bench-verified, gpqa). Citation paths: lowerIsBetter, modelId, parameters, rank, source.isExternal, source.name, source.url, value, verified. | 166 rows store an http(s) source.url. 1 do not. 21 of the stored URLs contain README.md or /blob/. | No price key |
- Arena overall rows: 100
- LiveBench table rows: 66
- Specialist rows with a stored card URL: 166
- Specialist rows whose URL points at a README blob: 21
| Citation piece | Stored on the shortlist row | Still only on the card |
|---|---|---|
| Model id | mindlab-research/Macaron-V1-Coding-Venti | Whether the publisher’s page uses this same string |
| Benchmark | swe-bench-verified (SWE-bench Verified); dataset SWE-bench/SWE-bench_Verified | Any other split named only in the README |
| Metric and reported value | 85.6 % resolved | A different percent printed on the card, if one is there |
| Verification flag | false — community submission, not marked verified | The harness description, which this row does not store |
| Card link | https://huggingface.co/mindlab-research/Macaron-V1-Coding-Venti/blob/0a7ec8fdf72e0757decabd90cc3305bed1fed1c6/README.md#L68-L76 — link name Model Card | The README body at that URL. The file stores the link, not the body. |
| Clocks | Dataset timestamp August 16, 2026; snapshot saved October 6, 2026 | The date you opened the card. It is not a field in this file. |
Read the worked example as a split, not as a winner. The shortlist row holds the benchmark id, the metric, the reported value, the flag, and the timestamps. The URL may point at a README line. Pointing at a line is not the same as storing the line. The selection rule is string matching on the URL: the first saved row whose link contains README.md or /blob/. That match only chooses the row. It is not a quality rank. A full field dump of a different row, chosen by a different rule, is on the card essay. Use that page when you need every stored field. Use this page when you need to decide which citation the field belongs to.
A license string on an Arena row, when the file has one, is still a leaderboard field. It is not the card’s license paragraph, and it is not legal advice. Organization is the string the Arena file stored. It may or may not match the account that published a card. Do not weld the two together because the brand looks familiar. LiveBench ids in our file often include an effort suffix and, as the inventory states, no organization column. You cannot recover a provider card from that id by guessing the lab. You can search, open a page, and then cite the page with the date you opened it.
The specialist link is the one card pointer these files do store. Even then, the pointer is not the page. A link name of “Model Card” does not mean we reviewed the card. An external bit, when the row has one, is a flag on the link, not a review of the organization. If the URL and the model id differ, copy both strings. The card essay explains that mismatch. This page only requires that you notice it before you publish one of the two strings as if it were both.
How do you keep a card figure and a shortlist figure apart?
Write two sentences when you have two sources. Sentence one is the shortlist: file, id, metric or rating, value, flag if there is one, publish or release clock, and save time. Sentence two is the card: URL, the date you opened it, and the words you are using. If the percents differ, both percents stay. Averaging them creates a third number nobody published. If the card has no date, say the date is missing. If our row has no card URL, which is the Arena and LiveBench case unless the inventory says otherwise, do not borrow a specialist link from a different id to decorate them.
The half-hour sheet tells you to leave a cell blank rather than invent a bridge. This page is the same habit one step earlier. A blank card cell means you have not opened a page. A blank shortlist cell means the number is not in the file you are allowed to cite. Either blank is a finished line. Filling it from memory is how a note starts quoting a model the buyer will not call, at a score the file does not hold.
What you must not imply, from either surface: that Model VS ran the benchmark, that we endorse the model, that a README percent is our row, that an Arena rank is a license to use the weights, or that a card’s adjective is a category average. Methodology keeps the surfaces apart so those implications do not sneak in through one blended sentence. The compare tool will show saved fields for two ids. It will not show the card body, and it will not show a price.
Then stop. If the decision is a purchase, the card or the vendor price page is where the contract terms live. The next essay, cost-per-quality triage, refuses to invent a dollar figure and tells the reader to write the public list price themselves. If the decision is a pilot, return to How we rate. A card and a shortlist can both be true and still leave the pilot undone. Keeping both citations, and shipping neither model, are allowed results. The choice of surface has done its job when a reader can tell which sentence came from a page they would have to open, and which sentence came from a file we saved.
Questions about a card versus a shortlist
When is a provider card enough to cite?
A card is enough when the sentence is about what that page said, and you name the URL and the date you opened it. A rank, a rating, a LiveBench average, or a specialist value still has to come from the saved shortlist.
Do the leaderboard files include the card text?
Arena and LiveBench store no card URL. The specialist file stores a link and a link name, not the README body. Open the link before you quote the card.
Can a card's benchmark table replace the shortlist row?
No. A README figure can be a different split, harness, or date. Keep it as a second citation. Do not overwrite the saved row, and do not average the two figures.
Does the Arena license field replace the card's license?
No. The license field is whatever string the Arena file stored. It is not the license paragraph on a card we did not save. Read the card, or the vendor terms, before you decide a use is allowed.