benchmaxxed

Sources and method

Who ran the test

Start with the release notes, model card and price list. Check what shipped, where it is available and what the rates cover. A catalogue or gateway entry alone does not establish a release.

A benchmark run by the lab that made the model is a vendor claim. For an independent test, name the organisation that ran it. Keep the model, effort setting, test and read date with the result when they affect the comparison.

What a number means

A token rate is a price on a rate card. A task cost depends on what the model reads and writes, including cached input. A worked example states its assumptions. A rank belongs to a particular board, and a demo shows an example of what shipped.

Write-ups link the numbers to the sources. Corrections and updates go on the relevant write-up.

The verdict scale

Switch
Evidence shows a better or cheaper fit for a named group of users.
Wait
The change looks plausible. The evidence still needs an independent check.
Skip
The change makes no material difference to what you would build.