chore: refresh AI model price book - #28224
Closed
cdrci wants to merge 21 commits into
Closed
Conversation
The price book is regenerated by hand with `make gen/aibridge-prices`, so newly launched models stay unpriced and changed prices stay stale until someone remembers to run it. Add a weekly workflow that regenerates both artifacts from live models.dev data and opens a pull request when the output changes. The refresh is never merged automatically: prices are customer-visible cost numbers, so every change goes through human review. A generator failure, which happens by design when upstream drops a pinned or curated model, is announced in Slack. The pull request body carries a generated summary of models added, models removed, and prices changed, rendered by a new deterministic scripts/aibridgepricesdiff tool. Add CODEOWNERS entries for the price book, the frontend catalog, and both generators so the right reviewer is requested on every refresh.
The pull request diff already shows exact prices for every added, removed, and changed row, so repeating them in the body duplicated what a reviewer can read directly. Keep the counts and the model lists, which answer what moved without restating the diff.
… fields The per-field count did not match the model list below it and needed a qualifier to reconcile. Counting models keeps every figure on the same unit.
… fields The summary lists models per category, so tracking which individual price field moved produced detail nothing rendered. Compare rows as a unit and carry model keys through, which removes the per-field change type, the field accessor table, and the two name-mapping helpers. Every category now holds at most one entry per model, so a single key comparator gives a total order and the stable-sort requirement disappears. Verified byte-identical output against the previous implementation on a live upstream snapshot.
…ir own lines Cases with more than one model packed both rows onto a single line, which made the difference between the old and new snapshot hard to spot when reviewing.
…ixtures Struct literals packed two fields onto a line, and cases mixed an inline old snapshot with a multiline new one, so the two sides did not line up when read side by side. Inline both only when each snapshot holds a single row.
…ror messages Substring assertions left section order, spacing, and stray content unverified: reordering the Removed and Changed sections passed the suite. Compare the whole rendered summary for both the empty and populated cases. Assert on error contents rather than mere failure, and cover a missing input file, which the workflow hits if the snapshot step is ever skipped.
The identity was a plausible-looking address that is not verified on any GitHub account, so refresh commits would show an unlinked author. Use the bot identity that backport.yaml and cherry-pick.yaml already use, whose numeric noreply address associates the commit with a real profile.
The style guide's word-choice table lists AI Bridge as a form to avoid. Identifiers stay as they are, since they name the paths and make targets this workflow drives: coderd/aibridge/prices, make gen/aibridge-prices, and scripts/aibridgepricesgen.
The refresh PR touches exactly these two files, so directory-level entries on the generators never fired on it. The knownModels directory also holds hand-written TypeScript, which would have requested review on unrelated frontend work.
New artifacts follow the aigateway naming; a file named aibridge would be invisible to anyone grepping the current name. The branch and concurrency group move with it, since neither exists yet. Paths, make targets, and the scripts keep their aibridge names, which match what they refer to.
…RICES_SLACK_WEBHOOK Matches the workflow name and the <SCOPE>_SLACK_WEBHOOK convention. The secret does not exist yet, so renaming now costs nothing.
DO NOT MERGE. Drop this commit after verifying the run.
The alert built its text in double quotes, so bash kept \n as two characters and jq --arg escaped the backslash. Slack printed a literal \n mid-sentence instead of breaking the line; build the text with printf.
cdrci
requested review from
DanielleMaywood and
jdomeracki-coder
as code owners
August 17, 2026 19:03
The step only runs after the refresh has already failed, so exiting 0 hid the more serious problem: nobody was told. Mark the step red and name the secret, now that it exists and an empty value means a misconfiguration.
cdrci
force-pushed
the
bot/aigateway-prices-refresh
branch
from
August 17, 2026 19:11
5529c51 to
e94d687
Compare
cdrci
force-pushed
the
bot/aigateway-prices-refresh
branch
from
August 17, 2026 19:14
e94d687 to
113d659
Compare
evgeniy-scherbina
added a commit
that referenced
this pull request
Aug 17, 2026
gh pr list/create/edit go through GraphQL, whose queries resolve reviewer and team fields that require the read:org scope. cdrci's token carries only repo and workflow, so the calls succeed while no pull request exists and fail once CODEOWNERS attaches a reviewer: the first refresh would open a PR and every run after it would fail. Observed on this branch: run 32058047624 created #28224, then 32058781222 and 32059086614 both failed on gh pr list with a read:org scope error. The equivalent REST endpoints need only repo.
cdrci
force-pushed
the
bot/aigateway-prices-refresh
branch
from
August 17, 2026 19:28
113d659 to
e62f776
Compare
evgeniy-scherbina
added a commit
that referenced
this pull request
Aug 17, 2026
gh pr list/create/edit go through GraphQL, whose queries resolve reviewer and team fields that require the read:org scope. cdrci's token carries only repo and workflow, so the calls succeed while no pull request exists and fail once CODEOWNERS attaches a reviewer: the first refresh would open a PR and every run after it would fail. Observed on this branch: run 32058047624 created #28224, then 32058781222 and 32059086614 both failed on gh pr list with a read:org scope error. The equivalent REST endpoints need only repo.
cdrci
force-pushed
the
bot/aigateway-prices-refresh
branch
3 times, most recently
from
August 17, 2026 20:07
4922e80 to
05eeab4
Compare
gh pr list/create/edit go through GraphQL, whose queries resolve reviewer and team fields that require the read:org scope. cdrci's token carries only repo and workflow, so the calls succeed while no pull request exists and fail once CODEOWNERS attaches a reviewer: the first refresh would open a PR and every run after it would fail. Observed on this branch: run 32058047624 created #28224, then 32058781222 and 32059086614 both failed on gh pr list with a read:org scope error. The equivalent REST endpoints need only repo.
cdrci
force-pushed
the
bot/aigateway-prices-refresh
branch
from
August 17, 2026 20:16
05eeab4 to
6aa0cac
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Price book changes
36 models added, 11 models removed, 47 models changed.
Added
Removed
Changed
Review notes
Regenerated by
make gen/aibridge-pricesfrom the livemodels.dev catalog. Both artifacts come from one
snapshot, so they ship together:
coderd/aibridge/prices/data/prices.jsonsite/src/pages/AgentsPage/components/ChatModelAdminPanel/knownModels/knownModelsGenerated.jsonThese are customer-visible cost numbers taken from upstream data, so this
PR is never merged automatically. The summary above lists what moved; check
the diff for exact figures before approving.
Opened automatically by the aigateway-prices-refresh workflow.