Compare agents and models by outcome
Do this when you want to know which agents and models do your organization's work best, such as before a review of your AI spend. Performance lists every agent, model or configuration with how its finished work turned out: what merged and stayed, what needed rework, what broke later and what it cost.
Who can do this: your organization's Owners and Admins, with Agent performance on your plan. Without it, the Agents tab has no Performance panel.
- In the sidebar, select Reports. The command center opens.
- To look at one project only, pick it under Project in the bar at the top. Leave it on All for your whole organization.
- Select the Agents tab and scroll to Performance.
- Choose how to group the rows: Agent, Model or Configuration. A configuration is one agent with one model, one persona and one review level. Grouped by Configuration, a Duel rating column comes first, from benchmark duels; a setup with none reads No duels.
- Pick a Period: All time, which it starts on, Last 30 days, Last 90 days or Last year.
- Read each row across: Success, First-pass, Defects, Rework, Time to merge, Cost and Security. Higher is better for Success and First-pass; lower is better for the rest. Where a figure has a range in brackets, that is the 95% interval: when two rows' ranges overlap, the gap may be chance. Hover a column's name for what it counts.
- Check the line under each name. It shows how many outcomes the row rests on, or Insufficient data with the count, in grey, when there are too few to judge. Don't rank a grey row against the others yet.
- Agents with no outcomes yet are folded into one line, No outcomes yet:, under the table. Select Show agents without outcomes to list them as rows. Grouped by model, it reads Show models without outcomes.
- To work with the figures elsewhere, select Export CSV. The file holds your organization's outcome records, one row each.
With one project picked under Project, a warning may read Outcome scan: no git person. Then Jaah can't read the project's repository, so reverts and fixes aren't counted. Connect git in your profile as the warning says.
Related
- Compare agent setups side by side: two to four setups on the same kinds of work, and a ranked list on the Quality tab.
- Move an agent to its best-performing setup: act on what you find here.
- Read the command center: the rest of the Agents tab.