Skip to main content

Check whether quality is improving

The tiles at the top of the command center's Quality tab answer one question: is the work getting better or worse? Each tile sets a figure for the range you pick against the same length of time just before it. Use them for a monthly review, or after you change how a project's agents run, to see whether the change helped.

Who can do this: requires the org owner or org admin role. Only Owners and Admins see the command center.

  1. In the sidebar, select Reports. The command center opens.

  2. In the bar, pick a range long enough to hold real work, such as 30d. To check one project, pick it in Project. Every tile follows both.

  3. Select the Quality tab. The tiles sit at the top.

    The Quality tab's tiles: Tasks done, Success rate, Zero-touch, Cycle median, Cost per PR, Review score, Outcome success, First-pass merges, Defects per 100 and Cost per success

  4. Read the first six tiles:

    • Tasks done: tasks finished in the range.
    • Success rate: tasks done out of tasks done or failed.
    • Zero-touch: the share of tasks done with no question, no comment and no other step by a person.
    • Cycle median: the middle time from a task's creation to its finish.
    • Cost per PR: AI spend for each merged pull request.
    • Review score: the weekly AI review's average, from 1 to 5.
  5. Under a figure, read the line that ends vs before, such as −2.6 pts vs before. It is the change against the period before. A tile with no such line had nothing to compare with.

  6. Hover a tile. The hover gives the exact figure, Window before with the earlier figure (or no data), and how the figure is worked out.

  7. If your plan includes agent performance, read the four tiles after Review score. They judge each finished task by what happened to its change afterwards:

    • Outcome success: the share of judged work whose change survived.
    • First-pass merges: merges that went in on the first try, with no failing check and no commit by a person.
    • Defects per 100: merges later linked to a defect, for every 100 merges.
    • Cost per success: AI spend for each piece of work whose change survived.
  8. Hover one of those four. After the formula, n is how many results it counts and the 95% interval is the range the true figure most likely falls in, such as n=19, 95% interval 69–97. A small n or a wide interval means a change against the period before may be chance.

A figure with no data shows — with the reason under it, never 0. A small amber dot after a figure means it is partial: some work behind it could not be counted. Hover the tile to read what was left out.

What to do next​