Skip to main content

Check whether a release broke production

After a release, check it before you move on: did the run itself fail, did errors or latency rise once it was live, and did the health checks pass? This guide starts from the release. To start from an incident instead, see Find the pull request behind an incident. What Jaah records about a release, and when it measures one, is in How Production QA works.

Before you start: the release run pages are part of Production QA, a plan feature. While your plan does not include it, the page of a release shows Not on your plan. Metrics and probes appear only for a project whose releases are connected and, for probes, whose Production QA switch is on: see Turn on Production QA for a project.

Who can do this: anyone who can view the project.

  1. Select Tasks, then the project in the sidebar.
  2. Open the release: Run in the Release view opens the latest one. For an earlier one, find its release in the Production section, folded to one line with its version, and select Run on it.
  3. On Overview, read what shipped and how the run ended:
    • PRs lists each pull request in the range; one made by a task links to that task.
    • Other commits lists commits no known pull request carries, when there are any.
    • Failed at names the job and step that failed.
    • Run opens the run in your repository host, to read its log.
  4. Select the Metrics tab. For Error rate and p95 latency, compare Baseline (before the release) with After. Change is marked Breached when it passed the project's limit. While the window is still open the tab reads Pending — measured and a time; come back then.
  5. Select the Probes tab. Under Results, each health URL and smoke command reads Passed or Failed, with its HTTP status or exit code. Open Output on a failed one to see what it printed.
  6. If a probe failed, select Incident and its number beside it, or the time on its row under Incidents, to open the incident it filed. From there, follow Find the pull request behind an incident to pin it on a pull request.

The page refreshes every 15 seconds, so a release still Releasing moves on without a reload. Not measured: on Metrics says why nothing was measured, such as "not a production release".

  • Release runs — every field and tab on the page of a release.
  • Release to production — release again after a failure from the project's Release step.
  • Incidents — the incident a failed probe or regression files.