Turn on Production QA for a project
Turn on Production QA for a project so Jaah records its releases, watches for regressions and files tasks from its production incidents. Releases are recorded once they're connected; filing tasks and running probes start only when the project's Production QA switch is on.
Before you start: Production QA is a plan feature. While your plan leaves it out, the project has no Production QA tab. If the tab is there but the switch is greyed out with Off for this org beside it, Production QA isn't on for your org yet: you can still fill in and save every other setting, so the project is ready when it is.
Who can do this: changing the tab's settings and connecting releases need the permission to edit the project; anyone else sees Maintainers connect releases. Connecting a Bitbucket repository's releases and creating the automations is for Org Admins; anyone else sees Org Admins connect Bitbucket releases. and Org Admins manage these automations. in those sections.
Connect releases and switch it on
- In the sidebar, select Directory, then the Projects tab.
- Select the project's name to open its settings.
- Select the Production QA tab. Use the arrows at the end of the tab strip if it doesn't fit.
- Under Connect releases, list what runs the project's releases, one per line, then select Connect. The field depends on where the repository lives:
- GitHub: Workflows to add, the workflow that deploys Production; its runs count as releases. It comes filled in with the workflow that runs on a push to the Production branch and deploys a GitHub environment. When more than one does, Jaah fills in the one whose runs deployed the branch and lists the rest under Also found; select one to add it. Check the field before you connect. A repository whose GitHub Pages build from the branch, with no workflow file, connects with the field empty.
- GitLab: CI config, the pipeline file whose jobs count as releases.
- Azure DevOps: Pipelines to add, the pipeline or release names whose runs count as releases.
- Bitbucket: Production branch, and optionally Release statuses.
- On GitHub, Jaah adds the webhooks to the repository itself, through the Jaah GitHub App or the GitHub account of a member who administers the repository, and Connect releases lists them. If none of those can, one line says what to do: Install app opens the App's install page (then select Connect again), and Set up manually shows the webhooks to add yourself, as below. On any other host, a dialog lists the webhooks to add, with where to add them. The secrets are shown only now, so add each one before you select Done. Connect releases then lists the connected webhooks instead of Not connected.
- Under Releases, check Release workflows (Release pipelines on Azure DevOps): runs of what it lists count as releases, and Connect adds what it connected. It is the one list that counts.
- Tick the Production QA box at the top of the tab.
- Select Save. To drop your unsaved changes instead, select Discard.
- Under Automations, select Create defaults. It adds the automations that file tasks from an incident, a failed release and a health breach. The button shows only while the project has none.
Tune it (optional)
These settings sit on the same tab and save with the same Save button.
- Revert policy, under Releases: Off files nothing when a release causes a regression, Draft files a draft revert task, and Auto-file files one that starts on its own.
- Regression: Window (min) is how long Jaah measures after a release, with fewer than Min events nothing is measured, and Error rate × and p95 latency × are the multiples of the earlier figures that count as a breach.
- Probes: select Add URL under Health URLs to check a URL for its Expected status, or Add command under Smoke commands to run a Command within its Timeout (s). Give each a Name. They run from a node serving the project after each production release; select Remove to drop one.
Related
- What Production QA records and when it files a task: How Production QA works.
- Where a project's settings and tabs sit: How projects work.
- How the automations it creates run, and how monitoring alerts come in: How automations work.