Userbag.co.uk

Given the challenge I had in my own recent googling, I thought it would be worth while putting together this quick blog post to provide a simple/direct answer as to how to configure a GitHub Action on a pull request, so that CI tasks are only run when a certain reviewer has been added.

Quick background

• We had a large application with a significant test suite.
• We try to create our pull requests early in the development of a feature to aid visibility.
• We need all pull requests to require a successful run of the test suite before they can be merged.
• Finally: We didn’t want to keep run the test suite over and over unnecessarily (for example every time the branch is pushed to while being developed).

As such our preferred mechanism was for the test suite to only run when a specific reviewer is attached to the pull request. In our case the reviewer is our service account.

Solution

Lets assume our existing Github action looked something like the below. The Github action is set to run whenever a new PR is opened, a new reviewer is added or when new commits are pushed in to the pull request itself.

name: ci

on:
  pull_request:
    types: [ opened, review_requested, synchronize ]

jobs:
  run_our_ci:
    runs-on: ubuntu-latest
    steps:
    - name: Run our CI
      run: |
        echo "Hello world!"

Currently this will be run every time one of those actions takes place, which isn’t something we want. The basic solution is to add a condition to the task, such as the below.

if: contains(github.event.pull_request.requested_reviewers.*.login, '<CI_SERVICE_ACCOUNT>')

The above will cause triggered actions to skip whenever the condition is not met. The condition in this case being that one of the attached reviewers has a login name matching <CI_SERVICE_ACCOUNT>.

The same can be achieved with a label instead using something along the lines of

if: contains(github.event.pull_request.labels.*.name, '<LABEL_NAME>')

With the above condition added to your task, your YAML should now resemble the below

on:
  pull_request:
    types: [ opened, review_requested, synchronize ]

jobs:
  run_our_ci:
    if: contains(github.event.pull_request.requested_reviewers.*.login, '<CI_SERVICE_ACCOUNT>')
    runs-on: ubuntu-latest
    steps:
    - name: Run our CI
      run: |
        echo "Hello world!"

If you now open a pull request with the above, you will note that each commit continues to get a “tick” next to it – but when you click in for detail the task itself will have skipped.

It is also worth being aware that making a base task skip will also cause any tasks that depend on it to skip as well (ie. any referencing the task as a “needs:”. This is useful in that you don’t have to add the “if” to every task, but can also be annoying given that tasks are considered to been run successfully (and skipped) rather than having not run at all by the Github protected branch feature.

This presented us with an issue, as due to the protection rules seeing “skip” as a “success” state, the branch protection rules will now happily let you merge a branch without any CI being run on it, something we certainly didn’t want.

To work around this (although not ideal), and ensure the branch protections do enforce that the full test suite has run before allowing a merge, we can add an inverse version of the job such as the below;

no_ci_has_run:
    if: "!contains(github.event.pull_request.requested_reviewers.*.login, '<CI_SERVICE_ACCOUNT>')"
    runs-on: ubuntu-latest
    steps:
    - name: "No tests have run"
      run: |
        exit 1

This task will explicitly fail whenever the main CI steps have been skipped. By adding this as a “required” check in your protected branch settings, you can therefore ensure that people can only merge the branch when the the full test suite has been run & passed on the given pull request.

This works due to the no_ci_has_run task “skipping” when the test suite is being run – which as mentioned above the branch protection feature sees as a success.

TLDR

The combined github actions YAML may look something like the below.

• The real CI task will only run when CI_SERVICE_ACCOUNT is added as a reviewer.
• The no_ci_has_run task is run whenever it is not, preventing the branch from being mergeable (If you use branch protection)
• Multi step tasks will automatically skip if a previous task’s conditional fails.

name: ci

on:
  pull_request:
    types: [opened, review_requested, synchronize ]

jobs:
  run_our_ci:
    if: contains(github.event.pull_request.requested_reviewers.*.login, '<CI_SERVICE_ACCOUNT>')
    runs-on: ubuntu-latest
    steps:
    - name: Run our CI
      run: |
        echo "Hello world!"
 no_ci_has_run:
    if: "!contains(github.event.pull_request.requested_reviewers.*.login, '<CI_SERVICE_ACCOUNT>')"
    runs-on: ubuntu-latest
    steps:
    - name: "No tests have run"
      run: |
        exit 1