JUnit Results on GitHub Actions

Using Build Scans

I tend to use Gradle for most of my Java projects. This guide uses Gradle. It depends on a feature called Gradle Build Scans.

First, let’s enable Gradle Build scans on your build by adding this to your settings.gradle.kts

plugins {
  id "com.gradle.enterprise" version "3.6.3"
}

gradleEnterprise {
  buildScan {
    termsOfServiceUrl = "https://gradle.com/terms-of-service"
    termsOfServiceAgree = "yes"
    publishAlwaysIf(System.getenv("CI")) (1)
  }
}

1	This will publish the build scan only for CI. You can replace it `publishAlways()` if you want to always publish build scans.

If you now run a gradle build, your build will emit the build url to the console

# ./gradlew build

...
...
BUILD SUCCESSFUL in 1m 0s
17 actionable tasks: 17 executed

Publishing build scan...
https://gradle.com/s/j67twx3r5jzqm (1)

1	This is the URL that contains a lot of information on the build, including the JUnit results.

Getting to these is a bit of a problem from a GitHub Pull Request. So let’s extract this to a file first.

import com.gradle.scan.plugin.PublishedBuildScan

gradleEnterprise {
  buildScan {
    // ... previous configuration
    buildScanPublished { PublishedBuildScan scan ->
      file("build/gradle-scan.md").text = (1)
          """Gradle Build Scan - [`${scan.buildScanId}`](${scan.buildScanUri})"""
    }
  }
}

1	This will write some markdown to a file which we can then use as a comment on the pull request.

Now when you run the same command, you’ll find a file in the build directory called gradle-scan.md that looks like this

Gradle Build Scan - [`mor6ne7ktkgwy`](https://gradle.com/s/mor6ne7ktkgwy)

Now, let’s get GitHub Actions to process with it.

I’ll assume that you have a file called .github/workflows/build-pr.yml that looks like this

name: Build PR

on:
  pull_request:
    branches: [ master ]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      # Other things

      - name: Set up JDK 11
        uses: actions/setup-java@v2
        with:
          java-version: 11
          distribution: zulu

      - name: Build with Gradle
        run: ./gradlew check build --stacktrace --parallel

Let’s add 3 more steps to it

      - id: get-comment-body (1)
        if: always()
        run: |
          body=$(cat build/gradle-scan.md)
          body="${body//'%'/'%25'}"
          body="${body//$'\n'/'%0A'}"
          body="${body//$'\r'/'%0D'}"
          echo ::set-output name=body::$body

      - name: Find Comment
        uses: peter-evans/find-comment@v1
        if: always()
        id: fc (2)
        with:
          issue-number: ${{ github.event.pull_request.number }}
          comment-author: 'github-actions[bot]'
          body-includes: Gradle Build Scan

      - name: Create or update comment (3)
        uses: peter-evans/create-or-update-comment@v1
        if: always()
        with:
          comment-id: ${{ steps.fc.outputs.comment-id }}
          issue-number: ${{ github.event.pull_request.number }}
          body: ${{ steps.get-comment-body.outputs.body }}
          edit-mode: replace

1	The first step extracts reads the content of the file and converts it to something that can be sent as the body of an HTTP request.
2	The second step finds if there is a previous comment we’ve made on the PR. If there is one, it captures the id of the comment so we can update it.
3	This step uses the comment and the comment id (if available), and then upserts a comment to the Pull Request.

Using GitHub Checks

This approach uses GitHub checks to achieve the same effect. The nice thing is: it annotates your Pull Request with the results of failed tests.

      - name: Publish Test Report
        uses: mikepenz/action-junit-report@v2 (1)
        if: always()
        with:
          report_paths: '**/build/test-results/test/TEST-*.xml'

Comparison

Using GitHub checks is nice - it puts things right in your Pull Request. However, it doesn’t give you access to all the information you expect to see from a JUnit report - STDOUT, STDERR and the stacktrace.

Gradle Build scans gives you access to that information. You still have to navigate one site away to get that.

The good news is you can use both at the same time.