In the previous SonarCloud post, we looked at how to set up SonarCloud code analysis for Salesforce using automatic analysis, as well as GitHub Actions to trigger scans from a CI job. If you want to include Apex test code coverage in the SonarCloud dashboard then running the scan from a CI job is required.

In addition to providing a quality dashboard in the SonarCloud web UI, scans can also display a summary within each pull request. This is known as pull request decoration.

If you have set up automatic analysis in GitHub, then results from the Sonar scan are already populating pull requests in GitHub, including bugs, vulnerabilities, security hot spots and code smells. Code coverage is missing, however, because code coverage reports are generated within a Salesforce org and need to be downloaded and shared with SonarCloud.

In this post, we will look at two approaches for building a workflow that kicks off when pull requests are created or modified and decorates the pull request with all of the SonarCloud quality metrics, including code coverage.

Simple Pull Request Workflow

This is a sample workflow that can work with projects using scratch org development. When a pull request is opened the workflow will create a new scratch org, test the build, run Apex tests and load the code coverage results into SonarCloud.

Prerequisites

In order to use the workflow below, you should have completed the following steps from the introductory post on integrating SonarCloud with Salesforce projects:

  • Create a SonarCloud account and import the GitHub project
  • Add the SONAR_TOKEN secret to your repository
  • Configure analysis parameters by adding the sonar-project.properties file to the GitHub project

In addition, we need access to a Dev Hub environment in order to create a scratch org:

  • Salesforce org with the DevHub feature enabled (developer or enterprise and up). Sign up for a free developer edition org here.

Step 1: Set Up GitHub Repository Secret

We need to add a repository secret enabling authentication to the Salesforce Dev Hub from the GitHub Action runner. Run the following command and copy the value of the ‘Sfdx Auth Url’ key

sfdx force:org:display -u {dev-hub-alias} --verbose

In the GitHub project, navigate to Settings -> Secrets and create a new repository secret called DEVHUB_SFDX_URL. Paste the value that you copied from the output of the Salesforce CLI command above.

Step 2: Create GitHub Action to Handle Pull Requests

Create a file called ci-pr.yml in the following directory of your GitHub project: .github/workflows

Add the following text to the file

name: Pull Request CI
on:
  workflow_dispatch:
  pull_request:
    types: [opened, synchronize, reopened]
    branches:
      - feature/*

jobs:
  validate-deploy-and-test:
    runs-on: ubuntu-latest
    steps:
      # Checkout the source code
      - name: "Checkout Source Code"
        uses: actions/checkout@v2

      # Install Salesforce CLI
      - name: "Install Salesforce CLI"
        run: |
          wget https://developer.salesforce.com/media/salesforce-cli/sfdx/channels/stable/sfdx-linux-x64.tar.xz
          mkdir ~/sfdx
          tar xJf sfdx-linux-x64.tar.xz -C ~/sfdx --strip-components 1
          echo "$HOME/sfdx/bin" >> $GITHUB_PATH
          ~/sfdx/bin/sfdx version
      
      # Store Secret for Dev Hub 
      - name: "Populate Auth File with DEVHUB_SFDX_URL Secret"
        shell: bash
        run: "echo ${{ secrets.DEVHUB_SFDX_URL}} > ./ENV_SFDX_URL.txt"

      # Authenticate to Dev Hub
      - name: "Authenticate Environment"
        run: sfdx auth:sfdxurl:store -f ./ENV_SFDX_URL.txt -a DevHub -d

      # Create Scratch Org
      - name: 'Create Scratch Org'
        run: sfdx force:org:create -f config/project-scratch-def.json -a scratch-org -s -d 1

      # Deploy Source to Scratch Org
      - name: 'Push Source to Scratch Org'
        run: sfdx force:source:push

      # Run Apex Tests in Scratch Org
      - name: "Run Apex Tests"
        run: sfdx force:apex:test:run -c -r json -d ./tests/apex -w 20

      # Save Code Coverage for Next Job
      - name: "Make Report Available"
        uses: actions/upload-artifact@v2
        with:
          name: apex-code-coverage
          path: tests/apex/test-result-codecoverage.json

       # Delete Scratch Org
       - name: 'Delete Scratch Org'
       run: sfdx force:org:delete -p -u scratch-org

  sonar-scan:
    needs: validate-deploy-and-test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis

      # Download the code coverage report
      - name: "Download coverage result from previous job"
        uses: actions/download-artifact@v2
        with:
          name: apex-code-coverage
          path: tests/apex

      # Use Sonar Cloud action with sonar-project.properties to scan project
      - name: "Sonar Cloud scan"
        uses: SonarSource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Workflow highlights

  • Runs whenever a pull requested is opened or updated, or can be triggered manually
  • Installs the Salesforce CLI, authenticates with the Dev Hub
  • Creates a scratch org, deploys the feature branch, then runs local tests, then downloads the coverage report
  • Deletes the scratch org
  • Uses the SonarCloud action to run the SonarCloud scan using the configuration from the sonar-project.properties file

Advanced Pull Request Workflow

The above workflow works great for building quality gates into small, simple projects. If you are working on a project that has a complicated build process, including package dependencies, multiple deployments and other scripted automations as part of the build process, then you will want to use a tool like CumulusCI to manage the build complexity.

This workflow is designed to work with projects using scratch org development, and also requires that the project is set up to use CumulusCI.

Prequisites

  • Complete the prerequisites in the Simple Pull Request Workflow and the previous post on integrating SonarCloud with Salesforce project
  • Add the DEVHUB_SFDX_URL repository secret
  • Project has a cumulusci.yml file in the root directory with project and CumulusCI configuration
  • The project builds successfully with the dev_org or ci_feature CumulusCI flows

Step 1: Add Repository Secret

You should have already added the DEVHUB_SFDX_URL secret to the repository. We need to add another repository secret to allow CumulusCI access to GitHub repositories. This is necessary when your project relies on other packages or projects and the dependencies can be managed by CumulusCI.

Create a personal access token for your GitHub account. Be sure to select repo and gist scope. You may need to authorize the token for use with SAML single sign-on if that is relevant for your GitHub account.

In the GitHub project, navigate to Settings -> Secrets and create a new repository secret called CUMULUSCI_SERVICE_GITHUB

For the value, enter the following JSON: 

{"username": "USERNAME", "password": "TOKEN", "email": "EMAIL"}

Replace USERNAME with your GitHub username, TOKEN with the Personal Access Token you just created, and EMAIL with your email address.

Note that it is fine to use your personal GitHub account for experimentation, but for most use cases you will likely want to create a dedicated GitHub account to be the service account in order to share this secret across multiple GitHub repositories.

Step 2: Create GitHub Action to Handle Pull Requests with CumulusCI

Create a file called ci-pr-advanced.yml in the following directory of your GitHub project: .github/workflows

Add the following text to the file

name: Pull Request to Main Branch
on:
  workflow_dispatch:
  pull_request:
    types: [opened, synchronize, reopened]

env:
  CUMULUSCI_SERVICE_github: ${{ secrets.CUMULUSCI_SERVICE_github }}

jobs:
  validate-build:
    name: validate-deploy-and-test
    runs-on: ubuntu-latest
    steps:
      # Checkout the source code
      - name: "Checkout source code"
        uses: actions/checkout@v2

      # Install Salesforce CLI & Authorize DevHub
      - name: "Install Salesforce CLI"
        run: |
          mkdir sfdx
          wget -qO- https://developer.salesforce.com/media/salesforce-cli/sfdx/channels/stable/sfdx-linux-x64.tar.xz | tar xJ -C sfdx --strip-components 1
          echo $(realpath sfdx/bin) >> $GITHUB_PATH
      - name: Authenticate Dev Hub
        run: |
          echo ${{ secrets.DEVHUB_SFDX_URL }} > ./ENV_SFDX_URL.txt
          sfdx force:auth:sfdxurl:store -f ./ENV_SFDX_URL.txt -d

      # Install CumulusCI & Dependencies
      - name: Set up Python
        uses: actions/setup-python@v1
        with:
          python-version: "3.8"
      - name: Install CumulusCI
        run: |
          python -m pip install -U pip
          pip install cumulusci

      # Create directory to save coverage report if necessary
      - name: Create Directory for Test Results
        run: |
          mkdir -p tests/apex

      # Build Scratch Org & Get Coverage Report using CCI Flow
      - name: Build Org and Execute Tests
        run: |
          cci flow run ci_feature --org dev

      # Delete Scratch org
      - name: Delete Scratch Org
        if: ${{ always() }}
        run: |
          cci org scratch_delete dev

      # Save code coverage for next job
      - name: "Make report available"
        uses: actions/upload-artifact@v2
        with:
          name: apex-code-coverage
          path: tests/apex/test-result-codecoverage.json # Wildcards can be used to filter the files copied into the container. See: https://github.com/actions/upload-artifact

  sonar-scan:
    needs: validate-build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis

      # Download the code coverage report
      - name: "Download coverage result from previous job"
        uses: actions/download-artifact@v2
        with:
          name: apex-code-coverage
          path: tests/apex

      # Use Sonar Cloud action with sonar-project.properties to scan project
      - name: "Sonar Cloud scan"
        uses: SonarSource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Workflow Highlights

  • Runs whenever a pull requested is opened or updated, or can be triggered manually
  • Installs the Salesforce CLI, authenticates with the Dev Hub, installs CumulusCI
  • Runs the ci_feature flow, which creates a scratch org, installs dependencies, deploys the feature branch, runs pre and post deploy scripts, then runs local tests, then downloads the coverage report
  • Deletes the scratch org
  • Uses the SonarCloud action to run the SonarCloud scan using the configuration from the sonar-project.properties file

Step 3: Update CumulusCI Configuration

The standard behavior of the ci_feature flow is to run tests with the run_tests task as the 4th step in the flow. I haven’t been able to get that command to download the code coverage report with detailed coverage data, and so I have changed the behavior of the ci_feature flow to use the standard Salesforce CLI command for running tests and downloading the report instead.

Open the cumulusci.yml file in the root of your project. Add the following to the bottom of the file

flows:
    ci_feature:
        steps:
            4:
                task: dx
                options:
                    command: force:apex:test:run
                    extra: --outputdir ./tests/apex --wait 20 --codecoverage --resultformat json

Note that if you already have a flows section in the file, just add the ci_feature configuration within that section.

Summary

With either of these workflows, SonarCloud will add a comment to the pull request with a summary of the scan metrics, including code coverage.

SonarCloud Pull Request Decoration in GitHub for Salesforce project.
SonarCloud pull request decoration in GitHub

Note that the code coverage is only calculated on a pull request when there are new lines of actual code (not comments) added to a file that is included in the analysis scope. For example, in the initial setup of this project in SonarCloud, we included files that end in .trigger and .cls that don’t include “test” in the file name as the scope of the analysis.

Detailed information about the new lines of code and related coverage are available in the SonarCloud UI.

SonarCloud pull request analysis with code coverage in Salesforce project
SonarCloud pull request analysis with code coverage data

Resources

Tags

Leave a Comment

Your email address will not be published. Required fields are marked *