Integrating CodeScene with Azure DevOps Pull Requests

You can integrate CodeScene with Azure DevOps to automatically run delta analyses on pull requests. This provides automated feedback through quality gates, helping your team enforce code health and quality standards early in the development process.

1. Prerequisites

Before enabling the integration, make sure that:

  • The CodeScene host URL is configured under
    Global Configuration → System → Host/Network Settings.

  • Your CodeScene instance is accessible from Azure DevOps, since Azure needs to send service hooks (webhooks) to CodeScene.

  • If you’re running CodeScene locally (for example, using the Docker trial setup), see the Important Note for On-Premises Trials at the end of this article.

2. Enable Pull Request Integration

  1. Go to your Projects page in CodeScene and locate the desired project.

  2. Open Project Configuration → PR Integration.

  3. Enable the Pull Request Integration.

  4. Select Azure DevOps as the integration type.

3. Create a Personal Access Token (PAT) in Azure DevOps

In Azure DevOps, API tokens are referred to as Personal Access Tokens (PATs).
Follow these steps to create one:

  1. Go to https://dev.azure.com.

  2. In the top-right corner, click on User Settings (⚙️) → Personal Access Tokens.

  3. Click + New Token.

  4. Enter a name (for example, CodeScene Integration).

  5. Choose your organization.

  6. Set an expiration date (30, 60, 90 days, or custom).

  7. Under Scopes, select Custom-defined permissions, then enable:

    • Code: Read & Write

    • Service Connections: Read, query, & manage

  8. Click Create.

  9. Copy your PAT and store it securely (for example, in a password manager), since it will not be shown again.

4. Configure Azure Credentials in CodeScene

In the Pull Request Integration tab, configure your Azure credentials using one of the following methods:

Option A: Use an Azure Connection Preset (Recommended)

  • Select an existing Connection Settings Preset for Azure if you’ve already created one.

  • Learn more about connection presets here.

  • Click Save Configuration.

Option B: Configure Credentials Manually

Under the Azure Credentials section, provide the following:

  • Azure Personal Access Token - must include these permissions:

    • Code: Read & Write

    • Service Connections: Read, query, & manage

Then toggle Advanced Settings and verify:

  • Azure API URL – Leave blank to use the default API URL for your Azure organization.

  • CodeScene Host URL – Used by Azure service hooks to reach your CodeScene instance (leave blank to use the global host URL).

  • Click Save Configuration.

5. Optional: Configure PR Feedback Options

CodeScene can provide several types of automated feedback on your Azure pull requests.

Add Threads to Specific File Locations

Instead of posting one overall comment, CodeScene can create comment threads on specific lines where code degradations or improvements are detected.

Set Comment Thread Status to “Active” for Degradations

By default, CodeScene sets comment threads to Active for findings that degrade Code Health. You can control this behavior using this setting.

Set Main Comment Status to “Active” for High-Risk Pull Requests

When enabled, high-risk changes have their main comment thread marked as Active, even if there are no degradations.

6. Customize PR Feedback

You can customize what CodeScene checks in each pull request by adjusting your Code Quality Profile:

  1. In your project, navigate to Configuration → Delta Analysis.

  2. Select a Code Quality Profile that matches your project’s standards. We recommend starting with the Bare Minimum profile.

  3. Learn more about quality profiles here.

7. Test the Configuration

After configuration, test that everything works correctly:

  1. Create a Pull Request in Azure DevOps.

  2. Verify that CodeScene:

    • Posts a review comment in the PR discussion thread.

Important Note for On-Premises Trials

When running CodeScene locally (for example, via a Docker trial setup):

  • Azure DevOps service hooks require inbound communication from Azure’s servers to your CodeScene instance.

  • If your instance isn’t publicly accessible, Azure cannot deliver webhook events.

Workaround: Use a Tunneling Service

You can use a tool like ngrok to temporarily expose your local CodeScene instance to the internet:

  • ngrok creates a secure tunnel between Azure DevOps and CodeScene.

  • This allows service hook events to work without modifying your firewall or public IP configuration.