Overview

CodeScene's PR Refactoring Agent fixes Code Health issues detected in your pull requests. When a PR fails your quality gates, a reviewer can trigger the agent, which runs in your own CI environment, refactors the affected code, and commits the changes back to the same PR branch.

This article covers what is currently supported, what you need before you start, how setup works on each platform, how to get Verified (signed) commits on GitHub, and how to troubleshoot the most common problems.

New to the agent? Start with Getting Started with CodeScene's PR Refactoring Agent.

What is currently supported

Note: Azure DevOps support was added in CodeScene 7.5.11. If you run CodeScene On-premise, check that your version includes it.

CodeScene deployment: The agent works with both CodeScene Cloud and CodeScene On-premise (self-hosted).

AI providers: The agent uses the LLM you configure. Supported providers are:

• Anthropic (anthropic/...)

• OpenAI (openai/...)

• Google (google/...)

• GitHub Copilot (github-copilot/...)

CodeScene recommends using a modern LLM with strong coding capabilities.

Runners: The agent runs on Linux runners/agents only (amd64 or aarch64)

Skills: The agent supports two workflows:

• skill:fix-code-health-degradations – fixes only the Code Health regressions introduced by the PR, without touching pre-existing technical debt.

• skill:uplift-code-health – raises Code Health for selected files toward a target score, in incremental steps.

Prerequisites

Before you set up the agent, make sure you have:

1. CodeScene PR integration enabled for the repository, so CodeScene reviews your PRs and reports quality gate results.

2. A CodeScene Personal Access Token (PAT)

   • Cloud: create one at codescene.io/users/me/pat.

   • On-prem: go to Configuration → Authentication → Personal Access Tokens in your CodeScene instance. You will also need your CodeScene On-prem URL.

3. An API key for your AI provider (Anthropic, OpenAI or Google), or a GitHub Copilot subscription if you use Copilot models (see Using GitHub Copilot below).

4. Access to the model you intend to use. The model must be available and enabled for your account or organization with your AI provider.

5. Permission to add CI configuration and secrets to the repository (GitHub Actions secrets, GitLab CI/CD variables, or an Azure Pipelines variable group).

6. A provider token with write access to the PR branch:

Tip: Your code stays on your own runner. Credentials are managed as CI secrets, and the agent does not send source code to any third party other than the AI provider you configure. See the PR Refactoring Agent documentation for privacy and security details.

Setup

The exact setup steps are maintained in each platform's agent repository. Always follow the README in the repository for your platform, and use the latest released version shown there.

⚠️ Important: Don't copy workflow examples from old internal wikis, blog posts or other repositories. An outdated version of the agent is a common cause of failed runs.

GitHub

1. Add your secrets under Settings → Secrets and variables → Actions:

   • CODESCENE_ACCESS_TOKEN (your CodeScene PAT)

   • At least one AI provider secret: ANTHROPIC_API_KEY, OPENAI_API_KEY, GOOGLE_API_KEY or OPENCODE_AUTH_JSON (GitHub Copilot)

2. Add the workflow file .github/workflows/refactoring-agent.yml from the README, with the permissions block shown above.

3. Set the model input, including the provider prefix, for example anthropic/<model-name>.

4. On-prem only: set codescene_onprem_url. If your instance uses custom certificates, also set ca_bundle.

5. Trigger the agent by clicking Fix Code Health degradations in CodeScene's PR comment, or by commenting /cs-agent on the pull request.

GitLab

1. Add CI/CD variables under Settings → CI/CD → Variables: CS_ACCESS_TOKEN, GITLAB_TOKEN, COMMAND, MODEL and your AI provider key.

2. Include the agent template in your .gitlab-ci.yml, as shown in the README.

3. On-prem only: set CODESCENE_ONPREM_URL, and CA_BUNDLE if you use custom certificates.

4. Trigger the agent by clicking the play button on the manual refactoring job in the merge request pipeline.

Azure DevOps

1. Create a GitHub service connection under Project Settings → Service connections. The pipeline uses it to fetch the agent template.

2. Add a variable group under Pipelines → Library with CS_ACCESS_TOKEN, AZURE_TOKEN and your AI provider key.

3. Add the pipeline from the README.

4. On-prem only: set CS_ONPREM_URL.

5. Run the agent from Pipelines → your pipeline → Run pipeline, entering the PR ID and target branch.

Note: The name of the On-prem URL setting differs by platform: codescene_onprem_url (GitHub), CODESCENE_ONPREM_URL (GitLab) and CS_ONPREM_URL (Azure DevOps).

Using GitHub Copilot

GitHub Copilot only supports an interactive OAuth device flow. There is no officially supported API key or PAT flow for CI use. To use Copilot models:

1. Run opencode auth login locally, choose GitHub Copilot, and complete the device authorization.

2. Copy the github-copilot object from ~/.local/share/opencode/auth.json as single-line JSON.

3. Save it as the OPENCODE_AUTH_JSON secret.
   

⚠️ Note: Copilot tokens can expire. If the agent stops authenticating, re-run opencode auth login and update the secret. Agent runs are billed to the Copilot account whose credentials you store, not to the developer who triggered the run.

Signed and Verified commits (GitHub)

If your GitHub organization uses a ruleset or branch protection that requires signed commits, use agent version v1.1.4 or later.

How it works: When the PR branch is in the same repository, the agent creates the commit through GitHub's Git Database API, without supplying a custom author, committer or signature. GitHub then signs the commit itself and shows it as Verified. You don't need to provide a GPG or SSH key.

Requirements

• The workflow has permissions: contents: write.

• The agent uses the default ${{ github.token }}, or a GitHub App installation token with:

  • Contents: Read and write

  • Permission under your branch protection and rulesets to update the PR branch

  • Access to the repository that owns the PR branch

Limitations

• A Personal Access Token can authorize writes, but it doesn't give GitHub an app identity to sign the commit, so commits made with a PAT are not Verified.

• Forks: the agent can't create a signed commit in a fork using the base repository's token.

• Fallback: if the token is missing or GitHub rejects the API operation, the agent falls back to a regular local git commit and push. That commit is not signed, so a repository that requires signed commits may reject the push.

How to check: Look in the workflow logs for created GitHub-verified commit. If it's missing, the signed path wasn't used.

Note: Signed commits are currently documented for GitHub only.

Common problems and symptoms

Troubleshooting steps

1. Check that the run started, and read its logs

• GitHub: open the Actions tab and find the refactoring agent run for your PR.

• GitLab: open the merge request pipeline and the refactoring job.

• Azure DevOps: open the pipeline run.

If no run exists, check that the workflow file is on the branch that GitHub uses for issue_comment events (normally the default branch), and that /cs-agent appears in the triggering comment.

2. Turn on detailed logs

Set print_logs: true (GitHub), PRINT_LOGS (GitLab) or the print_logs parameter (Azure DevOps) to include the agent's detailed logs in the job output, then re-run.

Tip: Before refactoring, the agent runs preflight checks for Anthropic, OpenAI, Google and GitHub Copilot. These confirm that your AI provider is reachable and that the model exists, and stop the run early if not. Errors from other providers appear in the workflow logs instead.

3. Fix model errors

• Always include the provider prefix, for example anthropic/..., openai/..., google/... or github-copilot/....

• Check that the model name exactly matches a model your provider offers. For GitHub Copilot, the available model names are listed at models.dev/providers/github-copilot.

• Check that the model is available and enabled for your organization in your AI provider's console. If the provider rejects the model, CodeScene can't work around it.

4. Refresh expired Copilot credentials

Re-run opencode auth login locally and update the OPENCODE_AUTH_JSON secret (see Using GitHub Copilot).

5. Fix rejected pushes on repositories that require signed commits

1. Upgrade to agent v1.1.4 or later.

2. Use the default ${{ github.token }} or a GitHub App installation token, not a PAT.

3. Make sure the workflow has contents: write and that your rulesets allow the token to update the PR branch.

4. Confirm the PR branch isn't in a fork.

5. Re-run and look for created GitHub-verified commit in the logs.

6. Add missing CI variables

Check the variable names against your platform's README. On Azure DevOps, make sure the variable group is linked to the pipeline. On GitLab, check whether the variables are marked Protected: protected variables are only available to pipelines on protected branches.

7. Check the runner

Use a Linux runner (amd64 or aarch64), such as ubuntu-latest or a self-hosted Linux agent. The runner must be able to download the agent release from GitHub (github.com/codescene-oss/refactoring-agent-releases), so allow this through your network or proxy.

8. Configure certificates for On-prem

If your CodeScene instance uses a custom or internal certificate authority, pass a PEM bundle via ca_bundle (GitHub) or CA_BUNDLE (GitLab).

9. If the "Install it" message persists

Check that the workflow file is committed to the repository and references the official codescene-oss/pr-refactoring-agent action.

If your organization doesn't want to use the agent, you can disable the "Our agent can fix these. Install it" message.

10. Allow time on large repositories

The agent clones the repository as part of the run, so large repositories with deep Git history take longer before refactoring and validation finish. Check the run's progress in your CI logs before re-triggering it.

11. Re-running other checks after the agent's commit

GitHub doesn't start new workflow runs for pushes made with the default GITHUB_TOKEN. If you need your other checks to run on the agent's commit, re-run them manually or push a follow-up commit.

What to send to CodeScene Support

If the problem persists, contact CodeScene Support and include:

• Deployment: CodeScene Cloud, or On-prem plus your CodeScene version

• Git provider: GitHub, GitLab (including whether it's self-hosted) or Azure DevOps

• Agent version: the tag or version you reference in your workflow or pipeline

• The PR/MR link where you triggered the agent

• How you triggered it: the Fix button, a /cs-agent comment, the GitLab play button or a manual Azure pipeline run

• The model string you configured (for example anthropic/...) and your AI provider

• Your workflow or pipeline file, with all secrets and tokens removed

• The full run log, with print_logs enabled, from the run that failed

• The token type you use for the git provider (default github.token, GitHub App, PAT, or project access token) and its permissions or scopes

• Branch rules: whether your repository requires signed commits or uses rulesets or branch protection on the PR branch, and whether the PR comes from a fork

• Network: whether you use a proxy or custom certificates

• What you saw: screenshots of the CodeScene PR comment (for example, "Install it" versus "Fix")

Need More Help?

• Cloud customers: open the support widget in the bottom-right corner and start a chat with Eve.

• Enterprise support customers: contact us through your usual support channel.

• You can also ask questions in the CodeScene Community or open an issue in the agent's GitHub repository.

Related resources

• Getting Started with CodeScene's PR Refactoring Agent

• PR Refactoring Agent documentation (Cloud)

• PR Refactoring Agent documentation (On-prem)

• Disable the "Our agent can fix these. Install it" message