Setting up the Pull Request Integration for Context-Aware Quality Gates

CodeScene's Pull Request (PR) integration acts as an automated defense mechanism against technical debt, ensuring that all new and modified code aligns with your organization’s goals and quality standards. This integration automates the Code Health review process, simplifying code reviews and enhancing code maintainability and readability.

Primary Stakeholders

  • Developers: Benefit from automated reviews that enforce code quality standards.

  • Engineering Managers: Ensure organizational goals are met through tailored code quality profiles.

Enabling Codescene's Pull Request Integration

  1. Global Configuration: Apply a shared configuration across all CodeScene projects.

  2. Project-Specific Configuration: Override global settings with project-specific configurations.

  3. Select a Code Quality Profile: Choose between pre-defined profiles or use configuration as code for flexibility.

    1. Bare Minimum: Recommended starting point for new customers, enforcing critical Code Health rules like Low Cohesion, Deep Nested Logic, and Brain Functions. Prevents new technical debt and ensures no decline in Hotspot Code Health

    2. Clean Code Collective: Builds on the “Minimal Safety Net” with advisory rules backed by research. Prevents declines in Code Health for any file, matching the standards from the Code Red paper.

    3. Pay down tech debt: The most robust profile, enabling all checks and enforcing your specified goals (e.g., “Supervise” and “Refactoring”). Ideal for teams optimising productivity and addressing technical debt.

    4. Customizable Safeguards: Allows you to combine checks, like the “Minimal Safety Net,” with specific goals, such as targeted Code Health improvements

Custom Quality Gates: Advanced Config-as-Code

For advanced users, tailor checks by code area or team, providing precise control over quality gate application—unlike the uniform pre-defined profiles, which apply the same rules to all code inside the CodeScene project.

The configuration is specified in a JSON file and checked into version-control inside the Git repository where the config applies. The file must be placed under a .codescene/ directory in the repository root. The name of the JSON file is .codescene/custom-quality-gates.json.

Use case: Apply different quality profiles to different parts of the codebase

The example JSON file below illustrates how the matching_content_path filter lets you apply different quality profiles to different parts of the codebase: the src/Server component will get the “Bare Minimum” profile, while all other code must adhere to the more strict “Clean Code Collective” which includes advisory Code Health rules too.

{  "usage": "Persist this file inside your repositories as .codescene/custom-quality-gates.json",  "rule_sets": [  {  "quality_gates": "bare_minimum",  "quality_gates_doc": "Here we specify a quality profile. The options are: bare_minimum, clean_code_collective, pay_down_tech_debt.",  "matching_content_path": "src/Server/**",  "matching_content_path_doc": "Specify a glob pattern relative to this repo root."  }, {  "quality_gates": "clean_code_collective",  "quality_gates_doc": "Let's use a stricter profile for the rest of the code.",  "matching_content_path": "*/**",  "matching_content_path_doc": "Apply this rule to all code outside of src/Server."  }] }

Use case: Apply different quality profiles to different teams

The example JSON file below applies the matching_team filter to define custom rules for a fictitious “Backend Team”. Each time a member of that team modifies some code – no matter where that code is located in the repository – the corresponding quality gate gets triggered. Also note that the JSON file keeps the more strict profile for all other code. That is, if a member of any other team modifies the code, then the “Clean Code Collective” profile will be applied.

{  "usage": "Persist this file inside your repositories as .codescene/custom-quality-gates.json",  "rule_sets": [  {  "quality_gates": {  "hotspot_decline": true,  "critical_health_rules": false,  "refactoring_goals": false,  "supervise_goals": false,  "new_code_health": false,  "advisory_health_rules": false,  "codeowners_for_critical_code": false  },  "quality_gates_doc": "We only apply the gates to Hotspots inside the Server component; no other gates.",  "matching_content_path": "src/Server/**",  "matching_content_path_doc": "Specify a glob pattern relative to this repo root."  }, {  "quality_gates": "clean_code_collective",  "quality_gates_doc": "Let's use a stricter profile for all other code.",  "matching_content_path": "*/**",  "matching_content_path_doc": "Apply this rule to all other code changes."  }] }

How advanced configuration options applied

  • Rules are tried in order and the first rule that matches is applied.

  • If no rule is matched, all quality gates are disabled by default. That is, you won’t get any automated review.

  • If you want to use a different default set of gates, then specify a rule with "matching_content_path" : "**/*" as the last rule in the JSON file. For example, you could choose to have all gates disabled by default. (This use case is handy in case you roll out CodeScene in a monorepo and only want to apply the PR review on parts of the codebase).

  • The PR review always uses the .codescene/custom-quality-gates.json available on the branch. So if you make changes to the configuration on the main branch, you need to rebase any existing development branches before the profile changes are applied.

More information and examples can be found here.

Getting the Most Out of Codescene's Pull Request Integration

  • Select Appropriate Profiles: Start with "The Bare Minimum" for onboarding, then elevate to "Clean Code Collective" as teams adapt.

  • Use Custom Quality Gates: Tailor checks for different codebase sections or teams using the advanced configuration.

  • Suppress Findings: Use CodeScene’s suppression mechanism to manage findings pragmatically. 

Frequently Asked Questions

Q: Where can I find the full documentation of the PR integration?
A: Full documentation can be found here.

Q: What platforms does the integration support?
A: It supports GitHub, BitBucket, GitLab, Azure, and Gerrit (on-prem).

Q: How do I manage different quality standards for different teams?
A: Use the advanced config-as-code feature to apply specific rules to different teams or codebase sections.

Q: Can I track the impact of pull requests over time?
A: Yes, CodeScene visualizes PR impacts, showing statistics like checks performed, findings detected, and more.

Q: How do I enable status badges?
A: Configure status badges in the Project Configuration to monitor project health at a glance.

Q: If we enable the code quality profile as “configuration as code” how does it affect projects that don’t have the file present?
A: The configuration-as-code-option requires a custom-quality-gates.json file to be present in each repository. If that option is selected but the file isn’t present, then our analysis will report an error in the PRs. Our recommendation:

  • At an organizational level, use one of the pre-defined quality profiles, e.g. “minimal safety-net” or “clean code collective”.

  • Use configuration-as-code for individual projects, and ensure that all repositories within that CodeScene project contain the config file.