Scanning GitLab Repositories for Secrets Using Custom Secret Detection Rules

The article introduces the need for a critical security control when managing application development pipelines: protecting secrets like usernames, passwords, API keys, JSON web tokens, and private keys that developers may accidentally commit to a repository.

It shows how to identify or create a target repository (e.g., a public GitLab project) and configure a .gitlab/secret-detection-ruleset.toml file with custom rules for detecting generic passwords, JWTs, and other credentials.

[secrets]
description = 'CMG DevSecOps Crash Course, secrets custom rules configuration'

[[secrets.passthrough]]
type = "raw"
target = "gitleaks.toml"
value = """
title = "gitleaks config"
# Add regexes to the regex table
[[rules]]
description = "Test for Raw Custom Rulesets"
regex = '''Custom Raw Ruleset T[est]{3}'''

[[rules]]
description = "CMG Custom Generic Password"
regex = '''(?i)(password|passw)'''
tags = ["key", "Custom Password", "generic"]
"""

Additional custom rules are added for detecting JWTs and other patterns, with examples of the TOML syntax.

[[rules]]
description = "CMG Generic JWT"
regex = '''(?i)(jwt|jwt_token|secret|bearer)'''
tags = ["key", "JWT", "generic"]

A pipeline configuration template is included to run the secret detection job:

# This file is a template, and might need editing before it works on your project.
# This template moved to Jobs/Secret-Detection.gitlab-ci.yml in GitLab 14.0
# Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/292977
include:
  template: Jobs/Secret-Detection.gitlab-ci.yml

Test secret files (a JSON file with a password, a private key, and an API key) are added to the repository to demonstrate detection.

{
  "username": "SecurityBooBoo",
  "password": "Oh-Y3aH_CmG_2022-123"
}

-----BEGIN OPENSSH PRIVATE KEY-----
... (private key content) ...
-----END OPENSSH PRIVATE KEY-----

Apikey=33219b16f39fc40d3a1be5e9e6f56abe

The pipeline is triggered, and the article shows how to view the job results in the GitLab UI, including screenshots of the CI/CD pipeline, the secret_detection stage, and the output logs.

After the pipeline finishes, the generated secret_detection.json artifact can be downloaded, opened, and examined. Sample JSON output is provided, showing detected secrets with details such as ID, category, name, message, severity, confidence, source code extract, scanner information, and file location.

The guide concludes that while secret detection is valuable, it does not replace pre‑commit scanning or using a vault for secret management, and fixing leaked credentials can be painful.