Implement --registry flag for scanning all images in a Docker registry#4898
Open
jainlakshya wants to merge 12 commits intotrufflesecurity:mainfrom
Open
Implement --registry flag for scanning all images in a Docker registry#4898jainlakshya wants to merge 12 commits intotrufflesecurity:mainfrom
jainlakshya wants to merge 12 commits intotrufflesecurity:mainfrom
Conversation
Implements GitHub issue trufflesecurity#1739 Added --registry flag to scan all images in an OCI Distribution Spec compliant registry (Harbor, Nexus, Artifactory, etc.) using the /v2/_catalog endpoint. Changes: - Added --registry CLI flag with validation - Implemented GenericOCIRegistry for /v2/_catalog enumeration - Added Link header pagination support - Added bearer token authentication via --registry-token - Fixed UseDockerKeychain logic to not activate for registry scans - Added comprehensive test coverage Usage: trufflehog docker --registry registry.example.com trufflehog docker --registry harbor.corp.io --registry-token <token> Resolves: trufflesecurity#1739
|
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Fixed pagination bug where relative URLs from OCI Distribution Spec registries (Docker Distribution, Harbor, Nexus) would fail with 'unsupported protocol scheme' error. The OCI spec allows registries to return relative URLs in Link headers like </v2/_catalog?n=100&last=repo100>; rel="next". These need to be resolved against the base URL before making the next request. Changes: - Added resolveNextURL() to resolve relative URLs against base URL - Modified ListImages() to use URL resolution for pagination - Updated test to use relative URL (matching real OCI behavior) - Added test for absolute URL pagination (GHCR-style) Both relative and absolute URLs now work correctly.
Only disable Docker keychain for registry scans (which use registry API token). Namespace and image scans should still use Docker keychain when no bearer token is provided.
Previously resolveNextURL silently returned empty string on parse errors, causing pagination to stop without warning and potentially skipping repositories. Now returns explicit error so caller knows scanning is incomplete.
Add validation to prevent --registry from accepting public registry hosts (hub.docker.com, quay.io, ghcr.io) since they are already properly handled by --namespace with dedicated implementations using custom APIs. Public registries use different endpoints: - DockerHub: /v2/namespaces/<ns>/repositories - Quay: /api/v1/repository?namespace=<ns> - GHCR: api.github.com/users/<ns>/packages The --registry flag is designed for private OCI registries (Harbor, Nexus, Artifactory) that implement the standard /v2/_catalog endpoint.
Author
|
@kashifkhan0771 You're absolutely right. I've added validation to prevent --registry from accepting public registries (11d4342). Now if someone tries --registry hub.docker.com, they'll get a helpful error pointing them to use --namespace instead. The issue was that public registries use completely different APIs (DockerHub uses /v2/namespaces//repositories, not the standard /v2/_catalog), so using --registry with them would either fail or give weird results. |
Add sanitizeRegistryHost function to strip protocol prefixes and paths from --registry values before passing to DockerConfig. This prevents malformed URLs like https://https://harbor.corp.io/v2/_catalog. Users can now provide registry hosts in any format: - --registry https://harbor.corp.io - --registry http://localhost:5000 - --registry harbor.corp.io All are sanitized to clean hostnames for proper URL construction.
Use case-insensitive prefix detection to prevent hostname corruption when users provide mixed-case protocols like HTTPS:// or Http://. Previously, these would fail to match TrimPrefix and the path-stripping logic would truncate at the first / in ://, producing garbage like HTTPS: or Http:. Now correctly handles: - HTTPS://harbor.corp.io -> harbor.corp.io - Http://localhost:5000 -> localhost:5000 - HtTpS://registry.io -> registry.io
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Reviewed by Cursor Bugbot for commit 3f5dc7c. Configure here.
…ization 1. Extract normalizeRegistryHost helper to eliminate duplicated URL sanitization logic between isPublicRegistry and sanitizeRegistryHost. This reduces maintenance burden and ensures consistent behavior. 2. Add validation after sanitization to catch empty registry values that pass initial validation but become empty after normalization (e.g., 'https://', ' ', 'http://'). This prevents silent no-op scans with confusing behavior. Both functions now use the same normalization logic, and invalid inputs are caught with clear error messages.
Contributor
|
You need to run |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements #1739 - Adds --registry flag to scan all images in a Docker registry, similar to trufflehog github --org=...
Motivation
Currently, TruffleHog can only scan specific Docker images via --image or images within a namespace via --namespace (limited to DockerHub, GHCR, and Quay). This PR adds support for scanning entire private registries (Harbor, Nexus, Artifactory, etc.) by enumerating all repositories using the OCI Distribution Spec /v2/_catalog endpoint.
Changes
Usage Examples
Scan all images in a private registry (unauthenticated):
trufflehog docker --registry registry.example.com
Scan with authentication:
trufflehog docker --registry harbor.corp.io --registry-token
Scan with both registry token and image pull token:
trufflehog docker --registry nexus.internal --registry-token --token
Technical Details
Testing
Breaking Changes
None. This is a purely additive feature.
Checklist
Note
Medium Risk
Adds new network enumeration behavior and expands the Docker scan input surface (new CLI flag + protobuf field), which could impact scan coverage/perf and registry auth edge cases, but is largely additive and well-tested.
Overview
Adds a new
trufflehog docker --registrymode to scan all repositories in an OCI Distribution Spec–compatible registry by enumerating/v2/_catalog(with Link-header pagination) and then scanning each discovered image.Plumbs the new
registryfield through CLI/config/protobuf (sources.proto,sourcespb.Docker,sources.DockerConfig,Engine.ScanDocker) and tightens CLI validation/sanitization (mutual exclusion with--image/--namespace, blocks known public registries, requires--registry-tokento be paired with--namespace/--registry, and avoidsUseDockerKeychainfor registry scans). Includes unit tests covering listing, pagination, auth header handling, and error cases for the generic registry implementation.Reviewed by Cursor Bugbot for commit b44515c. Bugbot is set up for automated code reviews on this repo. Configure here.