You can’t secure what you don’t understand. And with container images, understanding means knowing exactly what’s inside — every package, every library, every potential vulnerability.
Most teams treat their container images as black boxes. They pull a base image, add their code, and push it to production. But that base image? It contains hundreds of packages you didn’t explicitly choose. Any of them could have known vulnerabilities.
Trivy makes the invisible visible. It’s an open-source vulnerability scanner that tells you exactly what’s in your images and what risks they carry.
Why Trivy?
There are many container scanners. I chose Trivy because:
- Single binary — No database server, no complex setup
- Speed — Scans complete in seconds, not minutes
- Comprehensive — OS packages, language dependencies, IaC misconfigurations
- CI-friendly — Exit codes and multiple output formats
- Free — No licensing complexity
Other options like Snyk, Anchore, or Clair are valid choices, but Trivy’s simplicity fits the low-friction philosophy I value.
Basic Pipeline Integration
Here’s a minimal GitLab CI configuration:
stages:
- build
- scan
- push
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
build:
stage: build
image: docker:24
services:
- docker:24-dind
script:
- docker build -t $IMAGE_TAG .
- docker save $IMAGE_TAG -o image.tar
artifacts:
paths:
- image.tar
expire_in: 1 hour
scan:
stage: scan
image: aquasec/trivy:latest
script:
- trivy image --input image.tar --exit-code 1 --severity HIGH,CRITICAL
allow_failure: false
push:
stage: push
image: docker:24
services:
- docker:24-dind
script:
- docker load -i image.tar
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker push $IMAGE_TAG
needs:
- build
- scan
Key points:
- Build first, scan second — The image is saved as a tarball artifact
- Exit code 1 — Fails the pipeline on HIGH or CRITICAL vulnerabilities
- Push only after scan passes — No vulnerable images reach the registry
Understanding the Output
When Trivy finds vulnerabilities, you get something like this:
┌──────────────────┬────────────────┬──────────┬────────────────────────────────────┐
│ Library │ Vulnerability │ Severity │ Installed Version │
├──────────────────┼────────────────┼──────────┼────────────────────────────────────┤
│ openssl │ CVE-2024-0727 │ HIGH │ 3.0.2-0ubuntu1.10 │
│ libcurl4 │ CVE-2024-2398 │ MEDIUM │ 7.81.0-1ubuntu1.15 │
│ python3.10 │ CVE-2024-0450 │ HIGH │ 3.10.12-1~22.04.3 │
└──────────────────┴────────────────┴──────────┴────────────────────────────────────┘
This is understanding. You now know:
- Which packages have issues
- How severe those issues are
- Which versions are affected
Without this visibility, you’re hoping nothing bad is running in your containers.
Severity Thresholds
Not every vulnerability deserves pipeline failure. Configure thresholds based on your risk tolerance:
# Fail only on critical - fast but risky
trivy image --exit-code 1 --severity CRITICAL
# Fail on high and critical - balanced
trivy image --exit-code 1 --severity HIGH,CRITICAL
# Fail on medium and above - strict
trivy image --exit-code 1 --severity MEDIUM,HIGH,CRITICAL
My recommendation: Start with HIGH,CRITICAL. You can tighten later, but starting too strict leads to alert fatigue and ignored warnings.
Handling False Positives
Sometimes Trivy reports vulnerabilities that don’t apply to your use case. Handle them with a .trivyignore file:
# CVE doesn't affect our code path
CVE-2024-0727
# No fix available yet, tracking in JIRA-1234
CVE-2024-2398
Add this to your scan:
trivy image --input image.tar --ignorefile .trivyignore --exit-code 1
Important: Document why you’re ignoring each CVE. Future you will thank present you.
Output Formats for Different Needs
Trivy supports multiple output formats:
# Human readable table (default)
trivy image --format table $IMAGE_TAG
# JSON for programmatic processing
trivy image --format json --output results.json $IMAGE_TAG
# SARIF for GitHub/GitLab security dashboards
trivy image --format sarif --output trivy.sarif $IMAGE_TAG
# HTML report for stakeholders
trivy image --format template --template "@contrib/html.tpl" --output report.html $IMAGE_TAG
For GitLab’s security dashboard integration:
scan:
stage: scan
image: aquasec/trivy:latest
script:
- trivy image --format template --template "@/contrib/gitlab.tpl" --output gl-container-scanning-report.json $IMAGE_TAG
- trivy image --exit-code 1 --severity HIGH,CRITICAL $IMAGE_TAG
artifacts:
reports:
container_scanning: gl-container-scanning-report.json
Now vulnerability findings appear directly in merge requests.
Scanning More Than Images
Trivy isn’t limited to container images. It scans:
# Filesystem (your project directory)
trivy fs --exit-code 1 .
# Git repository
trivy repo https://github.com/your/repo
# Kubernetes manifests
trivy config ./k8s/
# Infrastructure as Code
trivy config ./terraform/
A complete security stage might look like:
security-scan:
stage: scan
image: aquasec/trivy:latest
script:
# Scan source code dependencies
- trivy fs --exit-code 0 --severity HIGH,CRITICAL . --format table
# Scan Kubernetes manifests for misconfigurations
- trivy config ./k8s/ --exit-code 0 --severity HIGH,CRITICAL
# Scan container image (this one fails the build)
- trivy image --input image.tar --exit-code 1 --severity HIGH,CRITICAL
Database Caching
Trivy downloads vulnerability databases on each run. In CI, this adds latency. Cache it:
variables:
TRIVY_CACHE_DIR: .trivy-cache
scan:
stage: scan
image: aquasec/trivy:latest
cache:
key: trivy-db
paths:
- .trivy-cache
script:
- trivy image --cache-dir $TRIVY_CACHE_DIR --input image.tar
Or use a pre-populated image:
scan:
image: aquasec/trivy:latest
before_script:
- trivy image --download-db-only
script:
- trivy image --skip-db-update --input image.tar
My Production Configuration
Here’s what I actually run:
stages:
- build
- scan
- push
- deploy
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
TRIVY_CACHE_DIR: .trivy-cache
# Don't fail on unfixed vulnerabilities
TRIVY_IGNORE_UNFIXED: "true"
.trivy-template: &trivy-template
image: aquasec/trivy:latest
cache:
key: trivy-db
paths:
- .trivy-cache
scan-image:
<<: *trivy-template
stage: scan
script:
# Generate report for GitLab dashboard
- trivy image
--cache-dir $TRIVY_CACHE_DIR
--format template
--template "@/contrib/gitlab.tpl"
--output gl-container-scanning-report.json
--input image.tar
# Fail on fixable HIGH/CRITICAL
- trivy image
--cache-dir $TRIVY_CACHE_DIR
--exit-code 1
--severity HIGH,CRITICAL
--ignore-unfixed
--input image.tar
artifacts:
reports:
container_scanning: gl-container-scanning-report.json
needs:
- build
scan-config:
<<: *trivy-template
stage: scan
script:
- trivy config
--cache-dir $TRIVY_CACHE_DIR
--exit-code 1
--severity HIGH,CRITICAL
./k8s/
allow_failure: true # Warning only, don't block
Key decisions:
--ignore-unfixed— No point failing on vulnerabilities with no available fix- Config scan as warning — Misconfigurations are important but shouldn’t block every deployment
- Cached database — Faster builds, fewer network dependencies
Shifting Left vs. Gate Keeping
There are two philosophies for security scanning:
Gate keeping (what we’ve discussed):
- Scan at build time
- Block vulnerable images from registry
- Enforce standards before deployment
Shift left:
- Scan during development
- IDE plugins and pre-commit hooks
- Faster feedback loops
I recommend both. Gate keeping catches what slips through, but developers should see issues before they commit:
# Developer workflow
trivy fs . # Scan before commit
docker build -t myapp .
trivy image myapp # Scan before push
The pipeline is the safety net, not the primary feedback mechanism.
Integration with GitOps
If you’re using ArgoCD for deployments, add scanning to your promotion workflow:
promote-to-prod:
stage: deploy
script:
# Re-scan the specific image version before promotion
- trivy image --exit-code 1 --severity CRITICAL $IMAGE_TAG
# Update GitOps repo only if scan passes
- |
cd gitops-repo
kustomize edit set image myapp:${CI_COMMIT_TAG}
git commit -am "Promote ${CI_COMMIT_TAG} to production"
git push
rules:
- if: $CI_COMMIT_TAG =~ /^v[0-9]+/
This ensures images are rescanned before production promotion — vulnerabilities discovered after initial build won’t slip through.
Beyond Vulnerabilities: SBOM
A Software Bill of Materials (SBOM) is an inventory of everything in your image. Trivy can generate one:
trivy image --format spdx-json --output sbom.json $IMAGE_TAG
Why care about SBOMs?
- Compliance — Some industries require them
- Incident response — When a new CVE drops, you can quickly check all images
- Supply chain visibility — Know your dependencies
Store SBOMs alongside your images in your registry.
Common Pitfalls
Scanning the wrong layer
# Wrong - scans the trivy image itself
trivy image aquasec/trivy:latest
# Right - scans your built image
trivy image --input image.tar
Ignoring the exit code
# Wrong - always succeeds
trivy image $IMAGE_TAG || true
# Right - fail on findings
trivy image --exit-code 1 $IMAGE_TAG
Alert fatigue
Starting with MEDIUM severity and no ignores leads to hundreds of findings. Teams give up. Start strict on critical, expand gradually.
Why This Matters
Container scanning isn’t about compliance checkboxes. It’s about understanding what you’re running.
When a new critical CVE is announced, you want to know:
- Which of my images are affected?
- Which services run those images?
- How quickly can I patch?
Without scanning, these questions require manual investigation across every image. With Trivy in your pipeline, you have continuous visibility.
Security through understanding, not security through hope.
You can’t protect what you can’t see. Container scanning makes the contents of your images visible, turning black boxes into documented, auditable systems.
