dependency-vulnerability-scanning-in-ci v1.1.0A

Immutable. This exact content is served forever at /api/v1/blob/ba08757104ef70c7.

---
name: dependency-vulnerability-scanning-in-ci
description: CI/CD supply chain security gate that matches a Python dependency inventory
  against registered CVE/OSV advisories using PEP 440 version ordering, blocks builds
  on CRITICAL/HIGH findings, and emits a CycloneDX 1.6 SBOM.
domain: Infrastructure & DevOps
subdomain: Supply Chain Security & CI/CD
tags:
- vulnerability-scanning
- ci-cd-security
- pip-audit
- osv-scanner
- cve-database
- sbom
- supply-chain-security
brokers_frameworks:
- pip-audit
- OSV Database
- PyPA Advisory DB
- CycloneDX
- Python Dataclasses
version: "1.1.0"
author: algo-trading-skills-contributors
license: Apache-2.0
---

## When to Use

Use this skill in continuous integration (CI/CD) pipelines, automated build checks, and production trading system deployments. Third-party Python dependencies (such as `requests`, `urllib3`, `cryptography`, `pyyaml`) frequently contain security vulnerabilities (Remote Code Execution, Denial of Service, SQL injection). Deploying vulnerable packages to production trading servers exposes trade execution APIs and private keys to compromise. This module evaluates a dependency inventory against advisory records and hard-fails CI builds on `CRITICAL` or `HIGH` severity flaws.

## When NOT to Use

- **As a replacement for `pip-audit` or `OSV-Scanner`.** This engine does **not** fetch advisories or resolve a dependency graph. It is the *policy gate* you put behind a real scanner: run `pip-audit`/`OSV-Scanner`, feed its findings in via `register_advisory()`, and let this engine apply consistent severity and fail-closed rules. Using it alone means scanning against whatever advisories you happened to register.
- **For transitive dependency discovery.** It scans the inventory you hand it. If you pass only top-level `requirements.txt` entries, nested transitive packages are never examined — export a fully resolved inventory (`pip freeze`, a lockfile, or the scanner's own resolved list) instead.
- **For non-Python ecosystems.** Version ordering is PEP 440 and purls are emitted as `pkg:pypi/...`. npm/Maven/Go versions follow different ordering rules and will be mis-compared.
- **As an SBOM system of record.** `generate_sbom()` emits a valid CycloneDX 1.6 document for the inventory it was given; it does not capture licences, hashes, provenance, or build attestations that a full SBOM toolchain produces.

## Prerequisites

- A **fully resolved** dependency inventory (`package_name`, `current_version`) — not just direct dependencies.
- CVE advisory records (`cve_id`, `package_name`, `vulnerable_below_version`, `cvss_score`, `severity`, `fixed_in_version`, and, where the advisory states one, `introduced_in_version`).

## Workflow

1. **Lockfile & Inventory Parsing**:
   - Parse package names and installed versions from a resolved lockfile.
   - An empty inventory raises `ValueError` rather than reporting a clean scan — a parser that silently returned nothing must not read as "no vulnerabilities".

2. **Advisory Registration**:
   - Load advisories from the OSV/PyPA feed. An empty advisory database also raises `ValueError`: scanning against zero advisories always reports zero findings, which is the "stale CVE database" failure mode wearing a green badge.
   - Package names are normalised per PyPI rules (lowercased, `_` → `-`) before matching, so an advisory for `ruamel-yaml` still matches an installed `Ruamel_YAML`.

3. **Affected-Range Evaluation**:
   - Compare the installed version against the advisory range using **PEP 440** ordering, not string or naive integer comparison. `2.31.0rc1` sorts *before* `2.31.0` and is therefore still vulnerable; `2.31` and `2.31.0` are equal.
   - Honour `introduced_in_version` where the advisory has a lower bound — CVE-2023-32681 affects `requests >= 2.3.0, < 2.31.0`, so a project pinned to `2.2.0` is not affected and must not be flagged.
   - If a version string cannot be parsed as PEP 440, **fail closed**: report the package as affected and record a scan warning. A gate must never clear a version it could not understand.

4. **Severity Resolution (fail-closed)**:
   - If the advisory's severity label is canonical (`CRITICAL`/`HIGH`/`MEDIUM`/`LOW`/`NONE`), use it.
   - If it is not — GitHub Security Advisories say `MODERATE`, feeds carry stray whitespace, some records have none at all — derive severity from the CVSS base score using the FIRST CVSS v3.1 rating scale. Do **not** bucket an unrecognised label into `LOW`.
   - If label and score disagree, gate on the **more severe** of the two and emit a warning.
   - If neither resolves, classify `UNKNOWN`, which blocks the build.

5. **CI Pipeline Gate & Remediation**:
   - If `CRITICAL`, `HIGH`, or `UNKNOWN` findings exist $\implies$ set `is_ci_build_passed = False` and block deployment.
   - Emit exact package upgrade remediation directives (e.g. "Upgrade `requests` 2.25.0 $\to$ 2.31.0").

6. **Audit Report & SBOM Generation**:
   - Output a structured `VulnerabilityScanReport` (including `scan_warnings`, which must be surfaced in the CI log, not swallowed).
   - Call `generate_sbom()` for a CycloneDX 1.6 JSON document. Output is deterministic — the `serialNumber` is a UUIDv5 over the component set and the timestamp is opt-in — so the SBOM diffs cleanly between builds.

> Full procedure: see `references/workflows.md`.
> Standards reference: see `references/standards.md`.
> Printable pre-flight checklist: see `assets/checklist.md`.

## Common Pitfalls

- **Ignoring Transitive Sub-Dependencies**: Scanning top-level packages while missing vulnerable transitive dependencies nested in `requirements.txt`.
- **Soft Warnings on Critical RCE Vulnerabilities**: Issuing non-blocking warnings for `CRITICAL` (CVSS $\ge 9.0$) vulnerabilities, allowing compromised builds to reach production.
- **Outdated CVE Databases**: Running CI security scans without updating the advisory database feed. A scan against a stale or empty feed returns "0 vulnerabilities" — indistinguishable from a genuinely clean build unless you assert the advisory count.
- **Comparing Versions as Strings or Split Integers**: `"2.9.0" < "2.31.0"` is `False` lexicographically, and `int()`-splitting explodes on any PEP 440 pre/post/dev suffix. Both silently clear a vulnerable package. Use PEP 440 ordering, and treat an unparseable version as affected rather than clean.
- **Bucketing Unknown Severity Labels as Low**: An `if/elif` chain over `CRITICAL`/`HIGH`/`MEDIUM` with an `else: low` fallback turns every non-canonical label — `MODERATE`, `"CRITICAL "` with a trailing space, an empty string — into a `LOW` finding that passes the gate, no matter how high the CVSS score is.
- **Treating "Vulnerable Below X" as the Whole Advisory**: Advisories are ranges. Modelling only the upper bound flags every older release, including ones predating the vulnerable code, and buries real findings in false positives.

## Verification

- Instantiate `DependencyVulnerabilityScannerEngine`. Register CVE-2023-32681 (`requests >= 2.3.0, < 2.31.0`; NVD CVSS v3.1 base score **6.1 MEDIUM**). Audit `requests==2.25.0` with `numpy==1.26.4`: the scanner flags one `MEDIUM` finding and recommends upgrading to `2.31.0`, and — because MEDIUM is below the blocking threshold — `is_ci_build_passed` stays `True`. Audit `requests==2.2.0` and confirm it is **not** flagged (below the advisory's introduced bound).
- Register an advisory with severity `"MODERATE"` and CVSS `9.8`; confirm it resolves to `CRITICAL` and blocks the build.
- Confirm `scan_dependencies([])` and a scan with no registered advisories both raise `ValueError`.
- Confirm `generate_sbom()` returns `bomFormat="CycloneDX"`, `specVersion="1.6"`, an RFC 4122 `urn:uuid:` serial number, and one `library` component per package.
- Run `python scripts/test_dependency_vulnerability_scanner.py`.

## Related Skills

- `dependency-pinning-and-reproducible-builds`
- `smart-contract-audit-requirements-before-defi-integration`
- `api-key-least-privilege-audit-tool`
- `immutable-infrastructure-for-trading-bots`