Probabilistic forecasting of CISA KEV catalog entry — methodology, sampling, calibration, performance metrics, and representative case studies.
The KEV Predictor is a supervised model that estimates, for each active CVE, the probability of inclusion in CISA's Known Exploited Vulnerabilities (KEV) catalog within a 180-day window from the evaluation timestamp. Unlike EPSS — which forecasts short-term exploitation probability from public telemetry — the KEV Predictor estimates the probability of institutional validation of active exploitation by a regulatory body (CISA), incorporating vendor history, CWE-family patterns, attack-vector classification, and the timeline of publicly available exploits.
Live evaluation against the current base yields ROC AUC of 0.913, precision@100 of 1.000, and an Expected Calibration Error (ECE) of 0.0155. Discrimination between KEV-listed CVEs and the remainder is strong; calibration is adequate at the extremes, with systematic under-prediction in mid-probability bins — an expected artifact of comparing prospective forecasts against retrospective labels.
CISA maintains the KEV catalog as the official inventory of vulnerabilities under active exploitation, subject to mandatory remediation deadlines for U.S. federal agencies (BOD 22-01). Inclusion in the catalog concentrates global operational attention — defense teams, EDR vendors, and cyber insurers treat KEV entry as a top-priority signal.
Anticipating which CVEs are on a trajectory toward the KEV catalog enables defenders to prioritize mitigation before compulsory publication — eliminating the lag between availability of exploitation evidence and institutional validation. The indicator is strictly complementary to EPSS: while EPSS captures aggregate community signal (mentions, scans, public exploits), the KEV Predictor captures the compositional signal — which combinations of vendor, vulnerability class, and attack vector have historically converged toward CISA validation.
The model does not replace EPSS, CVSS, or KEV itself; it contributes as an early-warning layer to the Threat Priority Index (TPI), the composite score surfaced in the platform.
We frame the forecast as a supervised binary classification problem. For each CVE, we define the snapshot timestamp at 30 days after NVD publication (sufficient for the initial intelligence ecosystem — exploits, PoCs, vendor advisories — to partially consolidate). The label is positive if the CVE enters the KEV catalog within 180 days of the snapshot; negative otherwise.
Features are organized into four groups, all evaluated at snapshot time — never incorporating information posterior to the label:
The base classifier is a Histogram-based Gradient Boosting
(scikit-learn's HistGradientBoostingClassifier), selected for its
native robustness to categorical features, direct support for missing values, and
low computational cost on small datasets (~10⁴ samples).
class_weight='balanced' compensates for the inherent class imbalance
(positives represent roughly 5–8% of the base).
Over the base classifier we layer isotonic calibration
(IsotonicRegression), trained on a separate fold via prefit
cross-validation. Isotonic calibration is non-parametric and monotonic, preserving
the GBM's original ranking while redistributing probabilities to align with
observed frequencies. The final pipeline is therefore a two-stage model —
functionally equivalent to a depth-two stacked ensemble — where the first stage
produces raw scores and the second transforms them into calibrated probabilities.
Hyperparameters (learning_rate, max_iter, max_depth, min_samples_leaf, l2_regularization, max_leaf_nodes) are selected via random search over 30 configurations, scored by 5-fold stratified cross-validation with ROC AUC as the selection metric. The random seed is fixed for reproducibility.
The dataset is built from all active CVEs with NVD publication earlier than 180 days before the training date (ensuring the labeling window has fully elapsed). We adopt a temporal (non-random) split to prevent contamination by future signals — a common pitfall in threat-intelligence datasets, where vendor features depend on events that come after a randomly sampled timestamp.
| Set | N | Positives | Prevalence |
|---|---|---|---|
| Train (snapshots 2022 — 2025-Q4) | 1.375 | 115 | 8,4% |
| Test (snapshots 2026-Q1) | 1.051 | 82 | 7,8% |
| Daily inference (all active) | 2.945 | 804* | 27,3%* |
* Prevalence at daily inference is higher because it includes CVEs already in KEV — used for discrimination evaluation, not for training.
| Metric | Value | Interpretation |
|---|---|---|
| ROC AUC | 0,913 | Strong discrimination; values above 0.90 are considered excellent in vulnerability-risk literature. |
| PR AUC (training) | 0,328 | Expected for an 8%-prevalence problem — random baseline is 0.08. |
| ECE (Expected Calibration Error) | 0,0155 | High-quality calibration — values below 0.02 indicate low divergence between predicted probability and observed frequency. |
| Precision@10 | 1,000 | Top-10 candidates: 10/10 confirmed as KEV. |
| Precision@50 | 1,000 | 50/50 KEV |
| Precision@100 | 1,000 | 100/100 KEV |
| Precision@250 | 0,988 | 247/250 KEV |
| Precision@500 | 0,892 | 446/500 KEV |
Distribution of predicted probability versus observed frequency of KEV membership. Both extremes are well calibrated; the intermediate bins exhibit systematic under-prediction — discussed in section 5.
| Probability bin | N | Predicted (mean) | Observed (actual KEV) | Deviation |
|---|---|---|---|---|
| [0,90 — 1,00] | 9 | 0,92 | 1,00 | +0,08 |
| [0,80 — 0,90) | 43 | 0,83 | 1,00 | +0,17 |
| [0,70 — 0,80) | 76 | 0,75 | 1,00 | +0,25 |
| [0,60 — 0,70) | 82 | 0,65 | 0,98 | +0,33 |
| [0,50 — 0,60) | 79 | 0,55 | 0,96 | +0,41 |
| [0,40 — 0,50) | 80 | 0,45 | 0,86 | +0,41 |
| [0,30 — 0,40) | 113 | 0,35 | 0,70 | +0,35 |
| [0,20 — 0,30) | 233 | 0,25 | 0,58 | +0,33 |
| [0,10 — 0,20) | 275 | 0,14 | 0,38 | +0,24 |
| [0,00 — 0,10) | 1.955 | 0,03 | 0,07 | +0,04 |
The model is trained to answer the question: "what is the probability that this CVE enters the KEV catalog within 180 days of the snapshot?". The live evaluation reported above compares that forecast against a retrospective label ("is this CVE currently in KEV"), which includes entries that occurred years after publication. For a CVE whose KEV entry happened five years ago, the model — trained on recent snapshots — correctly assigns roughly 50% prospective probability; but compared to the historical label, the observed frequency is 100%. This apparent divergence does not indicate a calibration defect; it reflects that the evaluation metric covers a longer temporal horizon than the model's forecast window.
For vendors without observable history in the training set, the vendor_prior_kev_* features degenerate to zero. In those cases the model falls back on compositional signals (CWE, CVSS, attack vectors), which typically produces conservative estimates. Vulnerabilities in emerging ecosystems (software supply chain, obscure IoT devices) may be under-classified until vendor history consolidates.
The 180-day labeling window is deliberate — it balances positive density (very short windows yield too few labels) with operational utility (very long windows become retrospective rather than predictive). CVEs that enter KEV more than 180 days after the snapshot are, in the current design, labeled as negatives in training. Empirically this introduces a structural false-negative pattern for slow-adoption vulnerabilities — those CISA catalogs only after years of sporadic exploitation.
Five cases drawn from the inference base on 2026-05-27 illustrate the model's behavior under different regimes — within-window hit, directionally correct but late, false positive, and false negative.
Microsoft Windows 10 vendor with high historical KEV rate
A paradigmatic case: the model assigned 78.1% probability at snapshot (publication + 30 days), and CISA included the CVE in KEV just 14 days after publication — well within the window. The 4.3 CVSS is deliberately low; the high prediction derives primarily from the vendor_prior_kev_* component (Microsoft accumulates dozens of KEVs per year) and the Windows context (critical operating system). Demonstrates that the model extracts signal beyond CVSS — which EPSS typically does not capture for vulnerabilities with low technical scores but high political relevance.
Ivanti Endpoint Manager Mobile CVSS 9,8 public exploit at snapshot
Classic combination: critical CVSS + a vendor (Ivanti) with documented history of active exploitation + public exploit prior to snapshot. The model scored 63.6%, and KEV entry occurred 69 days after publication — well within the 180-day window. This profile corresponds to the core of the training set: most true positives share exactly this signature.
Citrix ShareFile CVSS 9,8
The CVE entered KEV 183 days after publication — only three days outside the 180-day label window. As a binary label, it counts as negative during training, yet the model still assigns 82.8% probability today. This illustrates what the hard window discards: directionally correct forecasts whose validation time is marginally past the cutoff. Operationally, it would surface as a high-priority watchlist item on day one, indicating the model is useful even for vulnerabilities that technically "miss" the window.
Atlassian Jira Server CVSS 5,3 path traversal
A Jira path-traversal that circulated sporadically for more than three years before CISA validated active exploitation — KEV entry 1,184 days after publication. The model scores 73.8% today, aligned with the composition (Atlassian + CWE-22 path traversal + presence of public PoC). This profile defines the model's structural blind spot: vulnerabilities whose exploitation matures over years. The model identifies them correctly in compositional terms, but the 180-day operational window does not capture the dynamic. A structural fix would extend the window to 365–730 days, at the cost of less current labels and slower training cycles.
SnakeYAML (Java) CVSS 9,8 deserialization (CWE-502)
Deserialization flaw in a widely used Java library (SnakeYAML), with public PoCs and multiple references in downstream vendor advisories (Spring, Pivotal, Apache). The model scores 67% today — the composition "critical library + CWE-502 + CVSS 9.8 + high community visibility" usually converges to KEV. In 1,273 days, this has not happened. Hypotheses: (i) exploitation is typically indirect, via applications that consume the library, and CISA catalogs the affected application rather than the library CVE; (ii) wide adoption of safe loaders (SafeConstructor) reduced operational exploit viability. An instructive case of model limitation: compositional features correlate with KEV but are not causal.
Microsoft Exchange Server CVSS 8,8
The mirror image of the previous case: the model assigns only 2.1% probability today, yet CISA included the CVE in KEV on 2026-04-13 — 1,154 days after publication. Microsoft Exchange has a dense KEV history, so why did the model err? Three factors converge: (i) at the snapshot (2023-03-16) there was no documented public PoC for this specific CVE — Exchange was the target of several other concurrent CVEs that absorbed analytical attention; (ii) the initial NVD description undercharacterized the severity (described as a secondary elevation of privilege, later reclassified); (iii) KEV entry in 2026 was the result of an emergent exploitation campaign three years after publication, a profile the model — trained on recent snapshots — does not cover. Exemplifies the limitation discussed in section 5.3.
The KEV Predictor delivers, on the current active base, strong discrimination (ROC AUC 0.913) and adequate calibration at the extremes (ECE 0.0155). Top-K precision is particularly noteworthy: the 100 CVEs with highest probability are, in their entirety, confirmed by the KEV catalog. For operational use — daily prioritization, defense alerts, TPI integration — the ranking is reliable and the watchlist of candidates is actionable.
Limitations are structural, not defects: (i) the 180-day window excludes historically late KEV entries; (ii) cold start for new vendors produces conservative estimates; (iii) compositional features capture correlation rather than causation — libraries with critical signatures may never be individually cataloged if CISA chooses to index the consuming application instead. Future iterations should evaluate an extended window (365 days), incorporation of exploitation traffic signals (honeypots, CIRCL sightings), and product-family-specific fine-tuning.
As with any probabilistic indicator, the KEV Predictor is decision input — not final decision. We recommend composition with EPSS, CVSS, SSVC, and the consolidated TPI for operational judgment.
