Decision outcomes by group
Selection, true-positive and false-positive rates at the chosen threshold
SelectionTPRFPR
Project 65 · Model Governance
Command Center
A reproducible, diagnostic audit of prediction performance and outcome differences across groups.
Synthetic examples. Not suitable for real eligibility decisions.
Equity signal review
Absolute differences between groups. Smaller is not always better if predictive utility deteriorates.
Group performance summary
Thresholded confusion-matrix metrics calculated from the underlying records
Threshold Studio
Accuracy, selection gap and opportunity gap
Sweep 19 shared decision thresholds, from 0.05 to 0.95
A shared threshold is applied to both groups. Curves describe trade-offs, not fairness guarantees.
Choose a threshold
Explore audit criteria. This is a diagnostic tool, not an automated decision policy.
Why thresholds matter
Raising the cutoff can lower false positives but increase missed positive outcomes. A threshold that improves one equity metric may worsen another.
Threshold comparison table
Metrics for all candidate cutoffs, computed on the same records
Group Diagnostics
Confusion matrices
TP true positive, FP false positive, TN true negative, FN false negative
Calibration by group
Observed event rate and mean predicted score within score bands
A calibrated score can still produce group differences at a common decision threshold. Calibration is assessed descriptively.
Predictive diagnostics
Utility & Trade-offs
Error cost assumptions
Assign illustrative costs to false positive and false negative decisions.
Competing audit objectives
Lowest-cost candidate under the selected cost assumptions versus other objectives
Costs are invented for education. Do not use them to adjudicate employment, admission, healthcare or other consequential decisions.
Policy comparison
Robustness Lab
Stratified bootstrap uncertainty
Resample observations within each group, preserving sample sizes. 300 replicates with a fixed seed.
Intervals quantify sampling variability under the resampling assumptions. They do not correct selection bias, label quality, or measurement error.
Audit limitations & warnings
Bootstrap interval table
Data & Methodology
Import your own audit records
Load a CSV with one row per observation. All processing occurs locally in your browser.
Maximum 2 MB / 2,500 records. Requires exactly two group categories and both outcome labels.
Required columns
id,group,score,outcome,segment
- score: number from 0 to 1
- outcome: 0 or 1; labels are observational
- group: exactly two comparison categories
- segment: optional context descriptor
Export reproducible artifacts
Each download reflects the current imported data, threshold and audit parameters.
Definitions, safeguards and reproducibility
Classification metrics
Accuracy = (TP + TN)/N. TPR = TP/(TP + FN). FPR = FP/(FP + TN). Precision = TP/(TP + FP). Selection rate = (TP + FP)/N.
Group-gap metricsDemographic parity gap = absolute selection-rate difference. Equal opportunity gap = absolute TPR difference. Equalized odds gap = maximum of absolute TPR and FPR differences.
Score quality
Brier score = average squared difference between predicted score and binary outcome. Calibration bins compare observed outcome prevalence with average predicted score.
Governance boundaryDemonstration data are synthetic. Group-disparity metrics do not certify legal fairness, absence of discrimination or causal effects. Audit findings require domain review and may be inappropriate to use for individual decisions.