Review Analysis_7

Explore a brief overview of this analysis, including performance metrics, evaluation results, and key characteristics of the model. Visualizations below provide insights at a glance.

Model Performance Summary

Train Set
Dataset
  • Population 2672
  • Outcome Count 466
  • Prevalence 0.174
Discrimination
  • AUROC
    0.772
    95% CI: 0.751-0.792
  • AUPRC 0.392
  • Avg Precision 0.393
Calibration
  • Brier 0.1257
  • Brier scaled 0.1329
  • Eavg 0.0230
  • Emax 0.2060
Test Set
Dataset
  • Population 668
  • Outcome Count 116
  • Prevalence 0.174
Discrimination
  • AUROC
    0.778
    95% CI: 0.740-0.816
  • AUPRC 0.350
  • Avg Precision 0.355
Calibration
  • Brier 0.1248
  • Brier scaled 0.0935
  • Eavg 0.0287
  • Emax 0.0427
Cross Validation
Dataset
  • Population 2672
  • Outcome Count 466
  • Prevalence 0.174
Discrimination
  • AUROC
    0.686
    95% CI: 0.663-0.708
  • AUPRC 0.249
  • Avg Precision 0.250
Calibration
  • Brier 0.1350
  • Brier scaled 0.0652
  • Eavg 0.0267
  • Emax 0.2489

Threshold Summary

Bias & Fairness Assessment Disclaimer

This section presents a demographic breakdown of the dataset used to train the model, together with visualisations of the class balance, error rate and maximum equalised odds difference. The goal is to highlight whether model performance varies across demographic groups and to surface potential source of bias.

Class Balance

Shows the distribution of positive and negative outcomes within each group. This provides context on data availability and outcome representation. Strong imbalance or small sample sizes can affect the reliability of performance and fairness metrics.

Error Rates

Shows model errors across demographic pairs (e.g. Male 18–25, Female 26–35) using two complementary measures:

  • False Positive Rate (FPR): cases where the model incorrectly predicts a positive outcome.
  • False Negative Rate (FNR): cases where the model incorrectly misses a positive outcome.

Lower values indicate better performance for a group, while large differences between groups may suggest uneven performance or potential fairness concerns.

Equalised Odds Difference

Summarises the maximum difference in FPR and FNR between groups. Values close to zero suggest more consistent model behaviour, while higher values indicate greater disparity in error patterns across demographic groups.

Age Group Class Balance

Gender Class Balance

Race Class Balance

Ethnicity Class Balance

Age Group Race Error Rate

Age Group Gender Error Rate

Age Group Ethnicity Error Rate

Gender Race Error Rate

Gender Ethnicity Error Rate

Race Ethnicity Error Rate

Max Eo Diff