Explainable Machine Learning for Cross-Sectional Classification of Cognitive Impairment: A Leakage-Safe Study with Independent-Cohort External Validation across OASIS-1 and OASIS-2

Abstract

Machine learning models built on routinely collected clinical variables have been proposed as a low-cost alternative to neuroimaging for identifying cognitive impairment, but many published studies report optimistic performance because of preprocessing leakage, participant-level leakage, or validation within a single cohort. This study developed and externally validated a leakage-safe machine learning framework for classifying concurrent cognitive impairment, and quantified the contributions of the Mini-Mental State Examination (MMSE) and MRI-derived brain-volume measures. Models were developed on the OASIS-1 cohort (n = 235) and evaluated, without refitting, on the independent baseline OASIS-2 cohort (n = 150), with impairment defined as a Clinical Dementia Rating (CDR) above zero. All preprocessing was fitted only within cross-validated training pipelines, and five classifiers were tuned by grid search. A random forest using age, sex, education, socioeconomic status and the MMSE achieved an external area under the receiver operating characteristic curve (AUC) of 0.920 (95% CI 0.868–0.964) and a Brier score of 0.111. Removing the MMSE reduced the AUC to 0.749. Compared with the conventional MMSE < 24 threshold, which identified only 32% of impaired participants, the model achieved 80.0% sensitivity at 91.8% specificity. Adding brain-volume measures raised the external AUC to 0.963. Baseline clinical and imaging variables did not distinguish participants who later converted to impairment. Under leakage-safe development and independent-cohort validation, the framework is best suited as a calibrated case-finding aid rather than an early prediction tool.