1 INTRODUCTION

The Montreal Cognitive Assessment (MoCA) was developed as a brief screening test for mild cognitive impairment (MCI). It is widely used across the world in a variety of settings. The MoCA is recommended by the Alzheimer Society to objectively assess cognitive complaints in a clinical setting.

Even though more and more advocacy groups or policy makers favor screening for dementia, there is still a debate if screening in various populations is wise. However, the setting of old age psychiatry is different to our opinion. By knowing patient's cognitive functioning at referral, besides timely detecting dementia also to monitor all causes of MCI in old age psychiatry, one can adapt their (psychiatric) treatment; eg, pharmacotherapy (including compliance) or psychotherapy, especially as this population is at greater risk of changing cognitive functioning not only by age but also by (psychotropic) medication or because of the referral reasons. In The Netherlands, referrals to old age psychiatry consist of a mix of neurodegenerative and other psychiatric disorders, such as depression, bipolar disorders, schizophrenia, and severe anxiety disorders, all of which can be accompanied by poor cognitive functioning.

We introduced in our clinic a short cognitive assessment using the MoCA for all referred patients to lower doctors delay by adding an objective aid to triage those in need for specialized diagnostic route besides having baseline cognitive data. Therefore, we need to know its diagnostic test accuracy in this setting.

The MoCA shows good validity in multiple languages, although moderately so in Dutch in a geriatric memory clinic setting. It is important to validate the MoCA in specific settings, as the selection of subjects with different characteristics may influence the test characteristics of a scale such as the MoCA. This is especially relevant in case-control study designs using community-based healthy controls (HC), as this is not representative of the clinical reality. The MoCA has not yet been validated in old age psychiatry settings, where patients are referred with multidimensional causes for MCI and to our knowledge our study is the first to do so. Differentiation between cognitive impairment as a consequence of a psychiatric disease and/or as a consequence of early stage dementia is complicated and may affect the test-characteristics of the MoCA.

According to the Cochrane review, “the MoCA may help identify people requiring specialist assessment and treatment for dementia.”

We aim to validate the MoCA in this clinical setting following the standards for reporting diagnostic accuracy (STARD 2015) recommendations by using a cross-sectional study design. The purpose of the present study is to test the criterion validity (ie, can the MoCA predict a diagnose correctly) of the MoCA to detect MCI and early stage/mild dementia (MD) in an old age psychiatry cohort including referred but not cognitive impaired patients as primary comparisons. The reference standard consists of a multidisciplinary, consensus-based diagnosis in accordance with international criteria. The above cross-sectional design avoids the spectrum-bias of most case-control studies where the extremes of the spectrum of cognitive function were included. To illustrate this effect, we present as a secondary outcome the MoCA results in a case-control design, using community-based HC with normal cognitive aging as secondary comparisons.

2 METHOD

2.1 Sample

This study was performed in an old age (60 years +) psychiatry outpatient clinic in a large Dutch City (Utrecht), which offers services to the north-west side of the city and its rural surroundings (57.000 inhabitants of 60+ in the north-west). Between 2008 and 2018, all newly referred patients were eligible for this study. The inclusion criterion was the ability to give written informed consent. Therefore, patients referred with severe dementia (Global Deterioration Scale [GDS] greater than or equal to 6), Behavioral and Psychological Symptoms of Dementia (BPSD), or compulsory referrals were not included.

Participants were assessed by a multidisciplinary team, on all occasions including an old age psychiatrist and a trained psychiatric nurse practitioner. After referral, patients with an obvious cause of their cognitive complaints were excluded to resemble a clinical screening population: Those with a diagnosis of severe mid-stage dementia (GDS greater than or equal to 5), a recent history of substance abuse (<2 years), recent delirium (<6 months), or an acquired brain injury including cerebrovascular accident (CVA) or transient ischemic attack (TIA). In addition, patients with insufficient command of the Dutch language were excluded.

The secondary study compares the test properties of the MoCA with an unrealistic situation: a group of community-based HC, age 60+. They were recruited from acquaintances of patients or research assistants. Inclusion criteria were no cognitive complaints and no risk factors for cognitive dysfunction. Exclusion criteria were acquired brain injury including CVA or TIA, substance abuse, recent delirium, recent treatment for psychiatric or neurologic diseases, and use of medication that can alter cognitive functioning. From potential HC showing signs of cognitive impairment during the interview or with a MoCA score below 25, consent was obtained to interview the next of kin (n = 11), who were assessed with the Modified Informant Questionnaire on Cognitive Decline in the Elderly (IQCode). No potential HC had an IQCode higher than 3.5, which would indicate potential moderate cognitive impairment and would be an exclusion criterion.

The Committee for Research and Ethics of the institution approved this study (CWO-nr 1606). All participants gave their informed consent. Data are available on request due to privacy/ethical restrictions.

2.2 Measurements

Initial assessment was performed by an old age psychiatrist, including a medical history obtained from the next of kin and relevant laboratory tests for cognitive impairment. During the diagnostic procedure, the 15-item Geriatric Depression Scale (GDS15) and the Global Assessment of Functioning (GAF) were collected. Investigation of instrumental activities of daily living (IADL) was done by a psychiatric nurse practitioner on a home visit. When this initial assessment raised any suspicion of cognitive impairment, further assessment took place with a neuropsychological assessment (n = 289) and, when applicable, CT/MRI imaging and cerebrospinal fluid (CSF) analysis. The neuropsychological assessment, done by a neuropsychologist not aware of the MoCA score, was an extensive and comprehensive assessment including multiple tests in the domains of memory, attention, executive function, fluid intelligence, and language capacities. Full test of: Dutch reading test for adults to estimate premorbid intelligence (“Nederlandse Leestest voor Volwassenen” NLV); Proverbs; Zung 12; Self-rating Depression scale (ZDS); Raven Coloured Progressive Matrices; Questionnaire for orientation and personal and non-personal episodic memories (“Toutenburger Vragenlijst”); Visual Association Test(VAT); Fifteen words imprinting and recall or recognition; Copying of Drawings (Meander of Luria, Complex figure of Rey, House, Cube, Greek cross; D-KEFS|Trail Making Test A and B (TMT); Hooper Visual Organization test (VOT-short version); Calculation; spelling and reading; Binet- Bobertag story; Fluency- test category (and letter); Groninger Intelligence test (GIT); Clock reading and writing. Subtest of: Wechsler Adult intelligence scale; WAIS IV (Symbol substitution, Numerical series/Digit Span, Agreements/Similarities, Figures; Figure Weights). Wechsler Memory scale IV; WMS IV (numerical series). Behavioral Assessment of the Dysexecutive Syndrome (BADS; Keysearch test and Zoo-plan test).

The HC were interviewed and assessed by research assistants. The assessment was carried out in a single day and included the MoCA, the GDS15, and GAF.

2.3 Diagnostic test

All referred participants were assessed with a MoCA as soon as possible, within a maximum of 3 months from referral, by a trained research assistant or psychiatric nurse practitioner. This was independent from the diagnostic procedure. The MoCA was assessed during the feedback appointment of the initial assessment when the treatment plan was presented. The treatment plan included referral to our memory clinic for further assessment if there was doubt or suspicion of CI.

The MoCA consists of one page, covering the cognitive domains of executive function and visuospatial abilities, naming, short-term memory, attention and working memory, language, concentration, verbal abstraction, and orientation. It can be carried out within 10 minutes, with a maximum score of 30 indicating no errors were made. Scores were corrected for low education according to instructions, by adding one point to the total score of patients with 12 years of education or less. The original suggested cutoff for the diagnosis of CI was a score of (below) 26 (less than 26).

2.4 Reference test

The reference test was the diagnosis determined at multidisciplinary team meetings, including an old age psychiatrist, neuropsychologist, and geriatrician.

The diagnoses of dementia and MCI were supported by a minimum of a neuropsychological assessment and laboratory tests. The diagnoses were made in consensus and in accordance with the MCI criteria as proposed by an international consortium or the Dutch guideline on dementia. This guideline covers the criteria of DSM IV for dementia, NIA-AA/NINCDS-ADRDA for Alzheimer disease, NINDS-AIREN/AHA-ASA for vascular dementia, frontotemporal dementia (FTD) according to The Lund and Manchester Groups and the Consensus for Dementia with Lewy Body. The MCI group included those with MCI due to psychiatric causes, in accordance with the international consensus. No further differentiation of MCI was made in this study. The results of the MoCA were not used to diagnose MCI or dementia.

Referred patients without suspicion of CI during initial assessment were followed up for a minimum of 2 years to compensate for not having a neuropsychological assessment. Patients who did not meet the aforementioned criteria for a diagnosis of dementia or MCI during follow-up were classified as no cognitive impairment (NoCI). Patients who did meet the aforementioned criteria after the initial 3 months during follow-up were classified as inconclusive to be cautious (n = 3).

2.5 Statistical analyses

Results were compared within the referred patients with MD, MCI, or NoCI and between the groups total referred patients (MD + MCI + NoCI) and HC, using the Statistical Package for the Social Sciences (SPSS, version 22; SPSS Inc., Chicago, IL); chi squared test to compare sex and education. ANOVA to compare age, GAF, GDS15, and MoCA scores followed with a least significant difference (LSD) (and a Bonferroni not shown) post hoc test. An ANCOVA with age as a covariate was run additionally.

Using receiver operating characteristic (ROC) analysis, the area under the curve (AUC) was calculated as a measure for the diagnostic accuracy of the MoCA. As the MoCA can be used to detect dementia in a clinical setting as well as to rule out cognitive impairment in a clinical setting, we calculated different ROC curves: (a) to detect dementia in a clinical setting, (b) to detect cognitive impairment (MD + MCI) in a clinical setting, and (c) to detect MCI in in a subgroup of patients (MD excluded). To compare these analyses with previous case-control studies and to see the effect of bias, all analyses were repeated with HC.

PPV and NPV were calculated for the “optimal” cutoff scores as calculated by the Youden J index. Cronbach alpha was calculated for internal consistency of the MoCA.

3 RESULTS

3.1 Study groups

Out of 2204 referrals, 1337 were not eligible for this study. Eight hundred sixty-seven referred patients were assessed with a MoCA for this study (mean delay 21.5 days, 65% within 3 weeks of referral). After applying the exclusion criteria (Figure 1), a group of 710 participants remained: 81 MD, 153 MCI, 459 referred patients with no MCI or dementia (NoCI), and 17 inconclusive. Mean time needed for diagnosing was 40.5 days for the NoCI and 60.8 for the CI. For the secondary outcome, 84 HC were included of a group of 96 potential healthy volunteers (Figure 1). Two of them had an IQcode between 3.25 and 3.5 indicating minor decline over the past 10 years. All others were in between 3.0 and 3.25 indicating (almost) no decline.

Figure 1

_{Flowchart referred patients and healthy controls. MCI: mild cognitive impairment; NoCI: no cognitive impairment; HC: healthy controls; GDS: Global Deterioration Scale; IQCode: Informant Questionnaire on Cognitive Decline in the Elderly; BPSD: Behavioral and Psychological Symptoms of Dementia}

3.2 Demographic findings

Within the referred patients, there was a significant difference in age (ANOVA F = 26.0 P = .000) between the diagnostic groups, as expected. There was no significant difference between sex (P = .39) and education length (P = .142). Disability, as measured by the GAF, showed an expected difference: MCI best and the demented and NoCI (as most of them were psychiatrically ill) the worst GAF score (P = .001). The GDS15 shows no significant differences between the referred groups.

As for the secondary outcome, there were no significant differences in age, education, and sex between the population of referred patients and the HC (Table 1). The significant differences in GDS15 and GAF were as expected; the HC had significantly fewer depressive symptoms (GDS15-score) and better global functioning (GAF score).

Table 1. Key demographic and clinical characteristics

_{Note. Education and sex were compared between b, c, and d and between a and e with a chi‐squared test. Groups b, c, and d, were compared with ANOVA.Groups a and e were compared with t test.}

_{Abbreviations: GAF, Global Assessment of Functioning; GDS15, Geriatric Depression Scale 15 question version; MCI, mild cognitive impairment; MoCA,Montreal Cognitive Assessment; NoCI, no cognitive impairment.}

3.3 MoCA outcome

The mean MoCA scores differed significantly between groups; the differences in average MoCA scores between the individual referred groups were significant (P = .000), as well as those for the secondary outcome between combined total referred group and the HC (P = .000). The standard deviations (MCI towards NoCI) and range (all groups) of the referred groups did overlap and showed a wide distribution (Table 1). The internal consistency of the MoCA, as expressed by the Cronbach alpha on the standardized items (.761), was good. All 12 items of the MoCA contribute to a positive Cronbach alpha, as no item “if item deleted” gives a higher outcome (.708-.737).

The results of the ROC analysis, for clinical situations, are shown in Figure 2A,B. Table 2 displays the AUCs of these and additional analyses, as well as their sensitivity and specificity at the literature-recommended cutoff scores of 26 and 21. All AUCs were significantly different from 0.5 (no diagnostic accuracy), P < .001. The AUCs with HC as secondary comparison ranged between 0.90 and 0.98, an excellent accuracy. The MoCA performed less well in a clinical setting, with AUCs between 0.70 and 0.87.

Figure 2

_{Results of receiver operating characteristic (ROC) analysis. A, dementia versus no dementia (mild cognitive impairment [MCI] + no cognitive impairment [NoCI]); B, cognitive impairment (CI = mild dementia [MD] + MCI) versus NoCI [Colour figure can be viewed at wileyonlinelibrary.com]}

Table 2. The effect of using HC instead of NoCI as comparisons on area under the curve between variations of groups and their sensitivity and specificity at cutoff scores 26 and 21, often used in literature.

_{Note. Dem: dementia (n = 81); NoDem: no dementia (MCI + NoCI; n = 612); MCI: mild cognitive impairment (n = 153); NoCI: referred patients no cognitive impairment (n = 459); HC: healthy controls (n = 84); CI: cognitive impairment (Dem + MCI; n = 234). AUC: area under the curve. SE: standard error. Sens: sensitivity. Spec: specificity.}

For the original suggested cutoff score of 26 to discriminate MCI from HC, the sensitivity and specificity are 94% and 73%, respectively (in the original article 90% and 87%). Using the same cutoff score in a realistic setting (ie, discriminating against referred NoCI) leads to a drop in specificity to 37%. The clinical situation of detecting CI (MD + MCI) below this cutoff had a sensitivity of 95%.

A cutoff score for diagnosing dementia is still under debate but is often set around 21, which in our study results in a sensitivity of 90%. The specificity dropped from 99% using dementia vs HC to 74% in a clinical setting (dementia vs MCI + NoCI), and 63% for dementia vs MCI. To find the “best” cutoff score for our population, the specificity and sensitivity were calculated for different scores of the MoCA (Table 3).

Table 3. Sensitivity and Specificity at MoCA scores from 28 to 18

_{Note. Dem: dementia (n = 81); MCI: mild cognitive impairment (n = 153); NoCI: referred patients no cognitive impairment (n = 459); HC: healthy controls (n = 84); CI: cognitive impairment (Dem + MCI; n = 234).}

_{a(MoCA-D below score).}

The “optimum” cutoff scores against NoCI as calculated by the Younden index were less than 25 for detecting MCI, sensitivity 88% (95% CI, 81-92), specificity 49% (95% CI, 44-53); less than 23 for CI, sensitivity 75% (95% CI, 69-81), specificity 68% (95% CI, 63-72); and less than 21 for MD, sensitivity 90% (95% CI, 81-95), specificity 78% (95% CI, 74-81) and comparable with literature.

The PPV and NPV were calculated (Table 4) for the two scores with the highest computed Younden index. The PPV and the NPV show different results. The PPV was low in almost all situations whereas the NPV was high in all situations. Using a cutoff of less than 21 for dementia results in 31% of a positive MoCA having MD and 98% of a negative test having no MD. For detecting MCI at a cutoff of less than 26; 33% has indeed MCI when the MoCA is positive and 94% above this threshold will not have MCI.

Table 4. Positive and negative predictive values of cutoff scores with the highest Younden index

_{Note. Dem: dementia (n = 81); MCI: mild cognitive impairment (n = 153); NoCI: referred patients no cognitive impairment (n = 459); HC: healthy controls (n = 84); CI: cognitive impairment (Dem + MCI; n = 234). PPV: positive predictive value; NPV: negative predictive value; 95% CI, 95% confidence intervals.}

_{a MoCA-D below score.}

4 DISCUSSION

In this cross-sectional study, patients with dementia were significant older than those without. There were more females in each group, which is representative of the population referred to old age psychiatry. Age has been shown to be of influence, as MoCA scores decline with aging and can alter the (interpretation of) results. However, age has little unique variance and a correlation of less than 10%. An additional ANCOVA sensitivity analysis with age as a covariate still showed significant differences in MoCA scores between the different diagnostic groups in our study.

The GDS15, a geriatric depression scale, revealed no differences between the referred groups. This finding underscores again the necessity to be cautious when using a screening tool like the GDS15 in attempting to differentiate between or detect psychiatric causes of cognitive complaints.

Our study reproduced the significantly different mean MoCA scores reported in previous literature. Our secondary outcome, differentiating patients with MD or MCI from HC, shows comparable properties reported in previous case-control studies. But to avoid this spectrum bias, we studied the MoCA in a cohort of patients referred to old age psychiatry, which more accurately represents the clinical reality. This is illustrated in Table 3, where the AUC and specificity drop when the comparison is realistic (NoCI as comparisons) and not fictive (HC as comparisons). One can argue that this bias we underscore, by adding HC, is well-known and its effect on the AUC shown before. Apparently, it is still important to stress out the effect it has on optimum cutoff scores as the case-control study design is still the majority of the MoCA validation studies. Clinicians should be careful to use cutoffs based on those studies. Twenty-seven percent of the HC had a MoCA score below 26, compared with 63% of the referred NoCI. The MoCA scores of our NoCI patients match with that of a longitudinal, population based study (n = 2653; mean MoCA 23.36, 64% specificity less than 26) indicating we have a realistic comparison group. Even though there was a wide range of MoCA scores in our group, this occurred in a clinical setting and can be explained by the following.

False negative results were found in cases of high educational and/or professional levels or FTD in the dementia group. False positive results occurred due to a lack of motivation and/or attention in depressed, manic, or psychotic patients, with or without MCI. One may argue the latter should have been diagnosed with MCI due to their psychiatric conditions. However, it was the clinical opinion of the team, after IADL investigation, that their presentation was not persistent and did not justify a diagnosis of MCI, as the MoCA score was not taken into account.

There is a risk, including in this study, of a subjective decision whether MCI is diagnosed or not when a psychiatric disorder explains its etiology, despite the criteria for MCI being met. We minimized this by including a neuropsychological assessment during the diagnostic work-up when there was suspicion of persistent impaired cognitive functioning. In the future, the MoCA would make it easier and more objective to select these possible MCIs and identify those in need of a further work-up.

False positives (ie, a low MoCA score) due to unrecognized neurodegenerative MCI can be excluded in our study, as progression to any DSM IV diagnosis of cognitive impairment was monitored with a mean follow-up of 3.5 years.

This study shows it is safe to use a threshold of greater than or equal to 26 to indicate normal cognition (95% sensitivity for CI), taking specific situations, like a university degree or FTD, into account.

While the MoCA detects most MD (less than 21; 90% sensitivity) and MCI (less than 26; 94% sensitivity) below these cutoff scores, making it fit for screening, it is not suitable for diagnosing MD or MCI in our study population, as the PPV for MD and MCI are still only fair (31% and 33% PPV, respectively). The proportion of referred psychiatric patients scoring below these cutoff scores is too high for diagnostic purposes (22% and 63% of NoCI, respectively).

The MoCA is suitable for excluding dementia (greater than or equal to 21; NPV 92%-98%) and MCI (greater than or equal to 26; NPV 94%), if used to assess patients referred to an old age psychiatry setting. This, combined with the high sensitivity at these cutoffs, makes the MoCA a useful screening tool.

In the case of a positive test result, further work-up is usually necessary; the absolute amount of false-positives is substantial, since the majority of referred patients do not suffer from MD.

Using our study cohort as an example, applying a MoCA cutoff of less than 21 to screen 100 referred patients would lead to 33 patients receiving specialized diagnostic tests, of whom 14.7 would be NoCI, 8.2 MCI, and 10.5 correctly identified MD. One patient (1.15) with MD would not be detected using this cutoff score. This confirms that screening comes with its price, also in old age psychiatry.

We recommend further research to find methods that increase the specificity and improve selection of those in need of a specialized diagnostic pathway. The aforementioned weaknesses of our study—unrealistic scattering and seemingly missed CI diagnoses—would in practice be interpreted as part of a larger clinical picture; incongruous results would be reconsidered if these MoCAs are clinically relevant or correct, or considered as CI. This would increase the specificity of the MoCA. Further research should focus on the suspected CI referrals only and investigate if a MoCA reassessment after recovery from serious psychiatric episodes can lower the false positive rate. Another limitation is that we did not gave all the comparisons the same full diagnostic assessment due to practicality and resource constraints. Because adding the HC was mainly to underscore the spectrum-bias effect, this is to our opinion acceptable.

The NoCIs that were not suspected of CI, hence did not got a full diagnostic work-up, were followed for at least 2 years to compensate for this limitation. The NoCIs that were suspected of CI did get the same full diagnostic assessment. Excluding the GDS greater than or equal to 5 and BPSD could be seen as selection bias and a limitation. To our opinion, avoiding the extremes of the spectrum is a strength of our study. The clinical reality is that the obvious demented will not be screened whether they need a specialized diagnostic route. But including their low MoCA scores in the study would bias the results.

5 CONCLUSION

This study shows that validating the MoCA in a biased setting, ie, against HC, overestimates specificity. Our findings are in line with the literature, where lower cutoff scores are repeatedly suggested to tackle this problem.

Taking the above results into account, one can conclude that the MoCA can be useful in an old age psychiatric setting to confirm normal cognitive functioning and to identify those who are in need for a specialized diagnostic pathway. However, further research is necessary to minimize the number of false positives in the latter group.

INTRODUCTION

The Montreal Cognitive Assessment (MoCA) was developed as a brief screening test for mild cognitive impairment (MCI) and is used widely. The Alzheimer Society recommends the MoCA for objectively assessing cognitive concerns in clinical settings.

While the widespread screening for dementia remains a topic of debate, its application within old age psychiatry differs. Understanding a patient's cognitive function at referral not only helps detect dementia early but also allows for monitoring of all causes of MCI in this population. This knowledge can then guide adjustments to psychiatric treatment, including medication and therapy, especially since this group faces increased risks of cognitive changes due to age, psychotropic medications, or their referral conditions. In the Netherlands, referrals to old age psychiatry often involve a mix of neurodegenerative disorders and other psychiatric conditions like depression, bipolar disorders, schizophrenia, and severe anxiety disorders, all of which can be associated with cognitive impairment.

A brief cognitive assessment using the MoCA was introduced in the clinic to reduce delays in doctor assessment by providing an objective tool for triaging patients who require a specialized diagnostic pathway, in addition to collecting baseline cognitive data. Therefore, understanding the diagnostic accuracy of the MoCA in this specific clinical setting is essential.

The MoCA has demonstrated good validity across multiple languages, though its validity is only moderate in Dutch within a geriatric memory clinic. Validating the MoCA in specific settings is crucial because the characteristics of the study participants can influence the test's properties. This is particularly relevant for case-control studies that use healthy community controls, as this does not accurately represent the clinical patient population. Before this study, the MoCA had not been validated in old age psychiatry settings, where patients are referred with diverse causes of MCI. Distinguishing cognitive impairment resulting from a psychiatric illness from that caused by early-stage dementia is complex and can affect the MoCA's test characteristics.

According to a Cochrane review, the MoCA may assist in identifying individuals who require specialist assessment and treatment for dementia. This study aimed to validate the MoCA in this clinical environment by adhering to the STARD 2015 recommendations for reporting diagnostic accuracy, utilizing a cross-sectional study design. The primary objective was to assess the criterion validity of the MoCA in detecting MCI and early-stage/mild dementia (MD) within an old age psychiatry cohort, comparing these groups with referred patients who had no cognitive impairment. The reference standard for diagnosis involved a multidisciplinary, consensus-based assessment aligned with international criteria. This cross-sectional design helps mitigate spectrum bias, which is common in case-control studies that typically include individuals at the extremes of cognitive function. To further illustrate this bias, a secondary outcome involved presenting MoCA results in a case-control design, using healthy community controls with normal cognitive aging as comparisons.

METHOD

Sample

This study was conducted at an old age (60 years and older) psychiatry outpatient clinic in a large Dutch city (Utrecht), which serves a population of approximately 57,000 individuals aged 60 and above in its service area. Between 2008 and 2018, all newly referred patients were eligible for participation. The primary inclusion criterion was the ability to provide written informed consent. Patients referred with severe dementia (Global Deterioration Scale [GDS] score of 6 or higher), Behavioral and Psychological Symptoms of Dementia (BPSD), or those referred under compulsory orders were excluded.

A multidisciplinary team, consistently including an old age psychiatrist and a trained psychiatric nurse practitioner, assessed participants. Following referral, patients with clear causes for their cognitive complaints were excluded to ensure the study population resembled a clinical screening group. These exclusions included individuals with a diagnosis of severe mid-stage dementia (GDS score of 5 or higher), a recent history of substance use (within the past 2 years), recent delirium (within the past 6 months), or an acquired brain injury, such as a cerebrovascular accident (CVA) or transient ischemic attack (TIA). Additionally, patients with insufficient command of the Dutch language were excluded.

The secondary part of the study involved comparing the MoCA's test properties with a less realistic scenario: a group of healthy community controls (HC) aged 60 and older. These individuals were recruited through acquaintances of patients or research assistants. Inclusion criteria for HC included the absence of cognitive complaints and no risk factors for cognitive dysfunction. Exclusion criteria for HC were acquired brain injury (including CVA or TIA), substance use, recent delirium, recent treatment for psychiatric or neurologic diseases, and the use of medication that could alter cognitive functioning. For potential HC who showed signs of cognitive impairment during an interview or had a MoCA score below 25, consent was obtained to interview their next of kin (n = 11). These individuals were assessed using the Modified Informant Questionnaire on Cognitive Decline in the Elderly (IQCode). No potential HC had an IQCode higher than 3.5, which would have indicated potential moderate cognitive impairment and led to exclusion.

The Committee for Research and Ethics of the institution approved this study (CWO-nr 1606). All participants provided their informed consent. Data are available upon request due to privacy and ethical restrictions.

Measurements

An old age psychiatrist conducted the initial assessment, which included gathering medical history from the next of kin and performing relevant laboratory tests for cognitive impairment. During the diagnostic process, scores from the 15-item Geriatric Depression Scale (GDS15) and the Global Assessment of Functioning (GAF) were collected. A psychiatric nurse practitioner conducted investigations of instrumental activities of daily living (IADL) during a home visit. If this initial assessment raised any suspicion of cognitive impairment, further assessment was performed, including a neuropsychological assessment (n = 289) and, when appropriate, CT/MRI imaging and cerebrospinal fluid (CSF) analysis. A neuropsychologist, who was unaware of the MoCA score, conducted the neuropsychological assessment. This was an extensive and comprehensive evaluation covering multiple tests in the domains of memory, attention, executive function, fluid intelligence, and language capacities. The assessment included various standardized tests such as the Dutch reading test for adults (NLV), Proverbs, Zung 12, Self-rating Depression Scale (ZDS), Raven Coloured Progressive Matrices, Toutenburger Vragenlijst, Visual Association Test (VAT), Fifteen Words Imprinting and Recall or Recognition, Copying of Drawings (Meander of Luria, Complex figure of Rey, House, Cube, Greek cross), D-KEFS|Trail Making Test A and B (TMT), Hooper Visual Organization test (VOT-short version), Calculation, Spelling and Reading, Binet-Bobertag story, Fluency-test category (and letter), Groninger Intelligence test (GIT), Clock reading and writing. Subtests from the Wechsler Adult Intelligence Scale (WAIS IV) and Wechsler Memory Scale IV (WMS IV) were also used, along with the Behavioral Assessment of the Dysexecutive Syndrome (BADS; Keysearch test and Zoo-plan test).

Research assistants interviewed and assessed the healthy controls. Their assessment was completed in a single day and included the MoCA, GDS15, and GAF.

Diagnostic test

All referred participants underwent a MoCA assessment as soon as feasible, within a maximum of three months from referral, administered by a trained research assistant or psychiatric nurse practitioner. This assessment was independent of the formal diagnostic procedure. The MoCA was typically administered during the feedback appointment of the initial assessment, when the treatment plan was presented. The treatment plan included referral to a memory clinic for further assessment if there was any doubt or suspicion of cognitive impairment.

The MoCA is a single-page test that evaluates various cognitive domains, including executive function, visuospatial abilities, naming, short-term memory, attention, working memory, language, concentration, verbal abstraction, and orientation. It can be completed within 10 minutes, with a maximum score of 30 indicating no errors. Scores were adjusted for lower education levels by adding one point to the total score for patients with 12 years of education or less, according to test instructions. The original suggested cutoff for diagnosing cognitive impairment was a score below 26.

Reference test

The reference test for diagnosis was determined at multidisciplinary team meetings, which included an old age psychiatrist, a neuropsychologist, and a geriatrician.

Diagnoses of dementia and MCI were supported by at least a neuropsychological assessment and relevant laboratory tests. These diagnoses were made by consensus and adhered to the MCI criteria proposed by an international consortium or the Dutch guideline on dementia. This guideline encompasses criteria from DSM IV for dementia, NIA-AA/NINCDS-ADRDA for Alzheimer's disease, NINDS-AIREN/AHA-ASA for vascular dementia, criteria from The Lund and Manchester Groups for frontotemporal dementia (FTD), and the Consensus for Dementia with Lewy Body. The MCI group included individuals with MCI due to psychiatric causes, in line with international consensus. No further differentiation of MCI subtypes was made in this study. The results of the MoCA were not used in making the diagnoses of MCI or dementia.

Referred patients who did not present with suspicion of cognitive impairment during the initial assessment were followed for a minimum of two years to compensate for the absence of a full neuropsychological assessment. Patients who did not meet the aforementioned criteria for a diagnosis of dementia or MCI during this follow-up period were classified as having no cognitive impairment (NoCI). A small number of patients (n = 3) who met the diagnostic criteria after the initial three months, during follow-up, were cautiously classified as inconclusive.

Statistical analyses

Results were compared within the referred patient groups (MD, MCI, or NoCI) and between the combined total referred patients (MD + MCI + NoCI) and healthy controls (HC). Statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS, version 22; SPSS Inc., Chicago, IL). Chi-squared tests were used to compare sex and education levels. Analysis of Variance (ANOVA) was used to compare age, GAF, GDS15, and MoCA scores, followed by a least significant difference (LSD) post hoc test (with a Bonferroni correction also applied but not shown). An Analysis of Covariance (ANCOVA) with age as a covariate was also performed.

Receiver operating characteristic (ROC) analysis was used to calculate the area under the curve (AUC) as a measure of the MoCA's diagnostic accuracy. The MoCA can be used to detect dementia in a clinical setting and to rule out cognitive impairment. Therefore, different ROC curves were calculated: (a) to detect dementia in a clinical setting, (b) to detect overall cognitive impairment (MD + MCI) in a clinical setting, and (c) to detect MCI in a subgroup of patients (excluding MD). To compare these analyses with previous case-control studies and to assess the impact of bias, all analyses were repeated with healthy controls as the comparison group.

Positive Predictive Value (PPV) and Negative Predictive Value (NPV) were calculated for the "optimal" cutoff scores, determined using the Youden J index. Cronbach's alpha was calculated to assess the internal consistency of the MoCA.

RESULTS

Study groups

Of 2204 referrals, 1337 were not eligible for this study. A total of 867 referred patients underwent a MoCA assessment for this study, with a mean delay of 21.5 days from referral; 65% of these assessments occurred within three weeks. After applying the exclusion criteria (Figure 1), the study group comprised 710 participants: 81 with mild dementia (MD), 153 with mild cognitive impairment (MCI), 459 referred patients with no MCI or dementia (NoCI), and 17 inconclusive cases. The mean time required for diagnosis was 40.5 days for the NoCI group and 60.8 days for the cognitively impaired groups. For the secondary outcome, 84 healthy controls (HC) were included from an initial group of 96 potential healthy volunteers (Figure 1). Two of the healthy controls had an IQCode score between 3.25 and 3.5, indicating minor decline over the past 10 years, while all others scored between 3.0 and 3.25, indicating virtually no decline.

Demographic findings

Within the referred patient population, a significant difference in age (ANOVA F = 26.0 P = .000) was observed between the diagnostic groups, as expected, with patients with dementia generally being older. No significant differences were found in sex (P = .39) or education length (P = .142) among these groups. Disability, as measured by the GAF, showed expected differences: the MCI group exhibited the best GAF scores, while the dementia and NoCI groups (many of whom had psychiatric illnesses) had the worst GAF scores (P = .001). The GDS15, a measure of geriatric depression, showed no significant differences between the referred groups.

Regarding the secondary outcome, no significant differences in age, education, or sex were found between the overall referred patient population and the healthy controls (Table 1). As anticipated, significant differences were observed in GDS15 and GAF scores; healthy controls reported significantly fewer depressive symptoms (lower GDS15 scores) and exhibited better global functioning (higher GAF scores).

MoCA outcome

The mean MoCA scores differed significantly among the groups. Significant differences were observed in average MoCA scores between the individual referred patient groups (P = .000), as well as between the combined total referred group and the healthy controls (P = .000) for the secondary outcome. The standard deviations (from MCI towards NoCI) and the range (across all groups) of the referred groups showed overlap and a wide distribution (Table 1). The internal consistency of the MoCA, assessed by Cronbach's alpha on standardized items, was good (.761). All 12 items of the MoCA contributed positively to Cronbach's alpha, as no item deletion resulted in a higher outcome (.708-.737).

The results of the ROC analysis for clinical situations are presented in Figure 2A,B. Table 2 displays the AUCs for these and additional analyses, along with their sensitivity and specificity at the commonly recommended cutoff scores of 26 and 21. All AUCs were significantly different from 0.5 (indicating no diagnostic accuracy), with P < .001. When healthy controls were used as the secondary comparison group, the AUCs ranged from 0.90 to 0.98, demonstrating excellent accuracy. However, the MoCA performed less effectively in a clinical setting, with AUCs ranging between 0.70 and 0.87.

For the original suggested cutoff score of 26 to differentiate MCI from healthy controls, the sensitivity was 94% and specificity was 73% (compared to 90% sensitivity and 87% specificity in the original article). Applying the same cutoff score in a realistic clinical setting (i.e., discriminating against referred patients with no cognitive impairment) led to a substantial drop in specificity to 37%. In the clinical context, detecting cognitive impairment (MD + MCI) with a score below this cutoff yielded a sensitivity of 95%.

A definitive cutoff score for diagnosing dementia remains a subject of discussion but is often set around 21. In this study, a cutoff of 21 resulted in a sensitivity of 90% for detecting dementia. Specificity decreased significantly from 99% when comparing dementia patients with healthy controls, to 74% in a clinical setting (dementia vs. MCI + NoCI), and further to 63% when comparing dementia patients with MCI alone. To determine the "best" cutoff score for the study population, specificity and sensitivity were calculated for various MoCA scores (Table 3).

The "optimum" cutoff scores against patients with no cognitive impairment, as calculated by the Youden J index, were: below 25 for detecting MCI (sensitivity 88% [95% CI, 81-92], specificity 49% [95% CI, 44-53]); below 23 for overall cognitive impairment (CI) (sensitivity 75% [95% CI, 69-81], specificity 68% [95% CI, 63-72]); and below 21 for mild dementia (MD) (sensitivity 90% [95% CI, 81-95], specificity 78% [95% CI, 74-81]), which were comparable to findings in existing literature. The Positive Predictive Value (PPV) and Negative Predictive Value (NPV) were calculated (Table 4) for the two scores with the highest Youden index. The PPV was consistently low across almost all situations, whereas the NPV was high in all situations. For instance, using a cutoff of less than 21 for dementia resulted in 31% of positive MoCA results indicating MD, while 98% of negative tests truly indicated no MD. For detecting MCI at a cutoff of less than 26, 33% of positive MoCA results indeed had MCI, and 94% of those scoring above this threshold did not have MCI.

DISCUSSION

In this cross-sectional study, patients diagnosed with dementia were significantly older than those without, and each group included more females, which is representative of the population referred to old age psychiatry. Age has been shown to influence MoCA scores, with scores declining with aging, potentially altering results and their interpretation. However, age demonstrated minimal unique variance and a correlation of less than 10%. An additional ANCOVA sensitivity analysis, using age as a covariate, still revealed significant differences in MoCA scores among the various diagnostic groups in the study.

The Geriatric Depression Scale (GDS15) revealed no significant differences in depression scores among the referred groups. This finding further emphasizes the need for caution when using a screening tool like the GDS15 to differentiate or detect psychiatric causes of cognitive complaints.

The study replicated the significantly different mean MoCA scores reported in previous literature. The secondary outcome, which differentiated patients with mild dementia (MD) or mild cognitive impairment (MCI) from healthy controls (HC), showed properties comparable to those reported in earlier case-control studies. However, to avoid spectrum bias, the MoCA was studied in a cohort of patients referred to old age psychiatry, which provides a more accurate representation of clinical reality. Table 3 illustrates this point, showing that the Area Under the Curve (AUC) and specificity decrease when the comparison group is realistic (patients with no cognitive impairment, NoCI) rather than hypothetical (HC). While this bias is well-known and its effect on AUC has been previously demonstrated, it remains crucial to highlight its impact on optimal cutoff scores, as case-control study designs still constitute the majority of MoCA validation studies. Clinicians should exercise caution when using cutoffs derived from such studies. For instance, 27% of healthy controls had a MoCA score below 26, compared to 63% of the referred NoCI patients. The MoCA scores of the NoCI patients align with those from a longitudinal, population-based study (n = 2653; mean MoCA 23.36, 64% specificity below 26), indicating a realistic comparison group. Although a wide range of MoCA scores was observed in the study group, this occurred in a clinical setting and can be explained by several factors.

False negative results were observed in cases involving high educational and/or professional levels or frontotemporal dementia (FTD) within the dementia group. Conversely, false positive results occurred due to a lack of motivation and/or attention in depressed, manic, or psychotic patients, irrespective of whether they also had MCI. One might argue that the latter group should have been diagnosed with MCI due to their psychiatric conditions. However, based on the clinical opinion of the team following IADL investigation, their presentation was not persistent and did not warrant an MCI diagnosis, particularly since the MoCA score was not factored into the diagnostic decision.

A risk exists, including in this study, of subjective decisions regarding MCI diagnosis when a psychiatric disorder is considered its etiology, even if MCI criteria are met. This risk was minimized by incorporating a neuropsychological assessment during the diagnostic work-up when persistent impaired cognitive functioning was suspected. In the future, the MoCA could facilitate a more objective selection of potential MCI cases, helping to identify individuals in need of further evaluation.

False positives, specifically low MoCA scores due to unrecognized neurodegenerative MCI, could be largely excluded in this study because progression to any DSM IV diagnosis of cognitive impairment was monitored with a mean follow-up of 3.5 years. This study indicates that a threshold of 26 or higher can safely suggest normal cognition (95% sensitivity for cognitive impairment), provided specific situations like a university degree or frontotemporal dementia are considered.

While the MoCA effectively detects most mild dementia (scores below 21; 90% sensitivity) and mild cognitive impairment (scores below 26; 94% sensitivity) when scores fall below these cutoffs, making it suitable for screening, it is not appropriate for diagnosing MD or MCI in this study population. This is because the positive predictive value (PPV) for MD and MCI remains only fair (31% and 33% PPV, respectively). The proportion of referred psychiatric patients scoring below these cutoff scores is too high for diagnostic purposes (22% and 63% of patients with no cognitive impairment, respectively). The MoCA is suitable for excluding dementia (scores of 21 or higher; NPV 92%-98%) and MCI (scores of 26 or higher; NPV 94%) when used to assess patients referred to an old age psychiatry setting. This capability, combined with high sensitivity at these cutoffs, positions the MoCA as a valuable screening tool. If a test result is positive, further investigation is typically necessary. The absolute number of false positives is substantial, given that the majority of referred patients do not suffer from mild dementia. Using the study cohort as an example, applying a MoCA cutoff of less than 21 to screen 100 referred patients would result in 33 patients receiving specialized diagnostic tests. Of these, 14.7 would be patients with no cognitive impairment, 8.2 would have MCI, and 10.5 would be correctly identified as having MD. One patient (1.15) with MD would not be detected using this cutoff score. This illustrates that screening comes with associated costs, even in old age psychiatry. Further research is recommended to identify methods that increase specificity and improve the selection of individuals who require a specialized diagnostic pathway. The identified weaknesses of this study—unrealistic score scattering and seemingly missed CI diagnoses—would, in practice, be interpreted within a broader clinical context; incongruous MoCA results would be re-evaluated for clinical relevance or accuracy, or considered as indicative of cognitive impairment. This approach would enhance the MoCA's specificity. Future research should focus solely on suspected cognitive impairment referrals and explore whether MoCA reassessment after recovery from serious psychiatric episodes can reduce the false positive rate. Another limitation is that not all comparison groups received the same comprehensive diagnostic assessment due to practicality and resource constraints. However, as the inclusion of healthy controls primarily aimed to highlight the spectrum bias effect, this is considered acceptable. Patients with no cognitive impairment who were not initially suspected of cognitive impairment were followed for at least two years to compensate for this limitation. Those patients with no cognitive impairment who were suspected of cognitive impairment received the same full diagnostic assessment. The exclusion of patients with GDS scores of 5 or higher and those with BPSD could be viewed as a selection bias and a limitation. However, avoiding the extremes of the cognitive spectrum is considered a strength of this study. In clinical reality, individuals with obvious dementia are typically not screened to determine if they need a specialized diagnostic route. Including their low MoCA scores in the study would have biased the results.

CONCLUSION

This study indicates that validating the MoCA in a biased setting, such as against healthy controls, overestimates its specificity. The findings align with existing literature, which repeatedly suggests lower cutoff scores to address this issue.

Considering these results, the MoCA can be useful in an old age psychiatric setting to confirm normal cognitive functioning and to identify individuals who require a specialized diagnostic pathway. However, further research is necessary to minimize the number of false positives in the latter group.

INTRODUCTION

The Montreal Cognitive Assessment (MoCA) was developed as a brief screening tool for mild cognitive impairment (MCI) and is widely used globally. The Alzheimer Society recommends its use for objectively assessing cognitive concerns in clinical settings. While the idea of screening for dementia across different populations is debated, its value in old age psychiatry is distinct. Understanding a patient's cognitive function upon referral helps with timely dementia detection and monitoring of all causes of MCI in older adults. This knowledge allows for adapting psychiatric treatments, including medication adherence or psychotherapy, particularly since this patient group faces a higher risk of cognitive changes due to age, psychotropic medications, or their referral reasons. In the Netherlands, referrals to old age psychiatry often include a mix of neurodegenerative conditions and psychiatric disorders like depression, bipolar disorder, schizophrenia, and severe anxiety, all of which can involve cognitive difficulties.

To reduce delays in diagnosis, a short cognitive assessment using the MoCA was introduced in the clinic for all referred patients. This provides an objective aid for identifying individuals needing specialized diagnostic evaluation, alongside gathering baseline cognitive data. Therefore, understanding the MoCA's accuracy in this specific setting is crucial.

The MoCA generally shows good validity across various languages, though it has been moderately validated in Dutch geriatric memory clinics. It is important to validate the MoCA within specific clinical environments because the characteristics of the patient group can influence the test's performance. This is particularly relevant when studies use healthy community members as controls, which may not accurately reflect real-world clinical situations. The MoCA had not been validated previously in old age psychiatry settings, where patients present with diverse causes for MCI. This study is considered the first of its kind in this area. Distinguishing cognitive impairment caused by a psychiatric condition from early-stage dementia can be complex and may affect how the MoCA performs.

A Cochrane review suggests that the MoCA can help identify individuals who need specialized assessment and treatment for dementia.

This study aimed to validate the MoCA within this clinical setting by following established reporting standards for diagnostic accuracy (STARD 2015) and using a cross-sectional design. The primary objective was to assess the MoCA's criterion validity—its ability to correctly predict a diagnosis—in detecting MCI and early-stage or mild dementia (MD) within an old age psychiatry patient group, including referred patients who do not have cognitive impairment. The standard for diagnosis was a consensus-based decision made by a multidisciplinary team, adhering to international criteria. This cross-sectional design helps avoid "spectrum bias," which often occurs in case-control studies that compare individuals with extreme levels of cognitive function. To illustrate this bias, a secondary outcome involved presenting MoCA results using a case-control design that included healthy community controls with normal cognitive aging as comparisons.

METHOD

Sample

This study was conducted at an old age psychiatry outpatient clinic in a large Dutch city (Utrecht), which serves individuals aged 60 and older in its northwestern area and surrounding rural regions. All new patients referred between 2008 and 2018 were eligible if they could provide written informed consent. Patients with severe dementia (Global Deterioration Scale [GDS] 6 or higher), Behavioral and Psychological Symptoms of Dementia (BPSD), or those referred compulsorily were not included.

A multidisciplinary team, always including an old age psychiatrist and a trained psychiatric nurse practitioner, assessed participants. To ensure the study population resembled a typical clinical screening group, patients with clear reasons for their cognitive complaints were excluded. These included individuals with severe mid-stage dementia (GDS 5 or higher), a history of substance use within the past two years, recent delirium (within six months), or an acquired brain injury such as a cerebrovascular accident (CVA) or transient ischemic attack (TIA). Additionally, patients who did not have sufficient command of the Dutch language were excluded.

For the secondary study, which compared MoCA properties in a less realistic scenario, a group of healthy community controls (aged 60 and older) was recruited through acquaintances of patients or research assistants. Inclusion criteria for this group were no cognitive complaints and no risk factors for cognitive dysfunction. Exclusion criteria included acquired brain injury (CVA or TIA), substance use, recent delirium, recent treatment for psychiatric or neurological diseases, and use of medications that could alter cognitive function. For potential healthy controls who showed signs of cognitive impairment during the interview or scored below 25 on the MoCA, consent was obtained to interview their next of kin (11 individuals). These next of kin were assessed using the Modified Informant Questionnaire on Cognitive Decline in the Elderly (IQCode). No potential healthy controls had an IQCode higher than 3.5, which would have indicated potential moderate cognitive impairment and led to exclusion.

The Committee for Research and Ethics at the institution approved this study (CWO-nr 1606), and all participants provided their informed consent. Data are available upon request, subject to privacy and ethical restrictions.

Measurements

An old age psychiatrist conducted the initial assessment, which included gathering medical history from the patient's next of kin and performing relevant laboratory tests for cognitive impairment. During the diagnostic process, scores on the 15-item Geriatric Depression Scale (GDS15) and the Global Assessment of Functioning (GAF) were collected. A psychiatric nurse practitioner conducted an investigation of instrumental activities of daily living (IADL) during a home visit. If this initial assessment raised any suspicion of cognitive impairment, further evaluation occurred, including a neuropsychological assessment (for 289 patients) and, when appropriate, CT/MRI imaging and cerebrospinal fluid (CSF) analysis. A neuropsychologist, who was unaware of the MoCA score, performed the neuropsychological assessment. This was a thorough and comprehensive evaluation encompassing multiple tests across memory, attention, executive function, fluid intelligence, and language abilities.

The healthy controls were interviewed and assessed by research assistants. Their assessment, completed in a single day, included the MoCA, GDS15, and GAF.

Diagnostic test

All referred participants underwent a MoCA assessment as soon as possible, within a maximum of three months from referral, administered by a trained research assistant or psychiatric nurse practitioner. This assessment was independent of the main diagnostic procedure. The MoCA was typically administered during the feedback appointment following the initial assessment, when the treatment plan was discussed. The treatment plan included referral to the memory clinic for further assessment if there was any doubt or suspicion of cognitive impairment.

The MoCA is a single-page test that assesses various cognitive domains, including executive function, visuospatial abilities, naming, short-term memory, attention, working memory, language, concentration, verbal abstraction, and orientation. It can be completed within 10 minutes, with a maximum score of 30 indicating no errors. Scores were adjusted for lower education levels, as per instructions, by adding one point to the total score for patients with 12 years of education or less. The originally suggested cutoff score for diagnosing cognitive impairment was 26 or below.

Reference test

The reference standard for diagnosis was determined during multidisciplinary team meetings, which included an old age psychiatrist, a neuropsychologist, and a geriatrician.

The diagnoses of dementia and MCI were supported by at least a neuropsychological assessment and laboratory tests. Diagnoses were reached by consensus and adhered to international criteria for MCI or the Dutch guideline on dementia. This guideline covers criteria from DSM IV for dementia, NIA-AA/NINCDS-ADRDA for Alzheimer's disease, NINDS-AIREN/AHA-ASA for vascular dementia, The Lund and Manchester Groups for frontotemporal dementia (FTD), and the Consensus for Dementia with Lewy Body. The MCI group included individuals with MCI due to psychiatric causes, consistent with international consensus. No further differentiation of MCI subtypes was made in this study. The MoCA results were not used in determining the diagnoses of MCI or dementia.

Referred patients who did not show signs of cognitive impairment during the initial assessment were followed for a minimum of two years to account for not having a full neuropsychological assessment. Patients who did not meet the criteria for dementia or MCI during this follow-up period were categorized as having no cognitive impairment (NoCI). Patients who met the criteria after the initial three months during follow-up were cautiously classified as inconclusive (3 individuals).

Statistical analyses

Results were compared within the referred patient groups (mild dementia [MD], MCI, or NoCI) and between the combined group of all referred patients (MD + MCI + NoCI) and healthy controls (HC). Statistical analyses were performed using SPSS (version 22). A chi-squared test compared sex and education. ANOVA was used to compare age, GAF, GDS15, and MoCA scores, followed by a least significant difference (LSD) post hoc test (with a Bonferroni correction also performed but not shown). An ANCOVA with age as a covariate was also conducted.

Receiver operating characteristic (ROC) analysis was used to calculate the area under the curve (AUC), which indicates the MoCA's diagnostic accuracy. Since the MoCA can be used both to detect dementia and to rule out cognitive impairment in a clinical setting, different ROC curves were calculated: (a) for detecting dementia in a clinical setting, (b) for detecting cognitive impairment (MD + MCI) in a clinical setting, and (c) for detecting MCI in a subgroup of patients (excluding MD). To compare these analyses with previous case-control studies and to demonstrate the effect of bias, all analyses were repeated using healthy controls.

Positive predictive value (PPV) and negative predictive value (NPV) were calculated for the "optimal" cutoff scores determined by the Youden J index. Cronbach's alpha was calculated to assess the internal consistency of the MoCA.

RESULTS

Study groups

Out of 2204 referrals, 1337 were not eligible for this study. Eight hundred sixty-seven referred patients underwent a MoCA assessment for this study, with an average delay of 21.5 days, and 65% assessed within three weeks of referral. After applying the exclusion criteria (as outlined in Figure 1 of the original document), a group of 710 participants remained: 81 with mild dementia (MD), 153 with mild cognitive impairment (MCI), 459 referred patients with no MCI or dementia (NoCI), and 17 categorized as inconclusive. The average time needed for diagnosis was 40.5 days for the NoCI group and 60.8 days for the cognitively impaired (CI) group. For the secondary outcome, 84 healthy controls (HC) were included from an initial group of 96 potential healthy volunteers (as shown in Figure 1 of the original document). Two of these had an IQCode score between 3.25 and 3.5, indicating minor cognitive decline over the past decade, while all others scored between 3.0 and 3.25, suggesting little to no decline.

Demographic findings

Within the referred patient population, a significant difference in age was observed between the diagnostic groups (ANOVA F = 26.0, P = .000), which was expected. There were no significant differences in sex (P = .39) or length of education (P = .142). Disability, as measured by the GAF, showed an anticipated pattern: the MCI group had the best scores, while patients with dementia and the NoCI group (many of whom had psychiatric illnesses) had the worst GAF scores (P = .001). The GDS15 showed no significant differences among the referred groups.

Regarding the secondary outcome, no significant differences were found in age, education, or sex between the total referred patient population and the healthy controls (Table 1 of the original document). The significant differences in GDS15 and GAF were as expected, with healthy controls exhibiting significantly fewer depressive symptoms (lower GDS15 scores) and better overall functioning (higher GAF scores).

MoCA outcome

The average MoCA scores differed significantly across all groups. Significant differences were observed between the individual referred diagnostic groups (P = .000), as well as between the combined total referred group and the healthy controls (P = .000) for the secondary outcome. The standard deviations (particularly from MCI towards NoCI) and the overall range of scores (across all groups) within the referred populations overlapped and displayed a wide distribution (Table 1 of the original document). The internal consistency of the MoCA, as indicated by a Cronbach's alpha of .761 on standardized items, was good. All 12 items of the MoCA contributed positively to the Cronbach's alpha, with no item showing a higher outcome if deleted (.708–.737).

The results of the ROC analysis, relevant for clinical situations, are presented in Figure 2A,B of the original document. Table 2 of the original document displays the AUCs for these and additional analyses, along with their sensitivity and specificity at commonly used cutoff scores of 26 and 21. All AUCs were significantly different from 0.5 (indicating no diagnostic accuracy), with P < .001. When healthy controls were used as the secondary comparison, the AUCs ranged from 0.90 to 0.98, demonstrating excellent accuracy. However, the MoCA performed less effectively in a realistic clinical setting, with AUCs ranging between 0.70 and 0.87.

For the original suggested cutoff score of 26 to differentiate MCI from healthy controls, the sensitivity and specificity were 94% and 73%, respectively (compared to 90% and 87% in the original article). Using the same cutoff score in a realistic clinical setting (i.e., distinguishing from referred patients with no cognitive impairment) led to a notable drop in specificity to 37%. In this clinical context, detecting cognitive impairment (mild dementia + MCI) below this cutoff score had a sensitivity of 95%.

The optimal cutoff score for diagnosing dementia remains a subject of debate but is often set around 21, which in this study resulted in a sensitivity of 90%. However, the specificity decreased from 99% (when comparing dementia with healthy controls) to 74% in a clinical setting (dementia versus MCI + NoCI), and to 63% for dementia versus MCI. To identify the "best" cutoff score for the study population, the specificity and sensitivity were calculated for various MoCA scores (Table 3 of the original document).

Based on the Youden J index, the "optimal" cutoff scores against patients with no cognitive impairment were: less than 25 for detecting MCI (sensitivity 88% [95% CI, 81–92], specificity 49% [95% CI, 44–53]); less than 23 for detecting cognitive impairment (sensitivity 75% [95% CI, 69–81], specificity 68% [95% CI, 63–72]); and less than 21 for detecting mild dementia (sensitivity 90% [95% CI, 81–95], specificity 78% [95% CI, 74–81]), which aligns with existing literature.

The positive predictive value (PPV) and negative predictive value (NPV) were calculated for the two scores with the highest Youden index (Table 4 of the original document). The PPV and NPV showed different results. The PPV was consistently low in most situations, while the NPV was high across all scenarios. For example, using a cutoff of less than 21 for dementia means that 31% of positive MoCA results truly indicate mild dementia, whereas 98% of negative tests correctly indicate no mild dementia. For detecting MCI at a cutoff of less than 26, 33% of individuals with a positive MoCA result indeed have MCI, and 94% scoring above this threshold will not have MCI.

DISCUSSION

In this cross-sectional study, patients with dementia were significantly older than those without, and each group contained more females, which accurately reflects the population typically referred to old age psychiatry. While age has been shown to influence MoCA scores, with declines occurring as people age, its unique variance in this study was small, with a correlation of less than 10%. An additional sensitivity analysis using ANCOVA, with age as a covariate, still revealed significant differences in MoCA scores between the various diagnostic groups. The Geriatric Depression Scale (GDS15) showed no significant differences among the referred groups, emphasizing the need for caution when using such screening tools to differentiate or detect psychiatric causes of cognitive complaints.

This study replicated the significantly different mean MoCA scores previously reported in literature. The secondary outcome, which differentiated patients with mild dementia or MCI from healthy controls, showed properties comparable to those reported in earlier case-control studies. However, to avoid spectrum bias, this study evaluated the MoCA in a cohort of patients referred to old age psychiatry, which provides a more accurate representation of clinical reality. As illustrated in Table 3 of the original document, the area under the curve (AUC) and specificity decrease when the comparison group is realistic (patients with no cognitive impairment) rather than artificial (healthy controls). While this bias and its effect on AUC are known, it remains crucial to emphasize its impact on optimal cutoff scores, given that case-control studies still constitute the majority of MoCA validation research. Clinicians should therefore exercise caution when using cutoff scores derived from such studies. For instance, 27% of healthy controls had a MoCA score below 26, compared to 63% of the referred patients with no cognitive impairment. The MoCA scores of the no cognitive impairment patients in this study align with those from a longitudinal, population-based study (n = 2653; mean MoCA 23.36, 64% specificity below 26), suggesting a realistic comparison group. Although a wide range of MoCA scores was observed in the study group, this occurred in a clinical setting and can be explained by several factors.

False negative results were observed in cases of high educational and/or professional levels or frontotemporal dementia within the dementia group. Conversely, false positive results occurred due to a lack of motivation and/or attention in patients with depression, mania, or psychosis, regardless of whether they also had MCI. One might argue that the latter group should have been diagnosed with MCI due to their psychiatric conditions. However, based on the clinical team's assessment, which included IADL investigation and did not consider the MoCA score, their presentation was not persistent enough to warrant an MCI diagnosis.

There is a risk, including in this study, of subjective decisions regarding MCI diagnosis when a psychiatric disorder explains its origin, even if the criteria for MCI are met. This risk was minimized by incorporating a neuropsychological assessment during the diagnostic work-up when there was suspicion of persistent cognitive impairment. In the future, the MoCA could offer a more objective and streamlined way to identify potential MCI cases that require further evaluation. False positives (i.e., low MoCA scores) due to unrecognized neurodegenerative MCI were largely excluded in this study, as progression to any DSM IV diagnosis of cognitive impairment was monitored with a mean follow-up of 3.5 years.

This study indicates that using a threshold of 26 or greater to signify normal cognition is safe, provided specific situations, such as a university degree or frontotemporal dementia, are taken into account. While the MoCA effectively identifies most cases of mild dementia (scores below 21; 90% sensitivity) and MCI (scores below 26; 94% sensitivity), making it suitable for screening, it is not appropriate for diagnosing mild dementia or MCI in this study's patient population. This is because the positive predictive values (PPV) for mild dementia and MCI remain only fair (31% and 33% PPV, respectively). The proportion of referred psychiatric patients scoring below these cutoff scores is too high for diagnostic purposes (22% and 63% of patients with no cognitive impairment, respectively).

The MoCA is suitable for ruling out dementia (scores 21 or greater; NPV 92%–98%) and MCI (scores 26 or greater; NPV 94%) when used to assess patients referred to an old age psychiatry setting. This, combined with high sensitivity at these cutoffs, makes the MoCA a useful screening tool. However, in cases of a positive test result, further diagnostic work-up is usually necessary because the absolute number of false positives is substantial, given that most referred patients do not suffer from mild dementia. Using the study cohort as an example, applying a MoCA cutoff of less than 21 to screen 100 referred patients would lead to 33 patients undergoing specialized diagnostic tests. Of these, approximately 14.7 would have no cognitive impairment, 8.2 would have MCI, and 10.5 would be correctly identified with mild dementia. One patient (approximately 1.15) with mild dementia would not be detected using this cutoff score. This confirms that screening, even in old age psychiatry, carries a cost, primarily due to false positives. The study recommends further research to identify methods that enhance specificity and improve the selection of individuals who truly need a specialized diagnostic pathway. The study's identified weaknesses—such as unrealistic data scattering and seemingly missed cognitive impairment diagnoses—would, in clinical practice, be interpreted within a broader clinical context; incongruous MoCA results would be re-evaluated for clinical relevance or accuracy, or considered indicative of cognitive impairment. Such an approach would increase the MoCA's specificity. Future research should focus specifically on referrals where cognitive impairment is suspected and investigate whether reassessing MoCA scores after recovery from severe psychiatric episodes can reduce the false positive rate. Another limitation is that not all comparison groups received the same comprehensive diagnostic assessment due to practical and resource constraints. However, as the inclusion of healthy controls primarily aimed to highlight the spectrum bias effect, this limitation is considered acceptable. Patients with no cognitive impairment who were not initially suspected of cognitive impairment did not receive a full diagnostic work-up but were followed for at least two years to mitigate this limitation. Those with no cognitive impairment who were suspected of cognitive impairment did receive the same full diagnostic assessment. Excluding patients with GDS scores of 5 or higher and those with BPSD could be viewed as selection bias and a limitation. However, avoiding the extremes of the cognitive spectrum is considered a strength of this study, as in clinical reality, individuals with obvious dementia are typically not screened to determine if they need a specialized diagnostic route, and including their low MoCA scores would bias the results.

CONCLUSION

This study demonstrates that validating the Montreal Cognitive Assessment (MoCA) in a biased setting, such as against healthy controls, leads to an overestimation of its specificity. The findings align with existing literature, which consistently suggests that lower cutoff scores are needed to address this issue.

Considering these results, the MoCA can be concluded as a useful tool in an old age psychiatry setting. It helps confirm normal cognitive functioning and identify individuals who require a specialized diagnostic pathway for potential cognitive impairment. However, further research is necessary to minimize the number of false positives in the latter group, thereby improving the efficiency of the screening process.

Introduction

The Montreal Cognitive Assessment (MoCA) was developed as a quick test to identify mild cognitive impairment (MCI). It is now used widely around the world in many different healthcare settings. The Alzheimer Society recommends the MoCA for objectively checking memory problems reported by patients in a clinical environment.

While some groups support screening for dementia more broadly, there is ongoing discussion about whether this is always the best approach for all groups of people. However, the situation in old age psychiatry is different. Knowing a patient's cognitive function when they are referred helps doctors quickly spot dementia and also monitor all types of MCI in older adults. This information allows medical professionals to adjust a patient's treatment, including medications or therapy. This is especially important because older adults are at higher risk of changes in cognitive function due to age, medications, or their reasons for seeking care. In the Netherlands, patients referred to old age psychiatry often have a mix of brain disorders (like dementia) and other psychiatric conditions (such as depression, bipolar disorder, schizophrenia, and severe anxiety disorders), all of which can affect cognitive abilities.

In the clinic where this study took place, a short cognitive assessment using the MoCA was introduced for all referred patients. This was done to help doctors identify patients who needed specialized diagnostic tests more quickly and to collect baseline information about their cognitive health. Therefore, it became important to understand how accurate the MoCA is in this specific clinical setting.

The MoCA has shown good reliability in many languages, though its performance was only moderate in Dutch in a geriatric memory clinic. It is crucial to test the MoCA's accuracy in specific healthcare settings because the characteristics of the patients being studied can affect the test's results. This is particularly true for studies that compare patients with cognitive impairment to healthy people from the general community, as these healthy individuals may not represent real clinical patients. The MoCA had not been fully evaluated in old age psychiatry settings before this study, where patients are referred for many different reasons that can cause MCI. This study is believed to be the first of its kind. Distinguishing cognitive impairment caused by a mental health condition from impairment due to early dementia is complex and can influence how well the MoCA performs.

A review of medical studies suggests that the MoCA can help identify people who need specialized assessment and treatment for dementia.

This study aimed to validate the MoCA in an old age psychiatry clinic, following specific reporting guidelines for diagnostic tests. Researchers used a single-point-in-time study design to test whether the MoCA could correctly predict a diagnosis of MCI and early or mild dementia in older adult psychiatry patients. This included comparing results to patients who were referred but did not have cognitive impairment. The accurate diagnosis was determined by a team of specialists using agreed-upon international standards. This study design helped avoid a common problem in research where the test is evaluated only on individuals at the extreme ends of the cognitive spectrum. To show how this problem can affect results, the study also presented MoCA outcomes from a comparison that included healthy older adults from the community as a secondary group.

Method

Sample

This study was conducted in an outpatient clinic for older adults (aged 60 and above) in Utrecht, a large city in the Netherlands. The clinic serves residents from the northwest part of the city and nearby rural areas. All new patients referred between 2008 and 2018 were considered for the study. To be included, patients needed to be able to provide written informed consent. Therefore, patients referred with severe dementia, severe behavioral and psychological symptoms of dementia, or those who were referred against their will were not included.

Patients who had an obvious cause for their cognitive complaints were also excluded to ensure the study group resembled a typical clinical screening population. These exclusions included patients with severe mid-stage dementia, a recent history of substance use (within the past two years), a recent episode of delirium (within the past six months), or a brain injury like a stroke or mini-stroke. Additionally, patients who did not speak Dutch well enough were excluded.

For a secondary part of the study, a group of healthy adults aged 60 and above from the community was included. They were recruited through acquaintances of patients or research assistants. To be included, healthy individuals could not have any cognitive complaints or risk factors for cognitive dysfunction. They were excluded if they had a brain injury, substance use issues, recent delirium, recent treatment for psychiatric or neurological conditions, or were taking medications that could affect cognitive function. If potential healthy participants showed any signs of cognitive decline during an interview or had a low MoCA score, their next of kin were interviewed using a questionnaire about cognitive decline; none of these healthy participants showed signs of moderate cognitive impairment.

The study was approved by the institution's Committee for Research and Ethics. All participants provided their informed consent. The data can be made available upon request to protect privacy.

Measurements

The initial assessment was conducted by an old age psychiatrist, who gathered a medical history from the patient's next of kin and ordered relevant lab tests to check for cognitive impairment. During the diagnostic process, scores for depression (using the GDS15) and overall functioning (using the GAF) were also collected. A psychiatric nurse practitioner assessed how well patients could manage daily activities during a home visit.

If this initial assessment raised any suspicion of cognitive impairment, further tests were conducted. These included a comprehensive neuropsychological assessment for many patients (about 289 of them) and, when appropriate, brain imaging (CT/MRI) and cerebrospinal fluid analysis. The neuropsychological assessments were performed by a neuropsychologist who did not know the patient's MoCA score. These were thorough evaluations covering a wide range of cognitive abilities, such as memory, attention, executive function (like planning and problem-solving), fluid intelligence, and language skills.

The healthy control group underwent assessment by research assistants on a single day, which included the MoCA, the GDS15, and the GAF.

Diagnostic test

All referred participants took the MoCA as early as possible, within three months of their referral, and this was done by a trained research assistant or psychiatric nurse practitioner. This MoCA assessment was separate from the main diagnostic process. The MoCA results were discussed during the feedback appointment, when the treatment plan was presented. If there was any doubt or suspicion of cognitive impairment, the treatment plan included a referral to the memory clinic for further assessment.

The MoCA is a one-page test that assesses various cognitive areas, including executive function, visuospatial abilities, naming, short-term memory, attention, working memory, language, concentration, abstract thinking, and orientation. It typically takes about 10 minutes to complete. The highest possible score is 30, meaning no errors were made. Scores were adjusted for lower education levels by adding one point to the total score for patients with 12 years of education or less. The original suggested cutoff score for diagnosing cognitive impairment was below 26.

Reference test

The definitive diagnosis used for comparison was determined by a multidisciplinary team, which included an old age psychiatrist, a neuropsychologist, and a geriatrician.

Diagnoses of dementia and MCI were supported by a minimum of a neuropsychological assessment and laboratory tests. The diagnoses were made through team consensus and followed international criteria for MCI or the Dutch guidelines for dementia. These guidelines cover various types of dementia, including those outlined by DSM IV, Alzheimer's disease criteria, vascular dementia criteria, and criteria for frontotemporal dementia and dementia with Lewy bodies. The MCI group also included cases where MCI was due to psychiatric causes, consistent with international consensus. The study did not further differentiate between types of MCI. It is important to note that the MoCA results were not used to make the final diagnosis of MCI or dementia.

Referred patients who did not show signs of cognitive impairment during their initial assessment were followed for at least two years. This follow-up helped compensate for not having a neuropsychological assessment for everyone. Patients who did not meet the criteria for dementia or MCI during this follow-up period were categorized as having no cognitive impairment (NoCI). A small number of patients (three) who met the criteria for dementia or MCI after the initial three-month period during follow-up were classified as "inconclusive" to be cautious.

Statistical analyses

Results were compared among referred patients with mild dementia (MD), mild cognitive impairment (MCI), or no cognitive impairment (NoCI). Comparisons were also made between the entire group of referred patients (MD + MCI + NoCI) and the healthy control (HC) group. Standard statistical software (SPSS, version 22) was used for the analysis.

Chi-squared tests were used to compare sex and education levels between groups. ANOVA was used to compare age, overall functioning (GAF), depression scores (GDS15), and MoCA scores, followed by specific post-hoc tests to identify where differences lay. An additional analysis, ANCOVA, was performed using age as a factor to account for its potential influence.

Receiver operating characteristic (ROC) analysis was used to calculate the area under the curve (AUC), which indicates the MoCA's diagnostic accuracy. Different ROC curves were calculated to reflect various clinical scenarios: (a) detecting dementia, (b) detecting any cognitive impairment (MD + MCI), and (c) detecting MCI only (excluding MD). To compare these findings with previous studies and to show how bias can affect results, all analyses were repeated using the healthy control group for comparison.

Finally, positive predictive values (PPV) and negative predictive values (NPV) were calculated for the "optimal" cutoff scores, which were determined using a statistical index. Cronbach's alpha was calculated to assess the internal consistency of the MoCA.

Results

Study groups

Out of 2204 initial referrals, 1337 were not eligible for this study. A total of 867 referred patients underwent a MoCA assessment for the study. On average, the MoCA was administered 21.5 days after referral, with 65% of assessments taking place within three weeks. After applying additional exclusion criteria, the final study group included 710 participants: 81 with mild dementia (MD), 153 with mild cognitive impairment (MCI), 459 referred patients with no MCI or dementia (NoCI), and 17 whose status was inconclusive. The average time needed to reach a diagnosis was 40.5 days for the NoCI group and 60.8 days for those with cognitive impairment. For the secondary part of the study, 84 healthy controls (HC) were included from an initial group of 96 potential volunteers. Only two of these healthy controls showed slight indications of minor cognitive decline over the past decade, while the rest showed almost no decline.

Demographic findings

Among the referred patients, there was a significant age difference between the diagnostic groups, which was expected. However, there were no significant differences in sex or length of education. The measure of disability, assessed by the GAF, showed expected differences: patients with MCI had the best GAF scores, while those with dementia and those with no cognitive impairment (most of whom had psychiatric illnesses) had the worst scores. The depression scores (GDS15) showed no significant differences among the referred groups.

For the secondary outcome, comparing referred patients to healthy controls, there were no significant differences in age, education, or sex between the groups. As anticipated, the healthy controls had significantly fewer depressive symptoms and better overall functioning compared to the referred patients.

MoCA outcome

The average MoCA scores differed significantly between the diagnostic groups. Differences were also significant when comparing the combined group of all referred patients with the healthy control group. However, the range of scores within the referred groups, especially between MCI and NoCI, overlapped considerably, showing a wide distribution of scores. The MoCA showed good internal consistency, meaning its different items generally measure the same thing.

Statistical analysis (ROC analysis) showed that all results were significantly better than simply guessing. When healthy controls were used for comparison, the MoCA showed excellent accuracy (AUCs between 0.90 and 0.98). However, the MoCA performed less accurately in a clinical setting, where the comparisons were made between different groups of referred patients (AUCs between 0.70 and 0.87).

For example, using the original suggested cutoff score of below 26 to distinguish MCI from healthy controls resulted in high sensitivity (94%) but lower specificity (73%). When this same cutoff was applied in a realistic clinical setting, trying to distinguish MCI from referred patients who had no cognitive impairment, the specificity dropped significantly to 37%. To detect any cognitive impairment (mild dementia + MCI) at this cutoff, the MoCA had a sensitivity of 95%. A cutoff score often used for diagnosing dementia is around 21; in this study, it resulted in a sensitivity of 90%. While this cutoff showed very high specificity (99%) when compared to healthy controls, it dropped to 74% in a clinical setting (when distinguishing dementia from MCI and NoCI patients), and to 63% when distinguishing dementia from only MCI patients.

When the "best" cutoff scores were calculated for this study's population using statistical methods, a score below 25 was optimal for detecting MCI (88% sensitivity, 49% specificity), a score below 23 for any cognitive impairment (75% sensitivity, 68% specificity), and a score below 21 for mild dementia (90% sensitivity, 78% specificity). The positive predictive value (PPV), which indicates the likelihood of a positive test result actually being correct, was low in almost all situations. This means a low MoCA score often did not reliably confirm a diagnosis. In contrast, the negative predictive value (NPV), which indicates the likelihood of a negative test result actually being correct, was high in all situations. For example, a MoCA score below 21 for dementia meant that only 31% of those with a positive MoCA result actually had mild dementia. However, a negative test (score of 21 or higher) meant 98% did not have mild dementia. Similarly, for detecting MCI with a cutoff below 26, 33% of those with a positive MoCA had MCI, but 94% of those above this threshold did not have MCI.

Discussion

In this study, patients diagnosed with dementia were significantly older than those without. Each group had more females, which is typical for patients referred to old age psychiatry. Although age can influence MoCA scores, further statistical analysis in this study confirmed that significant differences in MoCA scores remained between the diagnostic groups even when accounting for age.

The geriatric depression scale (GDS15) showed no differences between the referred groups. This finding suggests that a simple screening tool like the GDS15 may not be effective at distinguishing between or identifying cognitive problems caused by psychiatric conditions alone.

This study confirmed previous findings that average MoCA scores differ significantly between diagnostic groups. The secondary comparison, distinguishing patients with mild dementia or MCI from healthy controls, showed results similar to earlier studies that used healthy individuals for comparison. However, this study aimed to avoid "spectrum bias" by studying a group of patients referred to old age psychiatry, which more accurately represents a real clinical situation. This difference is clear when comparing results: the MoCA's accuracy and its ability to correctly identify people without impairment drop when using real patients without cognitive issues for comparison, instead of healthy individuals. For example, 27% of healthy controls had a MoCA score below 26, compared to 63% of referred patients without cognitive impairment. The MoCA scores of these non-cognitively impaired patients in this study align with findings from a large population study, indicating that this study's comparison group was realistic.

The wide range of MoCA scores observed in the study's patient group can be explained by several factors. False negative results (a high MoCA score despite cognitive issues) were seen in patients with high education or professional backgrounds, or in those with certain types of dementia like frontotemporal dementia. False positive results (a low MoCA score without true cognitive impairment) occurred in patients who lacked motivation or attention due to depression, mania, or psychosis, sometimes even without MCI. The clinical team, after evaluating daily living activities, determined that these presentations were not persistent and did not warrant an MCI diagnosis, especially since the MoCA score was not used in their diagnostic decision.

A potential issue in studies like this is the subjective nature of diagnosing MCI, especially when a psychiatric disorder might explain the symptoms. This study minimized this risk by including thorough neuropsychological assessments when persistent cognitive impairment was suspected. In the future, the MoCA could offer a more objective way to identify potential MCI cases that need further investigation.

This study could rule out false positives due to unrecognized neurodegenerative MCI because patients without suspected cognitive impairment were monitored for an average of 3.5 years.

This study suggests that a MoCA score of 26 or higher can reliably indicate normal cognitive function (with 95% accuracy for ruling out cognitive impairment), although exceptions like very high education levels or specific types of dementia should be considered.

While the MoCA effectively detects most cases of mild dementia (scores below 21, 90% sensitivity) and MCI (scores below 26, 94% sensitivity), making it suitable for screening, it is not appropriate for definitive diagnosis in this study's patient population. The positive predictive value (PPV) for both mild dementia and MCI was relatively low (31% and 33% respectively). This means that a significant number of referred psychiatric patients without cognitive impairment also scored below these cutoffs (22% and 63% respectively), making the test unreliable for diagnostic purposes.

The MoCA is highly effective at ruling out dementia (scores of 21 or higher, 92%-98% accuracy) and MCI (scores of 26 or higher, 94% accuracy) when used for patients referred to old age psychiatry. This, combined with its high sensitivity at these cutoffs, makes the MoCA a valuable screening tool.

If a patient tests positive on the MoCA, further evaluation is usually necessary because a substantial number of those referred do not have mild dementia. This means there are many "false positives" in a screening context. For instance, if 100 referred patients were screened using a MoCA cutoff of less than 21, approximately 33 would be referred for specialized diagnostic tests. Of these, about 15 would be patients without cognitive impairment, 8 would have MCI, and 11 would be correctly identified as having mild dementia. Approximately one patient with mild dementia would be missed using this cutoff. This illustrates that while screening is beneficial in old age psychiatry, it comes with the "cost" of identifying many individuals who ultimately do not have a cognitive disorder.

Further research is recommended to find ways to improve the MoCA's ability to correctly identify those with cognitive impairment and to better select patients who truly need a specialized diagnostic pathway. The study acknowledges that some seemingly incorrect MoCA results might be interpreted differently in a broader clinical context. Future research could focus on patients with suspected cognitive impairment and explore whether re-evaluating MoCA scores after serious psychiatric episodes might reduce false positive rates. A limitation of this study was that not all comparison groups received the same extensive diagnostic assessment, though this was acceptable given the aim to highlight the "spectrum bias." The patients without suspected cognitive impairment were followed for at least two years to address this limitation. Also, excluding patients with severe dementia or severe behavioral symptoms was considered a strength, as it avoids biasing results with extreme cases who would not typically need screening for a specialized diagnostic route.

Conclusion

This study demonstrates that evaluating the MoCA in a biased setting, such as against healthy controls, leads to an overestimation of its ability to correctly identify individuals who do not have the condition. These findings align with existing research, which frequently suggests using lower cutoff scores to address this issue.

Considering these results, the MoCA can be a useful tool in old age psychiatry. It can help confirm normal cognitive function and identify individuals who require further specialized diagnostic assessment. However, additional research is needed to reduce the number of false positives in this latter group.

Introduction

The MoCA test was created to quickly check for mild thinking problems. It is used widely to help doctors see if someone has memory or thinking issues, especially when an older person feels their mind is not working as well. It is important to test how well the MoCA works in different real-world settings. This study looked at how well the MoCA test helped find mild thinking problems and early stages of memory loss in older adults who were seeing a doctor for mental health concerns. The goal was to see if the MoCA could correctly point to these issues in a group of patients who have a mix of problems, not just those who are clearly very sick or very healthy. This helps make sure the test results are useful for real patients.

Method

This study involved older patients (60 and up) from a mental health clinic in the Netherlands. Not included were patients with very severe memory loss or certain other serious health issues. Some healthy older people were also part of the study for comparison. Patients were checked by a team of mental health experts who looked at their medical history, daily activities, and sometimes gave detailed thinking tests. The MoCA test was given early on by trained staff, separate from other tests. The final diagnosis of whether someone had thinking problems or not was made by a team of experts using standard rules, and the MoCA test score was not used to make that final decision. The study then looked at how well the MoCA test results matched the expert diagnoses.

Results

About 700 patients from the clinic and 84 healthy people were included in the study. As expected, patients with memory problems were older. The MoCA scores were clearly different between the groups: healthy people scored highest, and those with memory loss scored lowest. The MoCA test seemed to work very well when comparing very healthy people to those with memory loss. However, when the MoCA was used with typical clinic patients (who might have various mental health issues affecting their scores), it did not work as well at pointing out those with mild thinking problems. This means the test was good at finding people who did have a problem, but it also often suggested a problem for people who did not have one, especially at common cutoff scores.

Discussion

This study showed that using the MoCA test with healthy people makes it seem better than it is for real clinic patients. In a clinic setting, factors like a patient’s education level or other mental health conditions can make their MoCA score look low even if they do not have memory loss. This can lead to "false positives," where the test says there is a problem when there is not one. The MoCA test is good for figuring out if someone does not have a major thinking problem if they score high enough. It is also good for finding people who might need more detailed memory tests. However, it is not precise enough to give a final diagnosis of mild thinking problems or dementia by itself. More research is needed to make the test better at finding only those who truly need more checks.

Conclusion

This study shows that it is important to test the MoCA in real patient groups, not just with very healthy people, because results can seem better than they are. The MoCA test can be helpful in clinics for older adults with mental health issues. It can confirm when someone's thinking is normal or help decide if someone needs more advanced tests for memory problems. However, more work is needed to reduce how many healthy people the test wrongly points out as having a problem.