Article Text
PDF
Abstract
Objective Dispersion, or variability in an individual’s performance across multiple tasks at a single assessment visit, has been associated with cognitive dysfunction (CD) in many neurodegenerative and neurodevelopmental disorders. We aimed to compute a dispersion score using neuropsychological battery (NB) tests and determine its association with CD in patients with SLE.
Methods CD was defined as a z-score of ≤−1.5 on ≥2 domains of the NB. To compute a type of dispersion score known as the intraindividual SD (ISD), the SD of age-adjusted and sex-adjusted z-scores was calculated for each visit in each patient. To estimate the association between ISD and cognitive status (CD and non-CD), we used multilevel logistic regression, adjusting for clinically important covariates.
Results A total of 301 adult patients with SLE completed the NB at baseline, 187 of whom were reassessed at 6 months and 189 at 12 months. CD was observed in 35.2% of patients at baseline, 27.8% at 6 months and 28.0% at 12 months. Prior to covariate adjustment, the mean ISD for non-CD was 1.10±0.31 compared with 1.50±0.70 for CD. After adjusting for ethnicity, education, employment, socioeconomic status and anxiety/depression, there was a statistically significant association between ISD and CD (OR for one-unit increase in ISD: 13.56, 95% CI 4.80 to 38.31; OR for 1/10th-unit increase in ISD: 1.30, 95% CI 1.17 to 1.44). Findings were valid across multiple sensitivity analyses.
Conclusion This is the first study to show that patients with SLE who were classified as having CD by the NB had more variability across the NB tests (ie, higher ISD score) compared with those who were not classified as having CD.
- systemic lupus erythematosus
- outcome assessment
- health care
- quality of life
- autoimmune diseases
- psychology
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- systemic lupus erythematosus
- outcome assessment
- health care
- quality of life
- autoimmune diseases
- psychology
Key messages
What is already known about this subject?
The assessment of cognitive dysfunction (CD) in patients with SLE typically involves obtaining standardised neuropsychological battery (NB) scores and classifying patients based on the number of scores that reach a specific threshold in a given number of domains.
What does this study add?
This study is the first to compute a dispersion score that captures an individual’s across-task performance variability in cognitive tests for patients with SLE.
A robust, statistically significant association exists between an increased dispersion score and the risk of CD in patients with SLE.
How might this impact on clinical practice or future developments?
The dispersion score gives us insight into the complex neurological processes that underlie cognitive functioning and ultimately a more complete understanding of CD in patients with SLE.
The dispersion score represents a novel and pragmatic way to interpret the scores of the NB, thereby providing additional information in screening/diagnostic procedures for CD.
Introduction
SLE is a chronic autoimmune disease characterised by multiple organ system involvement. 1 The American College of Rheumatology (ACR) has classified the effects of SLE on the nervous system as 19 neuropsychiatric SLE syndromes, among which cognitive dysfunction (CD) is one of the most common (prevalence: 38%; 95% CI 33% to 43%). 2 According to the ACR, CD is defined as ‘significant deficient functioning in at least one of the following cognitive domains: simple or complex attention, learning and memory, visuospatial processing, psychomotor speed, verbal fluency, reasoning ability, problem solving, and executive processes of planning, organization, and sequencing’. 3–6 These deficits have considerable effects on patients’ lives as they impact performance on daily tasks, role participation, employment status, as well as mental and emotional health. 7–9
The assessment of CD is typically carried out using the ACR’s validated neuropsychological battery (NB). This screening/diagnostic tool comprises 19 tests and is representative of 6 broad cognitive domains: manual motor speed, simple attention and processing speed, visual-spatial construction, language processing, learning and memory, and executive functioning. CD status is determined by the conversion of test scores into age-adjusted and sex adjusted z-scores, indicating how many SDs separate a patient from the average score in a given number of domains. 3 6 7 Although commonly used as a method for CD classification, there are novel approaches to analysing and interpreting rich test score data, such as that provided by NB tests, that have not yet been explored.
Intraindividual variability (IIV) has long been assessed as a metric of an individual’s functioning across behavioural, physiological and neuropsychological domains. 10–12 Inconsistency is one type of IIV, operationalised as the variability of an individual’s performance on a single task across multiple assessment visits. Dispersion is another type of IIV, operationalised as the variability in an individual’s performance across multiple tasks at one assessment visit. 13 Increased dispersion across cognitive tasks has been linked with attention-deficit/hyperactivity disorder (ADHD),14 postconcussive CD,10 cognitive decline in adults with dementia15 16 and contradictory study results for CD in ageing adults. 11 17 Interest in dispersion in the context of cognition is driven by theoretical assumptions that dispersion reflects compromised neurological mechanisms that may be attributed to disrupted neural networks, altered functional connectivity and executive dysfunction or impaired cognitive control. 17–19 Moreover, the ability to measure dispersion in one clinical assessment visit confers practical advantages over the measurement of inconsistency, which would require more than one assessment visit to obtain multiple measurements of the same test score.
Dispersion as an index of variability and its association with CD has not yet attracted wide attention in SLE research. Exploring dispersion in this context provides us with a more complete understanding of cognitive function as it serves as a new and pragmatic way to interpret the valid and reliable NB. The aim of our study was to establish a novel use for the dispersion score applied to the NB to increase its interpretability for patients living with SLE. Our objectives were to (1) compute a measure of dispersion, operationalised as the IIV of performance across tests of the NB; and (2) determine the association between this dispersion score and cognitive status (non-CD and CD).
Materials and methods
Study design
This cross-sectional study used prospectively collected longitudinal data on patients who were enrolled in the Toronto Lupus Clinic-Cognitive Study at the University of Toronto and assessed at baseline, 6 months and 12 months.7
Participants/Setting
Three hundred and one patients who attended the University of Toronto Lupus Clinic between January 2016 and October 2019 participated in the study. Inclusion criteria were (1) fulfilment of the revised ACR criteria for SLE classification or three criteria and a supportive renal biopsy20, (2) age between 18 and 65 years and (3) ability to provide informed consent. Exclusion criteria were (1) mental or physical disability preventing participation in the study and (2) lack of fluency in English, precluding completion of verbal items of the NB. Demographic information collected from participants included sex, age at enrolment, age at SLE diagnosis, disease duration at enrolment, ethnicity, highest education level achieved, employment status, postal code (a socioeconomic status proxy) and marital status.
Procedures and outcome measures
Clinical measures including disease activity and the presence of anxiety or depression were measured at each assessment visit. Disease activity was assessed using the SLE Disease Activity Index 2000 Glucocorticoid Index (SLEDAI-2KG).21 22 The SLEDAI-2KG is a modified version of the SLEDAI-2K index23 that accounts for glucocorticoid dose, allowing for more accurate assessments of the severity of activity within a descriptor of the SLEDAI-2K. Anxiety was measured using the Beck Anxiety Inventory, a 21-item patient-reported questionnaire that measures the severity of anxiety symptoms. Depression was measured using the Beck Depression Inventory-II, a 21-item patient-reported questionnaire that measures the severity of depressive symptoms. Individuals with a depression or anxiety score of ≥18 were defined as having depression or anxiety, respectively. This cut-off was deemed clinically meaningful for patients with SLE.24 25
At each assessment visit, patients were administered the NB, which consists of 19 cognitive tests representing 6 cognitive domains (ie, manual motor speed, simple attention and processing speed, visual-spatial construction, language processing, learning and memory [visuospatial and verbal], and executive functioning [untimed and timed]) (online supplemental appendix A). Our NB was identical to the ACR’s recommended cognitive battery,26 with the exception of the following tests: the Hopkins Verbal Learning Test-Revised27 replaced the California Verbal Learning Test, and the Trail Making Test A28 29 was added. Patients’ scores on the NB were transformed into age-adjusted and sex-adjusted z-scores, allowing for comparisons of the tests on the same scale. Cognitive status was also assessed at each visit based on the following classification criteria: (1) CD: a z-score of ≤−1.5 in two or more domains and (2) non-CD: a z-score of ≤−1.5 in one or fewer domains. A domain was defined as impaired if a z-score of ≤−1.5 was reached in at least one test in the following domains: manual motor speed, simple attention and processing speed, visual-spatial construction, and language processing or a z-score of ≤−1.5 in two or more tests in the following domains: learning and memory and executive functioning. 7
Supplemental material
Statistical analysis
Patient characteristics
Baseline patient demographics and characteristics along with the scores of different tests were expressed as mean±SD and median (IQR) for continuous variables, and total number of cases (proportion) for categorical variables, all stratified by cognitive status (CD and non-CD). To compare groups (CD and non-CD), absolute standardised differences were calculated and expressed as percentages, with a standardised difference of <10% denoting unimportant differences between groups.30
Missing data
We elected to omit the two tests from the manual motor speed domain (ie, finger tapping test: dominant hand and finger tapping test: non-dominant hand) from our primary analyses. Missing data in this domain were assumed to be missing not at random, such that missingness was related to the observation value itself (ie, missing values occurred in these tests because of patients’ inability to complete the tasks and if they were non-missing, they would have likely resulted in lower scores). The missing data from the remaining 17 NB tests were assumed to be missing at random due to challenges in the test administration. To address this missing data across the three visits, we performed multivariate imputation by chained equation (MICE). 31 A total of 10 imputations were carried out using the mi impute chained package in Stata V.16. 32
Computing the dispersion score
Dispersion was computed as the intraindividual SD (ISD) score, which collapses a patient’s performance across multiple tests at a given assessment visit into a single score. To obtain the ISD score, the SD of the age-adjusted and sex-adjusted z-scores from the included 17 tests was calculated for each assessment visit (ie, baseline, 6 months and 12 months),13 resulting in a maximum of three separate scores per patient. Each score contributed equally to the overall ISD score. A higher ISD score indicates greater variability across cognitive tests, whereas an ISD score closer to 0 reflects more homogeneity in these tests, regardless of performance on the individual test used to compute the ISD score.
Association between ISD score and cognitive status
The association between ISD score and the patient’s cognitive status (ie, CD and non-CD) was estimated using a mixed-effects model to account for patterns of correlation in ISD scores between multiple assessment visits observed in one patient. The dependent variable was cognitive status and the primary independent variable was ISD score. The following covariates were selected a priori for inclusion in the model based on their theoretical influence on neuropsychological performance: ethnicity (ie, black, white, Chinese, other), highest education level achieved (ie, eighth grade, high school, college, university), employment status (ie, employed, retired, home maker, student, disabled, sick leave, looking for work, other), a socioeconomic status proxy (Neighbourhood Income Quintile Before Tax; ie, lowest, medium-low, middle, medium-high, highest) which was translated from postal codes using the Statistics Canada Postal Code Conversion File Plus V.7C33 and the presence of anxiety or depression. Both age and sex were accounted for in the generation of the z-score. Relative goodness of fit of all multivariable models was assessed with Akaike information criterion and Bayesian information criterion values.
Sensitivity analyses
The following sensitivity analyses were carried out to re-examine the association between ISD score and cognitive status: (1) omitting high leverage points, defined as ISD scores above the 98th percentile, (2) complete case analysis with 17 tests, in which patient assessments with any missing test score values were completely excluded from analysis, (3) complete case analysis with the original 19 tests, including the manual motor speed domain, (4) redefining levels of the dependent variable by restricting non-CD to z-scores >−1.5 in all domains, thereby excluding patient assessments with a z-score of ≤−1.5 in only one domain (mild CD), and (5) including SLEDAI-2KG as a covariate in the model.
A two-tailed p<0.05 was considered statistically significant. Analyses were conducted with Stata V.16.
Results
Patients
A total of 849 patients with SLE were screened, 786 of whom were eligible for participation and 415 of whom provided informed consent. Of the 415 consenting patients, 38 withdrew from the study due to reasons such as being too busy/unable to dedicate time to the study (n=16), no longer wanting to participate (n=16), perceiving the study to be too long (n=5) and other reasons (n=1), and 76 provided consent but we have yet to collect their data (online supplemental appendix B). Baseline information was collected from 301 patients, 187 of whom were reassessed at 6-month follow-up and 189 at 12-month follow-up. In total, 149 patients completed all three assessments, 78 patients completed only two assessments and 74 patients completed only one assessment.
Supplemental material
Missing data occurred in all 17 test variables, with the highest proportion occurring in the Trail Making Test B task (8%). Because the data were missing due to random issues surrounding the administration of the test and unrelated to the observations, there is no cause for statistical concern. Missing data also occurred in the following clinical and demographic variables: presence of anxiety or depression (19.2%), highest education level achieved (1.3%) and socioeconomic status (<1%).
On completion of the NB across a total of 677 assessment visits, 211 patient assessments (31.2%) resulted in a CD classification (106 patients (35.2%) at baseline, 52 patients (27.8%) at 6 months and 53 (28.0%) at 12 months). Important baseline differences between patients without CD and patients with CD were noted on several characteristics (table 1).
Baseline characteristics of patients
Variability of z-scores
Prior to the computation of the ISD score, the variability of age-adjusted and sex-adjusted z-scores across cognitive tests was depicted for patients with and without CD (figure 1). The Rey Complex Figure Test Copy and the Trail Making Test B tasks demonstrated the greatest variability among all tests for both groups of cognitive status and were likely more influential on the magnitude of the ISD scores, compared with the other tests. The means and 95% CIs of z-scores for each test can be found in online supplemental appendix C.
Supplemental material
(A) The z-score variability across neuropsychological battery tests in patients without CD. (B) The z-score variability across neuropsychological battery tests in patients with CD. Each coloured line represents one assessment visit. The white line represents the mean z-score variability. CD, cognitive dysfunction; COWAT, Controlled Oral Word Association Test; HVLT-R, Hopkins Verbal Learning Test-Revised; RCFT, Rey Complex Figure Test; WAIS, Wechsler Adult Intelligence Scale.
ISD score
Among all observations across the three assessment visits, the average age-adjusted and sex-adjusted ISD score was 1.40 (SD=0.55). The minimum score was 0.35 and the maximum score was 5.28, with 98% of observations falling below 2.93. Prior to adjustment for ethnicity, highest education level achieved, employment status, socioeconomic status and the presence of anxiety or depression, the mean ISD score for the non-CD group was 1.10 compared with an ISD score of 1.50 for the CD group. The difference between the two groups was statistically significant (p<0.0001). ISD scores were stratified by cognitive status and by assessment visit in figure 2. On visual inspection, there appeared to be a consistently lower ISD score associated with the non-CD group across all three assessment visits.
ISD scores across three assessment visits for patients without and with CD. CD, cognitive dysfunction; ISD, intraindividual SD.
Association between ISD and cognitive status
Table 2 shows the results of the mixed-effects logistic regression model following MICE. After adjustment for ethnicity, highest education level achieved, employment status, socioeconomic status and the presence of anxiety or depression, there was a statistically significant association between ISD score and cognitive status (OR: 13.56; 95% CI 4.80 to 38.31; p<0.0001). For every one-unit increase in ISD score, there was nearly 14-fold increased odds of being classified as having CD. Since the range for the ISD score was only approximately five units, the ISD score variable was rescaled to estimate the OR of being classified as having CD for every 1/10th-unit increase in ISD (OR: 1.30; 95% CI 1.17 to 1.44; p<0.0001). Our model also indicated that the odds of being classified as having CD were statistically significantly lower for white (OR: 0.08; 95% CI 0.02 to 0.28; p<0.0001), Chinese (OR: 0.16; 95% CI 0.03 to 0.89; p<0.04) and other ethnicities (OR: 0.22; CI 0.05 to 0.96; p<0.04) compared with black individuals.
Multivariable random intercept logistic regression model
Sensitivity analyses
Sensitivity analyses are reported in table 3. Although point estimates and 95% CIs varied in size across the five models, the primary independent variable, ISD score, remained statistically significant. Across all analyses, there was a statistically significant association between being white and a lower odds of being classified as having CD, compared with being black. In model 4, the exclusion of the mild CD group resulted in a considerably larger effect size for the ISD variable, compared with our original analysis which included patients with mild CD in the non-CD group.
Sensitivity analyses examining the association between ISD score and cognitive status
Discussion
In this longitudinal study based in Toronto, Canada, we investigated the construct of cognitive dispersion, defined as the IIV across neuropsychological tests, for patients living with SLE. After adjusting for clinically important covariates, higher dispersion (ie, ISD) was associated with higher odds of CD.
Extant literature has explored the potential utility of dispersion as a sensitive marker of neural integrity in neuropathological patient populations. In the context of Alzheimer’s disease (AD), greater dispersion was found to be associated with an increased likelihood of being classified with AD (OR for 1/10th ISD: 1.20; 95% CI 1.04 to 1.38)15 and the development of incident Alzheimer’s dementia (HR for 1 ISD: 3.63; 95% CI 1.70 to 7.37). 16 Similarly, studies surrounding traumatic brain injury demonstrated that greater dispersion was associated with decreased global neurocognitive ability. 10 34 35 Lastly, some literature has focused on dispersion in neurodevelopmental conditions such as ADHD and autism spectrum disorder (ASD). Gonzalez-Gadea and colleagues14 showed that adults with ADHD and adults with ASD shared similarities in their cognitive profiles, displaying higher variability across executive functioning tasks compared with healthy controls. Our study was the first to examine the construct of dispersion in the context of cognitive dysfunction in patients with SLE. Consistent with previous research on IIV in other diseases, increased dispersion in SLE was observed to be associated with CD. These novel and important insights provided further evidence that IIV across cognitive tasks may reflect complex neurological processes that underlie cognitive functioning. This information may be valuable to patients as it would offer a more conceptually intuitive interpretation of their NB test scores. Instead of relying solely on the magnitude of age-adjusted and sex-adjusted z-scores, clinicians could explain that the degree of variability in the patient’s performance across NB tests helped to inform their CD diagnosis. Thus, our study represented an important step in obtaining a more complete understanding of CD in SLE for clinicians, researchers and patients.
Our main findings were robust to analytic technique, including the exclusion of extreme ISD values, complete case analysis using both the chosen 17 tests (omitting the manual motor speed domain) and the original 19 NB tests (including manual motor speed domain), the redefining of cognitive status categories to exclude mild CD (defined as a z-score of ≤−1.5 in only one domain), and the inclusion of the SLEDAI-2KG as a covariate. Across our sensitivity analyses, the results also provided evidence for a statistically significant association between being white, Chinese and of other ethnicities and a lower odds of having CD as compared with being black. It is unclear whether in this association there exist mediating factors such as cultural elements, structural processes and the quality of received healthcare that were not accounted for; therefore, we must cautiously interpret these findings. Additionally, these sensitivity analyses shed light on the possibility of three distinct levels of CD: unimpaired, mildly impaired and highly impaired. This was supported by the material increase in the effect size for the association between ISD score and CD following the reclassification of cognitive status into three levels and the omission of patients within the middle level (mild CD). Previous studies have acknowledged the presence of a heterogeneous, indeterminate CD group.7 36 This additional level within CD classification may be useful in guiding more appropriate statistical modelling for future SLE research.
Several limitations should be considered. First, our study used the same NB for the generation of the ISD score and the classification of cognitive status, since it is the gold standard for the assessment of cognitive function. Nevertheless, studies are needed to evaluate the association between ISD and CD according to an independent external measure such as a neuropsychological assessment by a psychologist or the use of a different NB. Second, our study was conducted at a single centre in the largest urban centre in Canada. Third, we only included patients with sufficient English ability for the completion of the NB. Consequently, generalisability of our findings to the broader Canadian SLE population may be reduced. Fourth, we did not use data from the two tests of domain 1 (manual motor speed) in the computation of the ISD as we reasoned that the high proportion and non-randomness of the missing data could lead to biased estimates. However, our sensitivity analysis suggests that the overall message concerning our primary independent variable (dispersion) would remain the same with the inclusion of the two tests. Lastly, despite our efforts in reducing the effects of known confounders, there will be residual confounding from unknown or unmeasured confounders due to the observational nature of the study.
Future studies may consider validating this measure of dispersion using other screening tools such as the Automated Neuropsychological Assessment Metrics,7 which could support a more cost-effective and practical way of screening for CD in patients with SLE. Furthermore, conducting longitudinal analyses to examine the long-term predictive ability of the dispersion score could improve the early detection of CD.
Conclusion
In this study, we computed a measure of across-task IIV and ascertained its association with cognitive status. Among adult patients with SLE, those who were classified as having CD by the NB had higher variability across NB tests (ie, higher ISD score) compared with those who did not have CD, after adjusting for clinically important covariates. Additional research is warranted to validate the use of dispersion in other screening and diagnostic tools, and evaluate the promise of dispersion in clinical practice.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Ethics statements
Patient consent for publication
Ethics approval
This project was approved by the University Health Network Research Ethics Board (CAPCR ID: 20-5573) and informed consent for entry into the study was provided by patients.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @patti_katz
Contributors All authors contributed to the drafting or revising of the article for important intellectual content. All authors approved the final version of the article to be published. Study conception and design: JWH, JPDM, KB, JS, MK, MCT, LR, DB, JW, MYC, MJF, NA, DB, RG, PK, ZT. Acquisition of data: ZT. Analysis and interpretation of data: JWH, JPDM, KB, JS, MK, MCT, LR, DB, JW, MYC, MJF, NA, DB, RG, PK, ZT.
Funding The Lupus Foundation of America provided financial support to JWH through the Gina M Finzi Fellowship programme grant. The SLE Cognitive Impairment programme has been supported by grants from the Arthritis Society of Canada, Physician’s Services Incorporated, the Canadian Institutes of Health Research, Lupus Research Alliance and the Province of Ontario Ministry of Economic Development, Job Creation and Trade. ZT holds a salary support award from the Department of Medicine, University of Toronto.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.