Article Text
PDF
Abstract
Objective To develop an improved score for prediction of severe infection in patients with systemic lupus erythematosus (SLE), namely, the SLE Severe Infection Score-Revised (SLESIS-R) and to validate it in a large multicentre lupus cohort.
Methods We used data from the prospective phase of RELESSER (RELESSER-PROS), the SLE register of the Spanish Society of Rheumatology. A multivariable logistic model was constructed taking into account the variables already forming the SLESIS score, plus all other potential predictors identified in a literature review. Performance was analysed using the C-statistic and the area under the receiver operating characteristic curve (AUROC). Internal validation was carried out using a 100-sample bootstrapping procedure. ORs were transformed into score items, and the AUROC was used to determine performance.
Results A total of 1459 patients who had completed 1 year of follow-up were included in the development cohort (mean age, 49±13 years; 90% women). Twenty-five (1.7%) had experienced ≥1 severe infection. According to the adjusted multivariate model, severe infection could be predicted from four variables: age (years) ≥60, previous SLE-related hospitalisation, previous serious infection and glucocorticoid dose. A score was built from the best model, taking values from 0 to 17. The AUROC was 0.861 (0.777–0.946). The cut-off chosen was ≥6, which exhibited an accuracy of 85.9% and a positive likelihood ratio of 5.48.
Conclusions SLESIS-R is an accurate and feasible instrument for predicting infections in patients with SLE. SLESIS-R could help to make informed decisions on the use of immunosuppressants and the implementation of preventive measures.
- lupus erythematosus, systemic
- epidemiology
- risk factors
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Severe infection is frequent in patients with systemic lupus erythematosus, and, while several risk factors have been identified, no clinically useful risk score has been developed to date.
WHAT THIS STUDY ADDS
The authors developed and internally validated an accurate and feasible score for the prediction of serious infection in clinical practice.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The score could help clinicians to make informed decisions on the use of immunosuppressants and the implementation of preventive measures.
Introduction
Patients with systemic lupus erythematosus (SLE) are at increased risk of severe infections that vary with the severity of the disease, use of immunosuppressants (including glucocorticoids), comorbidities and organ damage.1–7 Moreover, infection remains a leading contributor to mortality in patients with SLE.5 8–10
Properly estimating the risk of infection in patients with SLE is paramount if we are to balance immunosuppression and implement preventive measures. Unfortunately, very few predictive models of severe infection in patients with SLE have been published to date. One systematic literature review showed that most of those published were from retrospective cohorts and were subjected to methodological limitations and a high risk of bias.11 No evidence-based, widely validated, and suitable score for predicting severe infection in patients with SLE has been developed for use in daily clinical practice. Conversely, scores for predicting major infection have been successfully developed for other systemic immune-mediated rheumatic diseases, such as rheumatoid arthritis.12
Our group attempted to develop a tool for the prediction of severe infections in SLE. The SLE Severe Infection Score (SLESIS) was developed using data gathered from the retrospective cross-sectional phase of the Spanish Rheumatology Society Systemic Lupus Erythematosus Registry (RELESSER-TRANS)13 and validated in an external cohort, the University College London Hospital SLE cohort, which was also based on retrospective-longitudinal data. The original SLESIS incorporated seven predictors, including the Katz severity index (KSI).14 However, the performance of SLESIS was only moderate, with an area under the receiver operating characteristic curve (AUROC) of 0.63 (95% CI 0.56 to 0.70) at diagnosis and of 0.79 (95% CI 0.73 to 0.85) at the time of infection.
In the current study, we aimed to improve the ability of SLESIS to predict the risk of infection by reformulating the constituent variables and adding new markers, if appropriate, based on higher quality data from the prospective phase of the RELESSER register (namely, RELESSER-PROS). We also wished to improve the feasibility of our index by avoiding, if possible, inclusion of the KSI, which is cumbersome to calculate and has a limited degree of validation. Furthermore, we performed an internal validation of the resulting index.
Patients and methods
Design and participants
The data for this study were gathered from the RELESSER-PROS register, a multicentre prospective cohort of patients with SLE involving 39 Spanish hospitals. The RELESSER cohort comprises patients who meet ≥4 American College of Rheumatology (ACR) classification criteria for SLE, are under active follow-up, and have been recruited from the cross-sectional stage of the register (RELESSER-TRANS). Only patients with sufficient information regarding serious infection were included in the analysis. The general characteristics of the RELESSER register have been reported elsewhere.15 The baseline visits of RELESSER-PROS took place between 2014 and 2023, and the patients are under active yearly follow-up.
Data collection and variable definitions
Potential predictors were extracted from data collected at the baseline visit (visit 1) and comprise demographic data and clinical characteristics, disease activity (Safety of Estrogens in Lupus Erythematosus National Assessment–SLE Activity Index (SELENA-SLEDAI)) at baseline, severity (KSI), organ damage (Systemic Lupus International Collaborating Clinics (SLICC)-ACR/Damage Index) (SDI), comorbidities (Charlson comorbidity index), previous hospital admission for SLE, previous serious infections (any time after diagnosis of SLE), laboratory data (serum creatinine, lymphopenia <1000/mm3, hypocomplementemia) and treatments received (antimalarials, immunosuppressants, rituximab, glucocorticoids and prednisone dose (or equivalent) (ie, dose at visit 1, and maximum prednisone dose during the observation period). In order to avoid overfitting of the model, which would have led to performance overestimation, an effort was made to reduce the number of candidate predictors based on our previous studies and a thorough review of the literature.
The dependent variable was the occurrence of a serious infection (ie, one leading to hospitalisation or death) during the first year of follow-up.
Statistical analysis
Descriptive data were expressed as measures of central tendency and dispersion in the case of quantitative variables and as frequency tables and percentages in the case of qualitative variables. A total of 362 out of 1821 patients were excluded owing to missing data regarding serious infection. A bivariate analysis comparing included patients (‘valid case’) and excluded patients was carried out. Although the excluded group was characterised by a higher Charlson index, lower percentage of antimalarials and more frequent use of mycophenolate, the percentages for previous infection differed significantly between the groups (99 (5.5%) vs 11 (3.1%), p=0.027), with the analysis favouring the ‘valid’ group (ie, more previous infection in the ‘valid’ group) (see online supplemental table 1 for the complete set of results).
Supplemental material
A baseline comparison of patients in terms of severe infection during the first year of follow-up was performed using the t test or the Mann-Whitney test (continuous data) and the χ2 test with a Fisher exact test (categorical data).
Bivariate logistic regression was used to analyse the predictive effect of baseline variables on the development of severe infection in the first year of follow-up. A predictive model was built based on multivariate logistic regression models and included all the predictors reaching a p value <0.25 in the bivariate analysis (saturated model), with successive elimination of variables without discriminatory power. When multiple options were available for adjustment (eg, adjust for proportion with any glucocorticoid or proportion with a glucocorticoid dosage threshold), we based our decisions on exploratory regression analyses. The most parsimonious model with the lowest Akaike and Bayesian information criteria (AIC and BIC) values was chosen as the final model. The performance of the final model was evaluated based on discrimination and calibration parameters.
In order to seek a more realistic estimate of performance, the model was internally validated using bootstrapping techniques, which were based on all the data used in the development of the model and enabled more robust equations to be obtained. The Transparent Reporting of multivariable prediction model for Individual Prognosis of Diagnosis16 statement was followed for this publication (see online supplemental material).
Each predictor in the final adjusted model was transformed into a specific score item based on its corresponding logistic regression coefficient. The OR of each predictor was rounded up to the nearest integer for simplification. The sum of these values yielded the Systemic Lupus Erythematosus Severe Infection Score-Revised (SLESIS-R), whose performance was calculated using the AUROC. Finally, the cut-off point with the best validity parameters (sensitivity, specificity, likelihood ratio) was chosen.
The analysis was performed using STATA V.18 (STATA V.2023, Stata Statistical Software, Release V.18.0. College Station, Texas: Stata Corp LLC).
Results
A total of 1459 patients who had completed visit 2 (1 year of follow-up) or had had infections or died during the study period were included in the analysis. The mean (±SD) age was 49±13 years, 90% of patients were women and 94% were Caucasian. The mean disease duration was 14.2±8.8 years.
The clinical characteristics, laboratory findings, comorbidities and treatments are shown in table 1. At baseline, the mean SLEDAI was low (2.7±3.8).
Sample description and group comparisons
The frequency of cancer and diabetes was low in both cases and controls (table 1).
Up to 6% had had a prior major infection, that is, before entering the study, and 25% had been hospitalised with SLE. Twenty-five (1.7%) had experienced at least one serious infection in the first year of follow-up. A total of 13 patients (0.91%) died, 2 due to serious infection. Nine patients (0.49%) were admitted to the ICU; in 2 cases, admission was because of infection.
The results of the univariate analysis are shown in table 2. Patients with infection were older (OR=1.04; p=0.006), with more damage accrual (OR=1.29; p<0.0001) and comorbidity (OR=1.34; p<0.0001), including a higher frequency of chronic kidney disease (OR=3.82; p=0.004). The predictors that most increased the probability of serious infection in the following year were previous serious infection (OR=14.78; p<0.0001), previous hospitalisation (OR=15.50; p<0.0001) and cyclophosphamide (OR=12.38; p<0.002) or glucocorticoid dose ≥30 mg/day (OR=7.47; p=0.004) (table 2).
Univariate analysis
Predictive model building
The potential predictors with p≤0.25 that were entered into the multivariate logistic regression model were age, Charlson comorbidity index, chronic kidney disease, SDI, KSI, SLE-related hospitalisation, previous serious infection, treatments such as antimalarials, cyclophosphamide, mycophenolate as well as the maximum dose of glucocorticoids used during the observation period of ≥30 mg prednisone/day (or equivalent). In order to simplify construction of the index, we made the variable age dichotomous, namely, <60 years or ≥60 years.
Starting from a saturated model (all predictors with a bivariate p value of p≤0.250), we selectively eliminated variables without discriminatory power. A parallel stepwise procedure revealed no differences with the successive elimination approach. Eventually, the most parsimonious model, that is, that with the lowest AIC and BIC values, was chosen. According to our final adjusted multivariate model, the occurrence of a serious infection in the following year in SLE can be predicted from four variables: age ≥60 years (β=1.80; OR=6.06; p=0.002), previous admission for SLE (β=1.92; OR=6.84; p=0.007), previous infection (β=1.81; OR=6.09; p=0.002) and having received a maximum dose of glucocorticoids ≥30 mg (β=2.19; OR=8.93; p=0.010) (table 3). The KSI was eventually excluded from the model. Our model exhibited adequate performance, with 97.8% correct classification. The discrimination parameters revealed an AUROC of 0.874 (0.777–0.974), with adequate calibration (Hosmer-Lemeshow, p=0.932).
Adjusted final multivariate predictive model
Internal validation
The model was internally validated using a bootstrapping procedure, taking up to 100 samples with replacement and adjustment for overfitting of the model using a heuristic shrinkage factor. The ORs and ß-coefficients of the adjusted model are provided in the online supplemental table 2. This model revealed appropriate discrimination parameters, with a C statistic of 0.810 (0.715–0.893).
The robustness of each predictor, measured as the number of times that it is included in the 100 bootstrap samples, is displayed in the online supplemental table 3.
SLESIS-R index design
Up to 11 mathematical transformations of the final model were performed to create the index; of these, 6 were based on coefficients and 5 on the ORs of the model adjusted for overfitting (online supplemental table 4). This approach yielded 11 possible indices (online supplemental tables 5 and 6). No significant differences were observed between the 11 ROC curves obtained with these indices (online supplemental figure 1). Consequently, the OR-based transformation (avoiding the effect of the constant) with the best performance (higher AUROC), corresponding to number nine and consisting of rounding the OR of each predictor, was finally chosen.
The final SLESIS-R is shown in table 4. The score is based on values ranging from 0 to 17. The ROC curve of the SLESIS-R is displayed in figure 1. The resulting AUROC was 0.861 (0.777–0.946). The validity parameters are displayed in table 5. According to these parameters, a score ≥6 was chosen as the best cut-off point, exhibiting a sensitivity of 76% and specificity of 86.6%, with an accuracy of 85.9% and positive likelihood ratio of 5.48.
SLESIS-R index calculator
ROC curve for the SLESIS-R index. ROC, receiver operating characteristic curve; SLE, systemic lupus erythematosus; SLESIS-R, SLE Severe Infection Score-Revised.
SLESIS-R—validity parameters
Discussion
Based on data from a large, prospective multicentre cohort, we developed and internally validated an improved version of SLESIS, namely SLESIS-R, a score that is able to predict the risk of severe infection in patients with SLE during the following year. The performance of SLESIS-R was very favourable, notably improving on the previous version of the score in terms of the AUROC (0.861 (95% CI 0.777 to 0.946) vs 0.790 (95% CI 0.730 to 0.850)). The SLESIS-R also improved feasibility, given the greater simplicity of the new version and the exclusion of the KSI. This latest version includes only four clinical parameters, namely, age, previous SLE-related hospitalisation, previous severe infection and glucocorticoid dose ≥30 mg/day, all of which are readily available in the patient’s clinical records. The four parameters found are consistent with most previous studies regarding major infection-associated factors, which identify mostly age, glucocorticoid dose and previous serious infection as the best predictors of severe infection in SLE.1 2 4 6 7 11 17 In addition, the prospective nature of the data used to develop SLESIS-R, with a better-defined temporal framework, increases the reliability of the results.
Because of its simplicity and the fact that it is based on clinical parameters and not laboratory results, SLESIS-R could become a useful instrument for predicting infection in both daily clinical practice and observational studies and even in clinical trials in Caucasians. In fact, the use of numerical probabilities is to be preferred not only for decision-making but also in teaching materials and in communication between physicians.18 We think that our score improves prediction of the risk of infection, facilitating an informed decision-making process and supporting more careful implementation of preventive measures. Thus, in the case of a patient with an increased risk of serious infection, namely, a SLESIS-R score >6, this information should be considered when selecting therapy and for overall patient management (ie, taking extreme precautions to avoid serious infections, such as vaccinations, hygiene, smoking cessation, etc). Similarly, seeking early medical care in the case of fever would also be appropriate. Additionally, we should perhaps choose therapies with a reduced risk of infection or opt for more aggressive’ tapering of glucocorticoids.
Several previous studies have attempted to develop predictive models of infection in patients with SLE. All of them are discussed below and were based, in contrast to our study, on retrospective data analysis (with the exception of Torres-Ruiz et al19) and single-centre SLE cohorts.
The first formal attempt to develop a predictive model and rigorously test its performance was that of Yuhara et al,17 which was carried out in an Asian single-centre cohort. In contrast to SLESIS, the model of Yuhara et al was developed for inpatients with SLE. The independent predictors of infection, all of which were available at admission, were decreased serum albumin, increased serum creatinine and prednisolone ≥60 mg/day without methylprednisolone pulse therapy. Internal validation of the model yielded a valuable AUROC for cross-validation (0.846, CI not provided). However, it is difficult to generalise these results to an SLE outpatient population.
Torres-Ruiz et al19 built a predictive score based on prospective clinical data and immunological-laboratory tests, the ‘systemic lupus erythematosus infection predictive index’. The performance of the models, measured as the AUROC, was at most 0.75 (95% CI 0.56 to 0.85). However, a very low number of patients with SLE (ie, a total of 55 cases) were included in that study, and only 26% of the recorded infections were serious. Additionally, several of the immunological tests proposed are not widely available or standardised, thus limiting the feasibility of the index.
Restrepo-Escobar et al20 developed a model for predicting bacterial infection in Latin-American patients with SLE, although, again, this model was limited to nosocomial infections and was, therefore, unable to predict serious infection in outpatients with more stable disease in terms of activity. Furthermore, no score was derived from the data obtained in the analysis.
Finally, Wang et al21 conducted a study to evaluate the risk of major infection in an Asian SLE cohort and developed a prediction model that incorporated the following variables: SLEDAI >10, lymphocyte count <0.8×109 /L and serum creatinine >104 µmol/L. The authors identified patients at low risk of major infection (3%–5%) and patients at high risk of major infection (37%–39%) within the first 4 months in newly diagnosed SLE. Up to 69 infections were recorded in 494 patients (14%) in the first year of the disease, an incidence that is substantially higher than in our cohort. That discrepancy could be explained by ethnic differences (Caucasian vs Asian population) or by selection bias. Moreover, and in contrast to our design, the cohort studied by Wang et al was an inception cohort, with a higher level of baseline activity. This predictive model has not been validated to date.
Our study is subject to a series of limitations. First, the number of major infections was relatively low in the cohort, thus potentially compromising the stability of the models. Moreover, the patients included were predominantly Caucasian. Furthermore, given the low grade of disease activity in the cohort, the risk of infection associated with disease activity could be underestimated. Consequently, a more extensive and external validation process is required in order to test the performance of the SLESIS-R in external cohorts, ideally with a more severely ill patients, a higher number of serious infections and more ethnic diversity.
Conclusions
SLESIS-R is an accurate instrument for predicting serious infections SLE and proved feasible for daily clinical practice.
SLESIS-R is simple and easy to calculate. It could help clinicians to make informed decisions on the use of immunosuppressive or biological therapy in patients with SLE and, therefore, to implement preventive measures.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by Ethics Committee of Las Palmas. CODE: 130082. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We are grateful to the RELESSER group collaborators for their invaluable contributions to register data collection. We also are grateful to the employees of the Spanish Rheumatology Society Research Unit for their commitment and professionalism. Finally, we are also grateful to SER for their review of the English translation of the manuscript.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
X @iruafer@gobiernodecanarias.org
Contributors All the authors have contributed in collecting data and reviewing the manuscript, being able to make intellectual contributions in all phases of research development, that is design, statistical analysis and discussion. Guarantor author: IR-F
Funding The authors declare that financial support for this study was received from the Spanish Foundation of Rheumatology (2021 research grant to the main author). The RELESSER-PROS register received financial support from GSK.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer-reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.