Discussion
This is the first study to examine the QRISK2, QRISK3, SLECRE, Framingham and modified Framingham on a large SLE cohort linked to CVD outcomes. This study was a single-centre analysis of prospectively collected data of 1887 patients with SLE, 232 of whom had a CVD event. The 12.3% prevalence of CVD and the incidence of 8.3 per 1000 person-years in our cohort match the literature.12 13 In this study, we determined which of the five CVD risk tools most accurately predicts CVD in patients with SLE. Our results showed that CVD risk tools considering a diagnosis of SLE did calculate relatively higher risk scores than those that did not; however, this did not result in a sufficient improvement in predictive accuracy for CVD development.
The results of the AUC confirmed that these tools are not very different in their ability to predict CVD in patients with SLE. The FRS, mFRS and QRISK2 had the greatest c-statistics; however, when examining sensitivity and specificity, there are differences between each of these tools. These three risk tools all had high specificities, but low sensitivities, with the mFRS demonstrating the greatest sensitivity of 46.1%, with a modest compromise in specificity, compared with the other tools. The QRISK2 and FRS have such high specificity (93%) because they categorised the greatest proportion of patients as low risk, minimising the number of false positives identified for high CVD risk. With that came poor sensitivity, as there were many false negatives identified. The SLECRE had the greatest sensitivity (61%), but also had the lowest specificity (64%). The FRS and mFRS had identical c-statistics.
Different CVD risk tools have been examined in patients with SLE in prior studies24 where the FRS and SLECRE were compared, and the SLECRE was found to be superior to the FRS and consistent with the mFRS; however, this study only looked at the recategorisation of patients from low risk (≤20% 10-year CVD risk) to high risk (>20% 10-year CVD risk) according to the different tools. These data were not linked to CVD outcomes and so do not demonstrate the predictive ability of these tools. Additionally, as increasing numbers of prediction tools are being developed, there are calls for more studies that validate these prediction models.25 Furthermore, the lower c-statistics and sensitivities demonstrated in this study compared with the literature evaluating these tools on the general population demonstrate the need to further refine CVD risk assessment tools for SLE populations.25
We have shown disagreement among these tools. The kappa statistics demonstrate this, with the QRISK2 and FRS having the greatest agreement of any pair of tools, yet only having kappa statistics of 0.65, signifying a ‘moderate’ to ‘substantial’ level of agreement.19 20 Meanwhile, the QRISK2 and SLECRE had an agreement of 0.20, representing slight to no agreement. Thus, these disagreements can have direct implications on preventive treatment and provide insight into risk management by physicians treating patients with SLE. For example, in the general population, a wide range of patients with FRS ≥10% will be eligible for statin therapy to target a low-density lipoprotein cholesterol <2.0 mmol/L or >50% reduction.11 If there can be so much restratification of CVD risk according to these different tools, this can be representative of a physician’s appraisal of their patients’ CVD risk. If one physician applies the FRS to determine their patients’ CVD risk, they may under-rate their risk compared with another physician who applies the SLECRE. These different approaches could lead to different management to the benefit or detriment of the patient.
There are some limitations to our study. Data on both the Townsend deprivation index and family history of angina or heart attack in a first-degree relative younger than 60 were missing in calculating the QRISK3 score. The weighting of the Townsend score in the QRISK3 score is only 0.077 when calculating the HR to determine the 10-year CVD risk, which is relatively inconsequential, especially compared with a weighting of 1.72 for type 1 diabetes or 0.759 for SLE.13 The Townsend deprivation index is a measure of material deprivation specific to the UK population, calculated using four census variables for a geographical area.16 This metric is used as a marker of socioeconomic status (SES), where an association between low SES and increased CVD has been clearly demonstrated.26 Local metrics of SES such as the 2011 Ontario Marginalization Index or the 2006 Canadian Marginalization Index were not used as substitutes for the Townsend score as they are not directly comparable. Data on family history of CVD were only captured regularly in our protocol after 1999. A subset analysis was performed on 642 patients with family history data collected, comparing their QRISK3 scores with and without this variable, and in 121 CVD patients the mean difference in score was 0.87% and in 521 no CVD patients the mean difference in score was 1.24%. The weighted score of this parameter is 0.454 in calculating the QRISK3 HR to determine 10-year CVD risk, and its exclusion resulted in slightly under-rated CVD risks. Additionally, we did not have data collected on severe mental illness diagnoses; however, we used SF-36 data that were available on 360 patients to determine severe mental illness status, as done in other studies.21 For the missing data on 480 patients, imputation was performed. This could result in patients being randomly assigned to having severe mental illness. However, the relative weighting of severe mental illness in the QRISK3 is minimal compared with other components of this risk tool, resulting in a minimal change in risk score. Lastly, we used the Framingham definition in order to define CVD in our cohort selection. In the derivation and validation of the mFRS, coronary artery disease was examined, resulting in only angina, MI, CHF, CVD death and pacemaker insertion related to CAD to identify CVD.18 Stroke, MI, angina/coronary artery bypass graft and claudication were the only factors used to identify CVD in deriving the SLECRE. Finally, QRISK2 and QRISK3 did not include PAD or heart failure secondary to atherosclerosis in deriving their algorithms but included all of the other FRS CVD events in their definition. We aligned our definition of CVD with the FRS 20088 because it is the most established CVD risk tool available and it was inclusive of all CVD outcomes, thus creating the largest possible unified outcome for comparing tools’ performances and agreements. This comes with the caveat of including CVD outcomes in our analysis that were not used to derive the original risk tools. However, as seen above, each tool uses a different definition of CVD, necessitating a common definition that may not exactly match each individual tool. A sensitivity analysis was performed examining the predictive ability of the mFRS with angina, MI, CVD death excluding TIA, stroke and PAD, and there were no significant differences in sensitivity, specificity or c-statistics when compared with the original analysis, which included all events according to FRS 2008.8 Accordingly, we believe that although different CVD definitions were used by each tool, using the FRS 2008 definition4 of CVD would not have substantively affected the results. All tools’ c-statistics marginally improved under this sensitivity analysis examining patient risk scores at the date of enrolment in the Toronto Lupus Clinic, except for the SLECRE. We are aware of the 2013 American College of Cardiology/American Heart Association CVD risk calculator; however, the calculator’s age cut-off of 40–79 would have attenuated the cohort for analysis.27