Article Text
PDF
Abstract
Objective There is a lack of data on the use of telemedicine (TM) in SLE. SLE outcome measures remain complex, and clinicians and clinical trialists have raised concerns about the accuracy of virtual disease activity measures. This study evaluates the level of agreement between virtual SLE outcome measures and face-to-face (F2F) encounter. Here, we describe the study design, virtual physical examination protocol and demographics for the first 50 patients evaluated.
Methods and analysis This is an observational, longitudinal study of 200 patients with SLE with varying levels of disease activity from 4 academic lupus centres serving diverse populations. Each study participant will be evaluated at a baseline and a follow-up visit. At each visit, participants are evaluated by the same physician first via a videoconference-based TM and then a F2F encounter. For this protocol, virtual physical examination guidelines relying on physician-directed patient self-examination were established. SLE disease activity measures will be completed immediately after the TM encounter and repeated after the F2F encounter for each visit. The degree of agreement between TM and F2F disease activity measures will be analysed using the Bland-Altman method. An interim analysis is planned after the enrolment of the first 50 participants.
Ethics and dissemination This study has been reviewed by the Columbia University Medical Center Institutional Review Board (IRB Protocol #: AAAT6574). The full results of this study will be published after the final data analysis of 200 patients. The abrupt shift to TM visits due to the COVID-19 pandemic disrupted clinical practice and clinical trials. Establishing a high level of agreement between SLE disease activity measures obtained with videoconference TM and F2F at the same time point, will allow for improved assessment of disease activity when F2F data cannot be acquired. This information may guide both medical decision-making and provide reliable outcome measures for clinical research.
- lupus erythematosus, systemic
- outcome assessment, health care
- clinical trial
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
SLE is a complex autoimmune disease.
Disease activity measures have not been evaluated in a virtual setting.
WHAT THIS STUDY ADDS
This paper describes the protocol for a study evaluating the level of agreement of SLE disease activity measures between virtual and face-to-face visits, including virtual physical examination guidelines for SLE.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This protocol may help inform other SLE clinical trials planning on integrating telemedicine visits as well as individuals completing virtual SLE assessments in their clinical practice.
Introduction
SLE is a complex autoimmune disease characterised by sporadic and unpredictable flares of inflammation which can affect any organ in the body.1 There are multiple challenges in treating SLE including prolonged time to diagnosis, partially effective therapies, medication toxicity and negative impact on quality of life, personal relationships and ability to work.2–4 Outcome measures for SLE disease activity are complex5 6 and imperfect under the best of circumstances and many question their accuracy in telemedicine (TM) visits.
The widespread and quick implementation of TM during the COVID-19 pandemic changed the face of medicine. This caused significant interruptions in both rheumatology care and clinical trials.7 8 Face-to-face (F2F) clinical medicine was limited throughout 2020 and visits for clinical trial subjects were conducted remotely using TM platforms.8 However, SLE outcome measures have not been validated in virtual encounters, and disease activity measures were not reported during this time for many trials, leading to a loss of significant amounts of clinical trial data. TM could be used to preserve the integrity of clinical trials by minimising disruptions in care and alternative methods of data/outcomes collection can continue when in-person assessments are not feasible.8
While patients and providers are satisfied with TM in general, there are limited data on the use of TM in SLE and SLE clinical trials.9–11 Studies have demonstrated that TM can be used to achieve tight control of disease activity in rheumatoid arthritis (RA).12 Recently, So et al published the 6-month results of a randomised clinical trial using TM to follow-up patients with lupus nephritis (LN) and showed improved patient satisfaction and similar short-term disease control in patients with LN evaluated using TM compared with F2F care; however, TM follow-up was associated with more hospitalisations in patients with higher baseline disease activity measured by Physician Global Assessment (PGA) score.13
Additionally, a study by Piga et al evaluated the reliability of virtual video-assisted visits added to the tight control strategy for inflammatory rheumatic diseases, including 29 patients with SLE. While video visits showed ‘excellent sensitivity and specificity in identifying the need for treatment adjustment due to inadequate disease control’, the main driver for the low sensitivity of virtual consultation in identifying the need for treatment tapering was SLE diagnosis, ‘mostly because of discordance with F2F consultation in glucocorticoid tapering’.14 Although TM offers alternative methods of data collection when in-person assessments are not feasible, TM’s value in assessing disease activity in a complicated illness like SLE has not been established.8
Here, we report the study design for a study funded by the US Department of Defense to evaluate the accuracy of virtual SLE disease activity measures obtained from a video TM interview and virtual physical examination that could be used to guide medical decision-making and provide reliable outcome measures for clinical research, as well as the demographics of the first 50 patients. The protocol investigates how well physician assessments of SLE disease activity, obtained during virtual visits, agree with those obtained in F2F visits. We will also explore the impact of sociodemographic indicators and patient-reported measures such as pain, mood or fatigue on the degree of agreement between the virtual and F2F disease activity assessments by both patients and clinicians. Furthermore, we describe our standardised physical examination protocol for virtual encounters. We anticipate that our results will have the potential to preserve the integrity of clinical trials and improve clinical outcomes by minimising disruptions in care when F2F visits are not possible.
Methods and analysis
Study objectives/Hypothesis
The main objective of this study is to investigate whether physician assessments of SLE disease activity using standard, validated measures obtained during videoconference TM visits (termed ‘virtual disease activity measures’) correlate with those obtained in a F2F visit. We hypothesise that the estimated degrees of agreement between the virtual and F2F disease activity measures will be satisfactory. Secondary objectives include assessment of the impact of patient-related factors on the accuracy of the virtual disease activity measures by evaluating associations between disease activity, demographic and socioeconomic variables (including race, ethnicity, sex, education, insurance, zip code), medication use, comorbid illness and patient-reported outcome (PRO) measures for pain, fatigue and mood and degrees of agreement between the virtual and F2F disease activity measures. We anticipate that socioeconomic variables and severity of disease activity may impact agreement.
We will also evaluate sensitivity to change the virtual disease activity measures. The same cohort of subjects will be examined at a subsequent follow-up visit to assess the sensitivity to change of the virtual disease activity measures compared with the F2F disease activity measures. We anticipate that sensitivity to change will be similar between the virtual and F2F disease activity measures.
Study overview
A minimum of 200 patients with SLE with variable levels of disease activity (inactive to severe) will be enrolled across four sites to evaluate the virtual disease activity measures. Patients will be selected by convenience sampling. To avoid potential confounders of new patient visits, all patients must be known to the physician prior to visit 1 and other eligibility criteria are provided in table 1. At each visit, the patient will complete the PROs (table 2) first. The TM encounter will be followed by the F2F encounter on the same day with the same physician. SLE disease activity assessments (table 2) are completed by physicians immediately after each encounter and finalised when laboratory values become available. Physicians and patients provide qualitative feedback after each encounter (figure 1).
The flow of the study visits—both virtual and face-to-face—is outlined below. PRO, patient-reported outcome; TM, telemedicine.
Inclusion/Exclusion criteria
Summary of PROs and clinician-reported disease activity measures
Patient and public involvement
Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
The PROs and clinician-reported disease activity measures to be collected in the study are listed in table 2. Additionally, physicians complete the Systemic Lupus International Collaborative Clinics Damage Index.15 16
Extensive data are collected on changes in history, review of systems and physical examination findings between the TM and F2F encounters. Additionally, the physician is asked to comment on any change in plan triggered by the F2F evaluation.
An interim analysis is planned after enrolment of 50 subjects to identify components of the virtual assessments that may need additional attention. Data and qualitative feedback will be probed to identify areas contributing to the greatest discordance. These data will detect gaps in assessments that could be addressed with guided self-physical examination techniques implemented in the virtual-video visits to improve the precision of these evaluations. An additional 150 patients will then be enrolled using the same or refined instructions for the virtual visit, pending outcome of the interim analysis.
Statistical plan
The projected sample size is 200 enrolled patients. With data from two assessments per patient, the study will have 90% power with a two-sided type 1 error rate of 5% to detect a minimum intraclass correlation coefficient (ICC) of 0.21 between the virtual and F2F measures obtained at the same visit and between change from baseline and follow-up in these measures. In addition, the width of the 95% CI will be no greater than ±10% assuming the ICC is at least 0.5. The study will also have 90% power to detect a small minimum effect size of 0.23 when analysing the difference between the virtual and F2F measures using the paired t-test.
In the interim and final analysis, we will analyse the degrees of agreement between the virtual and F2F disease activity measures. For each continuous variable (eg, SLE Disease Activity Index (SLEDAI), British Isles Lupus Assessment Group (BILAG), Lupus Foundation of America rapid evaluation of activity in lupus (LFA-REAL) scores), we will first analyse the difference between the virtual and F2F measures from each patient and compute Bland-Altman limits of agreement (LOA): 
±1.96 SD, where 
is the sample mean of the individual differences and SD is the sample SD.17 If the LOA includes clinically significant differences, this suggests that agreement between virtual and F2F results is not satisfactory. A paired t-test or Wilcoxon signed-rank test will be performed to evaluate the statistical significance of the mean difference between virtual and F2F measures.
When analysing the final data, we will additionally assess whether the level of agreement between the virtual and F2F disease activity measures varies by disease activity, demographic and socioeconomic variables and PRO measures for pain, fatigue and mood. For this analysis, we will fit multiple linear regression models in which the magnitude of the difference between virtual and F2F measures is the outcome, and the variables listed above are the predictors. We will also estimate Bland-Altman LOA, as well as ICC and concordance correlation coefficients separately in a limited number of subgroups (eg, low/high socioeconomic status; low/high disease activity) to ensure that the sample size in each subgroup is sufficient (table 3).
Predictors, mediators and confounders of outcomes
We will also estimate the change in disease activity between the baseline and follow-up visits measured by the virtual and F2F disease activity measures and evaluate the degree of agreement between the two approaches. The same statistical methodology as described above will be applied, for the final 200 patients except that the outcome of interest here will be the magnitude of change in disease activity assessed virtually versus F2F.
Demographics of the first 50 participants and physical examination protocol
Fifty participants were enrolled, 25 have completed the follow-up visit so far. The baseline characteristics are summarised in table 4; 82% women, mean age 38.9±13. The current enrolment spans a wide range of physician-determined categories of disease activity (25% inactive, 56% mild/moderate, 18% severe). The study population is racially and ethnically diverse.
Demographic and clinical characteristics (n=50)
The virtual physical examination protocol includes examination of the head, eyes, ears, nose, throat; skin; cardiorespiratory; posture/gait/movements and joints. Guidance for the virtual examination is outlined in table 5.
Standardised virtual physical examination protocol
Ethics and dissemination
The central Institutional Review Board (IRB) for this observational study will be the Columbia University Medical Center IRB (IRB Protocol #: AAAT6574).
For this observational study, ONLY adverse events that are attributed to participation in the virtual, TM encounter will be reported. The intervention in this observational study is a TM encounter that will be conducted in an examination room in the clinic or research centre on the same day as the patient’s routine F2F visit. The risk of any adverse events occurring as a consequence of the TM encounter is minimal. Any adverse events that occur during the TM encounter that the physician attributes to the TM encounter will be recorded.
An interim analysis after the first 50 subjects was conducted to identify components of the virtual assessments that may need additional attention. However, the results of this study will be published after the completion of enrolment of all 200 patients to provide a fully powered analysis.
Discussion
The current publication discusses the study design for evaluating the level of agreement between SLE disease activity measures assessed with TM and F2F evaluations. We proposed consecutive, same-day evaluations of disease activity by the same rater as we believe this will provide the data needed to increase confidence in virtual evaluations for practising rheumatologists.
The main question that will be addressed by this study is whether the virtual encounter can capture the equivalent information to the gold standard, the F2F encounter. The two visits occur consecutively and are not intended to be independent encounters. The experts evaluate disease activity virtually and in person and determine if the F2F findings warrant a change in disease activity measures. We expect that rashes and arthritis will be the most difficult to assess virtually and therefore expect the Cutaneous Lupus Erythematosus Disease Area and Severity Index and Tender and Swollen Joint Count to have the most variability. The SLEDAI, BILAG, LFA-REAL mucocutaneous and musculoskeletal domains have less granularity, and a higher level of agreement would be expected for these activity measures.
We acknowledge that having patients examined by the same physician introduces recall bias, and as such would increase the level of agreement. Spacing out visits several days later was considered. However, there is major variability in SLE; the physical examination could be different, a delay in treatment would pose the ethical concern of withholding needed therapy, and an in person follow-up later would pose feasibility issues, scheduling visits within a time frame that works both for the patient and physician may be difficult, resulting in high dropout rates. While we considered adding multiple raters for each encounter, the ICC between investigators in SLE evaluations is modest (0.18–0.54)18 and the inter-rater variability could further confound the study. There is also the possibility that as the F2F encounter occurs after the virtual, it may increase the evaluator’s attention to specific physical examination details and introduce bias against agreement. We have also considered doing the F2F encounter before TM; however, this would clearly bias the TM encounter to agree with F2F.
Furthermore, the virtual physical examination findings are highly dependent on internet connection; poor internet connection results in missed physical examination findings as the physician is not able to visually verify the patient’s report and self-examination. We also recognise that since we cannot standardise lighting this may hinder the visualisation of mucocutaneous findings like oral ulcers and rashes. The TM joint examination heavily relies on patient report. While severe swelling may be obvious on the virtual physical examination, mild-to-moderate swelling is difficult to identify. Data from in-person RA studies have established that there is moderate inter-rater reliability among physicians as well as patients to physician swollen joint counts.19 20
There are multiple barriers to patient participation in SLE clinical trials. Patients have identified travel, childcare and missed work as factors that limit their participation;21 integration of TM visits into clinical trials could help alleviate these barriers. Patient interview and physical examination are critical components of the evaluation of lupus patients, and the ability of current technology to allow for accurate remote evaluations would be invaluable. Once standards for assessing responses to treatment are established using TM, virtual SLE outcome measures can be implemented in clinical trials and help address these barriers.
The ability to supplement regular clinic visits with TM could provide the opportunity to improve patient care, supporting more rapid adjustments in treatment, prevention of SLE flares, successful treat-to-target strategies and possibly lower costs.22 23 The development of methods to determine whether existing disease activity measures can be used to assess disease activity accurately in a virtual setting would provide much needed data when in-person visits are not feasible.
While some SLE therapeutic clinical trials have already implemented TM visits into their protocols, to our knowledge this is the first published standardised virtual physical examination protocol for SLE. The 200 participants to be evaluated in this study will provide the rigorous quantitative and qualitative data on the comparability between virtual and F2F disease activity measures needed to promote confidence in and acceptance of TM in lupus clinical care and research.
Conclusion
TM has the potential to be transformative in clinical trials, observational studies and clinical practice in SLE. If virtual assessments can be refined to provide accurate disease activity measures equivalent to what is achieved at F2F visits, it is also possible that the future clinical trials in lupus will include remote interim visits to decrease the burden on patients, researchers and sponsors.
Ethics statements
Patient consent for publication
References
Footnotes
Contributors All named authors meet the International Committee of Medical Journal Editors criteria for authorship for this article, take responsibility for the integrity of the work as a whole and have given their approval for this version to be published. ADA, MM, CyA, MYK, DLK, MD'E and CrA contributed to the concept and design of the manuscript. ADA, MM, CyA, MYK, DLK, MD'E, CrA, LK, WT and JB contributed to the data collection and statistical analysis of the manuscript. ADA and LK contributed to the drafting of the manuscript. All the contributing authors critically revised the manuscript.
Funding This work was supported by the US Department of Defense, grant number: W81XWH-21-1-0468.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.