Dataset Information

Dyspnea could be accurately assessed by a caregiver in hospitalized patients with respiratory diseases: Interrater reliability and agreement study.

ABSTRACT:

Background

The perception of dyspnea is a subjective feeling typically self-assessed by the patient. However, the assessment by a caregiver is sometimes required.

Objectives

The primary aim was to compare patient self-assessment and caregiver assessment of dyspnea (interrater reliability) using the modified Borg and visual analog scale (VAS) in hospitalized patients. The secondary aim was to compare dyspnea assessment between the two scales for patients and caregiver (inter-instrument reliability).

Methods

Self-assessment of dyspnea intensity of hospitalized patients with respiratory diseases was compared with caregiver's assessment. Dyspnea intensity was measured using two scales, the modified Borg scale (0-10 scale) and the 10 cm VAS. Mean difference and 95% confidence interval (CI) between assessors (i.e. patient versus caregiver) were calculated for each scale. Inter- and intra-rater reliability was calculated using intraclass correlation coefficients (ICCs).

Results

A total of 254 patients were recruited. The mean differences between patient and caregiver ratings were 0.31 (95% CI: 0.09, 0.53) for the modified Borg scale and 0.36 (95% CI: 0.06, 0.65) for the VAS scale. Interrater reliability was good for both scales with ICC of 0.79 (95% CI: 0.73, 0.84) for VAS and 0.82 (95% CI: 0.77, 0.86) for the modified Borg scale. The mean differences in scores between scales were 0.93 (95% CI 0.69, 1.17) for patients' ratings and 0.88 (95% CI 0.72, 1.04) for caregiver's rating. The inter-instrument reliability was moderate to good and similar for both assessors.

Conclusion

Dyspnea can be accurately estimated by caregivers when patients with lung diseases cannot self-report. Scores on the VAS to rate dyspnea were higher than the scores on the Borg scale.

SUBMITTER: Reychler G

PROVIDER: S-EPMC8721053 | biostudies-literature | 2021 Nov-Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Dyspnea could be accurately assessed by a caregiver in hospitalized patients with respiratory diseases: Interrater reliability and agreement study.

Reychler Gregory G Beaumont Marc M Latiers Anne-Claire AC Pieters Thierry T Fremault Antoine A

Brazilian journal of physical therapy 20210524 6

<h4>Background</h4>The perception of dyspnea is a subjective feeling typically self-assessed by the patient. However, the assessment by a caregiver is sometimes required.<h4>Objectives</h4>The primary aim was to compare patient self-assessment and caregiver assessment of dyspnea (interrater reliability) using the modified Borg and visual analog scale (VAS) in hospitalized patients. The secondary aim was to compare dyspnea assessment between the two scales for patients and caregiver (inter-instru ...[more]

PMID: 34119444

Similar Datasets

Project description:BackgroundDyspnea is a multidimensional experience similar to pain and is one of the most common clinical presentations in patients with respiratory diseases. Accurately evaluating the experience of dyspnea allows nurses and physicians to deliver better medical services to patients. The multidimensional dyspnea profile emphasizes the psychosocial factors of dyspnea and assesses immediate discomfort, sensory qualities, and the emotional responses of patients with dyspnea. At present, the validity, reliability, and test-retest reliability of the multidimensional dyspnea profile in patients with respiratory diseases in China are unclear.ObjectivesThe aim of this study was to investigate the validity, reliability, and test-retest reliability of the Chinese version of the multidimensional dyspnea profile and to assess the convergent validity between the Chinese version of the multidimensional dyspnea profile and the modified Medical Research Council Dyspnea Scale.MethodsThe factorial construct, intraclass correlations, internal consistency, and convergent validity of the Chinese version of the multidimensional dyspnea profile was evaluated using data from 231 inpatients with dyspnea from the respiratory department of a hospital. In the principal component analysis stage, 131 inpatients were evaluated. In the test-retest reliability analysis stage, 50 out of the 131 patients responded to the questionnaire again. In the confirmatory factor analysis, 100 inpatients from an independent sample were assessed.ResultsThe principal component analysis showed that the Chinese version of the multidimensional dyspnea profile had a two-factor structure: the immediate perceptual-related problem factor (6 items) and the emotional response-related problem factor (5 items). The convergent validity between the Chinese version of the multidimensional dyspnea profile and the modified Medical Research Council Dyspnea Scale was significant and acceptable based on the average variance extracted (r = .56, p < .001). The confirmatory factor analysis revealed a good model fit and provided support for the construct validity of the Chinese version of the multidimensional dyspnea profile. Overall, the internal consistency and intraclass correlation coefficient of the Chinese version of the multidimensional dyspnea profile were good.ConclusionThe 11-item Chinese version of the multidimensional dyspnea profile has acceptable validity and reliability in patients with respiratory diseases in China. In the future, more studies should be performed to further explore its clinical application.

Project description:BackgroundInterrater reliability, aka intercoder reliability, is defined as true agreement between raters, aka coders, without chance agreement. It is used across many disciplines including medical and health research to measure the quality of ratings, coding, diagnoses, or other observations and judgements. While numerous indices of interrater reliability are available, experts disagree on which ones are legitimate or more appropriate. Almost all agree that percent agreement (ao), the oldest and the simplest index, is also the most flawed because it fails to estimate and remove chance agreement, which is produced by raters' random rating. The experts, however, disagree on which chance estimators are legitimate or better. The experts also disagree on which of the three factors, rating category, distribution skew, or task difficulty, an index should rely on to estimate chance agreement, or which factors the known indices in fact rely on. The most popular chance-adjusted indices, according to a functionalist view of mathematical statistics, assume that all raters conduct intentional and maximum random rating while typical raters conduct involuntary and reluctant random rating. The mismatches between the assumed and the actual rater behaviors cause the indices to rely on mistaken factors to estimate chance agreement, leading to the numerous paradoxes, abnormalities, and other misbehaviors of the indices identified by prior studies.MethodsWe conducted a 4 × 8 × 3 between-subject controlled experiment with 4 subjects per cell. Each subject was a rating session with 100 pairs of rating by two raters, totaling 384 rating sessions as the experimental subjects. The experiment tested seven best-known indices of interrater reliability against the observed reliabilities and chance agreements. Impacts of the three factors, i.e., rating category, distribution skew, and task difficulty, on the indices were tested.ResultsThe most criticized index, percent agreement (ao), showed as the most accurate predictor of reliability, reporting directional r2 = .84. It was also the third best approximator, overestimating observed reliability by 13 percentage points on average. The three most acclaimed and most popular indices, Scott's π, Cohen's κ and Krippendorff's α, underperformed all other indices, reporting directional r2 = .312 and underestimated reliability by 31.4 ~ 31.8 points. The newest index, Gwet's AC1, emerged as the second-best predictor and the most accurate approximator. Bennett et al's S ranked behind AC1, and Perreault and Leigh's Ir ranked the fourth both for prediction and approximation. The reliance on category and skew and failure to rely on difficulty explain why the six chance-adjusted indices often underperformed ao, which they were created to outperform. The evidence corroborated the notion that the chance-adjusted indices assume intentional and maximum random rating while the raters instead exhibited involuntary and reluctant random rating.ConclusionThe authors call for more empirical studies and especially more controlled experiments to falsify or qualify this study. If the main findings are replicated and the underlying theories supported, new thinking and new indices may be needed. Index designers may need to refrain from assuming intentional and maximum random rating, and instead assume involuntary and reluctant random rating. Accordingly, the new indices may need to rely on task difficulty, rather than distribution skew or rating category, to estimate chance agreement.

Project description:ObjectiveThe presence of interictal epileptiform discharges (IED) in the electroencephalogram (EEG) is a key finding in the medical workup of a patient with suspected epilepsy. However, inter-rater agreement (IRA) regarding the presence of IED is imperfect, leading to incorrect and delayed diagnoses. An improved understanding of which IED attributes mediate expert IRA might help in developing automatic methods for IED detection able to emulate the abilities of experts. Therefore, using a set of IED scored by a large number of experts, we set out to determine which attributes of IED predict expert agreement regarding the presence of IED.MethodsIED were annotated on a 5-point scale by 18 clinical neurophysiologists within 200 30-s EEG segments from recordings of 200 patients. 5538 signal analysis features were extracted from the waveforms, including wavelet coefficients, morphological features, signal energy, nonlinear energy operator response, electrode location, and spectrogram features. Feature selection was performed by applying elastic net regression and support vector regression (SVR) was applied to predict expert opinion, with and without the feature selection procedure and with and without several types of signal normalization.ResultsMultiple types of features were useful for predicting expert annotations, but particular types of wavelet features performed best. Local EEG normalization also enhanced best model performance. As the size of the group of EEGers used to train the models was increased, the performance of the models leveled off at a group size of around 11.ConclusionsThe features that best predict inter-rater agreement among experts regarding the presence of IED are wavelet features, using locally standardized EEG. Our models for predicting expert opinion based on EEGer's scores perform best with a large group of EEGers (more than 10).SignificanceBy examining a large group of EEG signal analysis features we found that wavelet features with certain wavelet basis functions performed best to identify IEDs. Local normalization also improves predictability, suggesting the importance of IED morphology over amplitude-based features. Although most IED detection studies in the past have used opinion from three or fewer experts, our study suggests a "wisdom of the crowd" effect, such that pooling over a larger number of expert opinions produces a better correlation between expert opinion and objectively quantifiable features of the EEG.

Project description:PurposeMany neonates undergo electroencephalogram (EEG) monitoring to identify and manage acute symptomatic seizures. Information about brain function contained in the EEG background data may also help predict neurobehavioral outcomes. For EEG background features to be useful as prognostic indicators, the interpretation of these features must be standardized across electroencephalographers. We aimed at determining the interrater and intrarater agreement among electroencephalographers interpreting neonatal EEG background patterns.MethodsFive neonatal electroencephalographers reviewed 5-to-7.5-minute epochs of EEG from full-term neonates who underwent continuous conventional EEG monitoring. The EEG assessment tool used to classify background patterns was based on the American Clinical Neurophysiology Society's guideline for neonatal EEG terminology. Interrater and intrarater agreement were measured using Kappa coefficients.ResultsInterrater agreement was consistently highest for voltage (binary: substantial, kappa = 0.783; categorical: moderate, kappa = 0.562), seizure presence (fair-substantial; kappa = 0.375-0.697), continuity (moderate; kappa = 0.481), burst voltage (moderate; kappa = 0.574), suppressed background presence (moderate-substantial; kappa = 0.493-0.643), delta activity presence (fair-moderate; kappa = 0.369-0.432), theta activity presence (fair-moderate; kappa = 0.347-0.600), presence of graphoelements (fair; kappa = 0.381), and overall impression (binary: moderate, kappa = 0.495; categorical: fair-moderate, kappa = 0.347, 0.465). Agreement was poor or inconsistent for all other patterns. Intrarater agreement was variable, with highest average agreement for voltage (binary: substantial, kappa = 0.75; categorical: substantial, kappa = 0.714) and highest consistent agreement for continuity (moderate-substantial; kappa = 0.43-0.67) and overall impression (moderate-substantial; kappa = 0.42-0.68).ConclusionsThis study demonstrates substantial variability in neonatal EEG background interpretation across electroencephalographers, indicating a need for educational and technological strategies aimed at improving performance.

Project description:Response evaluation at regular intervals is indicated for treatment of metastatic breast cancer (MBC). FDG-PET/CT has the potential to monitor treatment response accurately. Our purpose was to: (a) compare the interrater agreement and reliability of the semi-quantitative PERCIST criteria to qualitative visual assessment in response evaluation of MBC and (b) investigate the intrarater agreement when comparing visual assessment of each rater to their respective PERCIST assessment. We performed a retrospective study on FDG-PET/CT in women who received treatment for MBC. Three specialists in nuclear medicine categorized response evaluation by qualitative assessment and standardized one-lesion PERCIST assessment. The scans were categorized into complete metabolic response, partial metabolic response, stable metabolic disease, and progressive metabolic disease. 37 patients with 179 scans were included. Visual assessment categorization yielded moderate agreement with an overall proportion of agreement (PoA) between raters of 0.52 (95% CI 0.44-0.66) and a Fleiss kappa estimate of 0.54 (95% CI 0.46-0.62). PERCIST response categorization yielded substantial agreement with an overall PoA of 0.65 (95% CI 0.57-0.73) and a Fleiss kappa estimate of 0.68 (95% CI 0.60-0.75). The difference in PoA between overall estimates for PERCIST and visual assessment was 0.13 (95% CI 0.06-0.21; p = 0.001), that of kappa was 0.14 (95% CI 0.06-0.21; p < 0.001). The overall intrarater PoA was 0.80 (95% CI 0.75-0.84) with substantial agreement by a Fleiss kappa of 0.74 (95% CI 0.69-0.79). Semi-quantitative PERCIST assessment achieved significantly higher level of overall agreement and reliability compared with qualitative assessment among three raters. The achieved high levels of intrarater agreement indicated no obvious conflicting elements between the two methods. PERCIST assessment may, therefore, give more consistent interpretations between raters when using FDG-PET/CT for response evaluation in MBC.

Project description:BackgroundAnterior cruciate ligament (ACL) rupture is the most common ligament injury treated surgically by orthopaedic surgeons. The gold standard for the treatment of the majority of primary ACL tears is ACL reconstruction. However, novel methods of repair, such as bridge-enhanced ACL repair (BEAR), are currently being investigated as alternatives to reconstruction. To assess patients for midsubstance repair suitability, clarify the prognostic implications of injury location and damage, and evaluate the results of a repair technique, it is important to have a baseline classification system or grading scale that is reproducible across surgeons, particularly for multicenter collaboration. Currently, no such system or scale exists.PurposeTo develop an arthroscopic ACL tear classification system and to evaluate its interobserver reliability.Study designCohort study (diagnosis); Level of evidence, 3.MethodsEleven fellowship-trained orthopaedic surgeon investigators reviewed 75 video clips containing arthroscopic evaluation of a torn ACL and then completed the 6-question ACL Pathology Evaluation Form. Agreement statistics including exact agreement, Fleiss κ, Gwet agreement coefficient 1 (AC1), and Gwet AC2 were then calculated to assess interobserver reliability.ResultsIn aggregate, the multiple assessments of observer reproducibility revealed that surgeon participants in this study, when evaluating the same injury, agreed roughly 80% of the time on whether (1) at least 50% of the tibial footprint remained, (2) the remaining tibial stump was ≥10 mm, and (3) the injury was therefore reparable using the BEAR procedure. Participants also agreed roughly 60% of the time on exactly how many suturable bundles were available. These characteristics are believed to be most important, among those studied, in determining whether a torn ACL is amenable to midsubstance repair.ConclusionThis study is the first of its kind to demonstrate the interobserver reliability of arthroscopic classification of ACL tears. We have demonstrated that this classification system, though not ideally reproducible, is reliable enough across surgeons at multiple institutions for use in multicenter studies.RegistrationNCT03776162 (ClinicalTrials.gov identifier).

Dataset Information

Dyspnea could be accurately assessed by a caregiver in hospitalized patients with respiratory diseases: Interrater reliability and agreement study.

Background

Objectives

Methods

Results

Conclusion

Publications

Dyspnea could be accurately assessed by a caregiver in hospitalized patients with respiratory diseases: Interrater reliability and agreement study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets