Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Agreement between visual inspection and objective analysis methods: A replication and extension.

ABSTRACT: Behavior analysts typically rely on visual inspection of single-case experimental designs to make treatment decisions. However, visual inspection is subjective, which has led to the development of supplemental objective methods such as the conservative dual-criteria method. To replicate and extend a study conducted by Wolfe et al. (2018) on the topic, we examined agreement between the visual inspection of five raters, the conservative dual-criteria method, and a machine-learning algorithm (i.e., the support vector classifier) on 198 AB graphs extracted from clinical data. The results indicated that average agreement between the 3 methods was generally consistent. Mean interrater agreement was 84%, whereas raters agreed with the conservative dual-criteria method and the support vector classifier on 84% and 85% of graphs, respectively. Our results indicate that both objective methods produce results consistent with visual inspection, which may support their future use.

SUBMITTER: Taylor T

PROVIDER: S-EPMC9323513 | biostudies-literature | 2022 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Agreement between visual inspection and objective analysis methods: A replication and extension.

Taylor Tessa T Lanovaz Marc J MJ

Journal of applied behavior analysis 20220427 3

Behavior analysts typically rely on visual inspection of single-case experimental designs to make treatment decisions. However, visual inspection is subjective, which has led to the development of supplemental objective methods such as the conservative dual-criteria method. To replicate and extend a study conducted by Wolfe et al. (2018) on the topic, we examined agreement between the visual inspection of five raters, the conservative dual-criteria method, and a machine-learning algorithm (i.e., ...[more]

PMID: 35478098

Similar Datasets

Ictal EEG source imaging in presurgical evaluation: High agreement between analysis methods.

Project description:PurposeTo determine the agreement between five different methods of ictal EEG source imaging, and to assess their accuracy in presurgical evaluation of patients with focal epilepsy. It was hypothesized that high agreement between methods was associated with higher localization-accuracy.MethodsEEGs were recorded with a 64-electrode array. Thirty-eight seizures from 22 patients were analyzed using five different methods phase mapping, dipole fitting, CLARA, cortical-CLARA and minimum norm. Localization accuracy was determined at sub-lobar level. Reference standard was the final decision of the multidisciplinary epilepsy surgery team, and, for the operated patients, outcome one year after surgery.ResultsAgreement between all methods was obtained in 13 patients (59%) and between all but one methods in additional six patients (27%). There was a trend for minimum norm being less accurate than phase mapping, but none of the comparisons reached significance. Source imaging in cases with agreement between all methods was not more accurate than in the other cases. Ictal source imaging achieved an accuracy of 73% (for operated patients: 86%).ConclusionThere was good agreement between different methods of ictal source imaging. However, good inter-method agreement did not necessarily imply accurate source localization, since all methods faced the limitations of the inverse solution.

| S-EPMC5176190 | biostudies-literature

Spatial Frequency Maps in Human Visual Cortex: A Replication and Extension.

Project description:In a step toward developing a model of human primary visual cortex, a recent study introduced a model of spatial frequency tuning in V1 (Broderick, Simoncelli, & Winawer, 2022). The model is compact, using just 9 parameters to predict BOLD response amplitude for locations across all of V1 as a function of stimulus orientation and spatial frequency. Here we replicated this analysis in a new dataset, the 'nsdsynthetic' supplement to the Natural Scenes Dataset (Allen et al., 2022), to assess generalization of model parameters. Furthermore, we extended the analyses to extrastriate maps V2 and V3. For each retinotopic map in the 8 NSD subjects, we fit the 9-parameter model. Despite many experimental differences between NSD and the original study, including stimulus size, experimental design, and MR field strength, there was good agreement in most model parameters. The dependence of preferred spatial frequency on eccentricity in V1 was similar between NSD and Broderick et al. Moreover, the effect of absolute stimulus orientation on spatial frequency maps was similar: higher preferred spatial frequency for horizontal and cardinal orientations compared to vertical and oblique orientations in both studies. The extension to extrastriate maps revealed that the biggest change in tuning between maps was in bandwidth: the bandwidth in spatial frequency tuning increased by 70% from V1 to V2 and 100% from V1 to V3, paralleling known increases in receptive field size. Together, the results show robust reproducibility and bring us closer to a systematic characterization of spatial encoding in the human visual system.

| S-EPMC11785079 | biostudies-literature

Visual inspection for diagnosing cutaneous melanoma in adults.

Project description: Not available

| S-EPMC6492463 | biostudies-literature

Methods of assessing categorical agreement between correlated screening tests in clinical studies.

Project description:Advances in breast imaging and other screening tests have prompted studies to evaluate and compare the consistency between experts' ratings of existing with new screening tests. In clinical settings, medical experts make subjective assessments of screening test results such as mammograms. Consistency between experts' ratings is evaluated by measures of inter-rater agreement or association. However, conventional measures, such as Cohen's and Fleiss' kappas, are unable to be applied or may perform poorly when studies consist of many experts, unbalanced data, or dependencies between experts' ratings exist. Here we assess the performance of existing approaches including recently developed summary measures for assessing the agreement between experts' binary and ordinal ratings when patients undergo two screening procedures. Methods to assess consistency between repeated measurements by the same experts are also described. We present applications to three large-scale clinical screening studies. Properties of these agreement measures are illustrated via simulation studies. Generally, a model-based approach provides several advantages over alternative methods including the ability to flexibly incorporate various measurement scales (i.e. binary or ordinal), large numbers of experts and patients, sparse data, and robustness to prevalence of underlying disease.

| S-EPMC8299998 | biostudies-literature

Guinier peak analysis for visual and automated inspection of small-angle X-ray scattering data.

Project description:The Guinier region in small-angle X-ray scattering (SAXS) defines the radius of gyration, Rg, and the forward scattering intensity, I(0). In Guinier peak analysis (GPA), the plot of qI(q) versus q2 transforms the Guinier region into a characteristic peak for visual and automated inspection of data. Deviations of the peak position from the theoretical position in dimensionless GPA plots can suggest parameter errors, problematic low-resolution data, some kinds of intermolecular interactions or elongated scatters. To facilitate automated analysis by GPA, the elongation ratio (ER), which is the ratio of the areas in the pair-distribution function P(r) after and before the P(r) maximum, was characterized; symmetric samples have ER values around 1, and samples with ER values greater than 5 tend to be outliers in GPA analysis. Use of GPA+ER can be a helpful addition to SAXS data analysis pipelines.

| S-EPMC5045725 | biostudies-literature

Univariate Statistical Analysis as a Guide to ¹H-NMR Spectra Signal Assignment by Visual Inspection.

Project description:In Proton Nuclear Magnetic Resonance (¹H-NMR) spectroscopy, the signals assignment procedure is normally conducted by visual inspection of the spectra, by taking advantage of the innate predisposition of human eye for pattern recognition. In the case of untargeted metabolomics investigations on food and body fluids, the complexity of the spectra may lead the user to overlook signals, independently from their biological relevance. Here, we describe a four steps procedure that is designed to guide signals assignment task by visual inspection. The procedure can be employed whenever an experimental plan allows for the application of a univariate statistical analysis on a point-by-point basis, which is commonly the case. By comparing, as a proof of concept, ¹H-NMR spectra of vaginal fluids of healthy and bacterial vaginosis (BV) affected women, we show that the procedure is also readily usable by non-experts in three particularly challenging cases: overlapping multiplets, poorly aligned signals, and signals with very poor signal-to-noise ratio. The paper is accompanied by the necessary codes and examples written in R computational language to allow the interested user gaining a hands-on impression of the procedure's strengths and weaknesses.

| S-EPMC6359365 | biostudies-literature

Relationship between porcine carcass grades and estimated traits based on conventional and non-destructive inspection methods.

Project description:As pork consumption increases, rapid and accurate determination of porcine carcass grades at abattoirs has become important. Non-destructive, automated inspection methods have improved slaughter efficiency in abattoirs. Furthermore, the development of a calibration equation suitable for non-destructive inspection of domestic pig breeds may lead to rapid determination of pig carcass and more objective pork grading judgement. In order to increase the efficiency of pig slaughter, the correct estimation of the automated-method that can accommodate the existing pig carcass judgement should be made. In this study, the previously developed calibration equation was verified to confirm whether the estimated traits accord with the actual measured traits of pig carcass. A total of 1,069,019 pigs, to which the developed calibration equation, was applied were used in the study and the optimal estimated regression equation for actual measured two traits (backfat thickness and hot carcass weight) was proposed using the estimated traits. The accuracy of backfat thickness and hot carcass weight traits in the estimated regression models through stepwise regression analysis was 0.840 (R 2) and 0.980 (R 2), respectively. By comparing the actually measured traits with the estimated traits, we proposed optimal estimated regression equation for the two measured traits, which we expect will be a cornerstone for the Korean porcine carcass grading system.

| S-EPMC8819329 | biostudies-literature

Objective analysis of the topological organization of the human cortical visual connectome suggests three visual pathways.

Project description:The cortical visual system is composed of many areas serving various visual functions. In non-human primates, these are broadly organised into two distinct processing pathways: a ventral pathway for object recognition, and a dorsal pathway for action. In humans, recent theoretical proposals suggest the possible existence of additional pathways, but direct empirical evidence has yet to be presented. Here, we estimated the connectivity patterns between 22 human visual areas using resting-state functional MRI data of 470 individuals, leveraging the unprecedented data quantity and quality of the Human Connectome Project and a novel probabilistic atlas. An objective, data-driven analysis into the topological organisation of connectivity and subsequent quantitative confirmation revealed a highly significant triple dissociation between the retinotopic areas on the dorsal, ventral and lateral surfaces of the human occipital lobe. This suggests that the functional organisation of the human visual system involves not two but three cortical pathways.

| S-EPMC5780302 | biostudies-literature

Lacrosse Athletes Load and Recovery Monitoring: Comparison between Objective and Subjective Methods.

Project description:Both objective (OM) and subjective (SM) methods are used in athletic studies, regardless of sport type, to identify and analyze load and recovery status of athletes. As little information exists about the comparison of these two methodologies, the aim of this study is to compare and contrast information that defines the relationship between both methods. Twelve international male lacrosse athletes participated in this study over the course of which participants heart-rate-variability and questionnaire-data were collected. Statistical analysis was performed to evaluate changes over time and correlations between used methods. Comparison between baseline values and competition showed a reduction in root-mean-square of successive differences (RMSSD) (p < 0.01) and the proportion of beat-intervals (NN) that differ by more than 50 ms divided by total number of NNs (pNN50) (p < 0.01). Further, RMSSD values showed differences during competition with large effects (p = 0.02; ?2 = 0.24). SM (p < 0.01) showed different progression during competition. Correlation was found for used SM and OM, when considered separately. No evidence for a reliable prediction of OM values using SM could be found. According to these findings, we recommend using a combination of SM and OM data to quantify the physiological stress of training and competition, respectively.

| S-EPMC7246625 | biostudies-literature

An annotated grain kernel image database for visual quality inspection.

Project description:We present a machine vision-based database named GrainSet for the purpose of visual quality inspection of grain kernels. The database contains more than 350K single-kernel images with experts' annotations. The grain kernels used in the study consist of four types of cereal grains including wheat, maize, sorghum and rice, and were collected from over 20 regions in 5 countries. The surface information of each kernel is captured by our custom-built device equipped with high-resolution optic sensor units, and corresponding sampling information and annotations include collection location and time, morphology, physical size, weight, and Damage & Unsound grain categories provided by senior inspectors. In addition, we employed a commonly used deep learning model to provide classification results as a benchmark. We believe that our GrainSet will facilitate future research in fields such as assisting inspectors in grain quality inspections, providing guidance for grain storage and trade, and contributing to applications of smart agriculture.

| S-EPMC10632488 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data