Dataset Information

External Validation of SpineNet, an Open-source Deep Learning Model for Grading Lumbar Disc Degeneration MRI Features, Using the Northern Finland Birth Cohort 1966.

ABSTRACT:

Study design

This is a retrospective observational study to externally validate a deep learning image classification model.

Objective

Deep learning models such as SpineNet offer the possibility of automating the process of disc degeneration (DD) classification from MRI. External validation is an essential step to their development. The aim of this study was to externally validate SpineNet predictions for DD using Pfirrmann classification and Modic changes (MC) on data from the Northern Finland Birth Cohort 1966 (NFBC1966).

Summary of data

We validated SpineNet using data from 1331 NFBC1966 participants for whom both lumbar spine MRI data and consensus disc degeneration gradings were available.

Methods

SpineNet returned Pfirrmann grade and MC presence from T2-weighted sagittal lumbar MRI sequences from NFBC1966, a dataset geographically and temporally separated from its training dataset. A range of agreement and reliability metrics were used to compare predictions with expert radiologists. Subsets of data that match SpineNet training data more closely were also tested.

Results

Balanced accuracy for DD was 78% (77-79%) and for MC 86% (85-86%). Inter-rater reliability for Pfirrmann grading was Lin's CCC=0.86 (0.85-0.87) and Cohen's κ=0.68 (0.67-0.69). In a low back pain subset these reliability metrics remained largely unchanged. In total, 20.83% of discs were rated differently by SpineNet compared to the human raters, but only 0.85% of discs had a grade difference greater than 1. Inter-rater reliability for MC detection was κ=0.74 (0.72-0.75). In the low back pain subset this metric was almost unchanged at κ=0.76 (0.73-0.79).

Conclusion

In this study, SpineNet has been benchmarked against expert human raters in the research setting. It has matched human reliability and demonstrates robust performance despite the multiple challenges facing model generalizability.

SUBMITTER: McSweeney TP

PROVIDER: S-EPMC9990601 | biostudies-literature | 2022 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

External Validation of SpineNet, an Open-Source Deep Learning Model for Grading Lumbar Disk Degeneration MRI Features, Using the Northern Finland Birth Cohort 1966.

McSweeney Terence P TP McSweeney Terence P TP Tiulpin Aleksei A Saarakkala Simo S Niinimäki Jaakko J Windsor Rhydian R Jamaludin Amir A Kadir Timor T Karppinen Jaro J Määttä Juhani J

Spine 20221230 7

<h4>Study design</h4>This is a retrospective observational study to externally validate a deep learning image classification model.<h4>Objective</h4>Deep learning models such as SpineNet offer the possibility of automating the process of disk degeneration (DD) classification from magnetic resonance imaging (MRI). External validation is an essential step to their development. The aim of this study was to externally validate SpineNet predictions for DD using Pfirrmann classification and Modic chan ...[more]

PMID: 36728678

Similar Datasets

Project description:BackgroundAlthough it has been suggested that lumbar disc degeneration (LDD) is a significant risk factor for low back pain (LBP), its role remains uncertain. Our objective was to clarify the association between LDD and LBP and whether mental distress modifies the association.MethodsParticipants of a birth cohort underwent 1.5-T lumbar magnetic resonance imaging at the age of 47. The association between the sum score of LDD (Pfirrmann classification, range 0-15) and LBP (categorized into "no pain", "mild-to-moderate pain", "bothersome-and-frequent pain") was assessed using logistic regression analysis, with sex, smoking, body mass index, physical activity, occupational exposure, education, and presence of Modic changes and disc herniations as confounders. The modifying role of mental distress (according to the Hopkins Symptom Check List-25 [HSCL-25], the Beck Depression Inventory and the Generalized Anxiety Disorder Scale) in the association was analyzed using linear regression.ResultsOf the study population (n = 1505), 15.2% had bothersome and frequent LBP, and 29.0% had no LBP. A higher LDD sum score increased the odds of belonging to the "mild-to-moderate pain" category (adjusted OR corresponding to an increase of one point in the LDD sum score 1.11, 95% CI 1.04-1.18, P = 0.003) and the "bothersome-and-frequent pain" category (adjusted OR 1.20, 95% CI 1.10-1.31, P < 0.001), relative to the "no pain" category. Mental distress significantly modified the association between LDD and LBP, as a linear positive association was consistently observed among individuals without mental distress according to HSCL-25 (adjusted B 0.16, 95% CI 0.07-0.26, P < 0.001), but not among individuals with higher mental distress.ConclusionsLDD was significantly associated with both mild-to-moderate and bothersome-and-frequent LBP. However, the co-occurrence of mental distress diminished the association between LDD and LBP bothersomeness. Our results strongly suggest that mental symptoms affect the pain experience.

Project description:<p>The Northern Finland Birth Cohorts program (NFBC) was initiated in the 1960s in the two northernmost provinces of Finland to study risk factors involved in pre-term birth and intrauterine growth retardation, and the consequences of these early adverse events on subsequent morbidity and mortality. The uniqueness of NBFCs is that the data of the cohorts were obtained from early fetal life (including maternal health during pregnancy) to adulthood. The NFBC1966 includes 12,058 live births to mothers in the two northern-most provinces of Finland. Two decades later, a second cohort of 9432 births was obtained (NFBC1986). In NFBC1966 pregnancies were followed prospectively from the first antenatal contact (10-16th week). After birth, the offspring were examined and then again underwent clinical evaluation at ages 1y, 7y, 14-16y and 31y. At each visit, a wide range of phenotypic, lifestyle and demographic data were gathered by questionnaires and clinical examinations. For the most part, NFBC1986 has undergone similar evaluations to NFBC1966. Linkage to national registries includes hospitalization, deaths, education, medication, pensions, and provides up-to-date demographic and clinical information for members of both cohorts. DNA samples were obtained from 5,923 subjects from NFBC1966 and 6688 subjects from NFBC1986. Data coverage, 96% of all births in 1966 and 99% in 1986, is highly representative for the whole population. The NFBC program comprises more than 20 different projects coordinated by the Center of Lifecourse Disease studies in Northern Finland (COLD) at Oulu University. The prospective data collected from the NFBCs form a unique resource, allowing the study of disease emergence, and of the importance of genetic, biological, social and behavioral risk factors.</p> <p>The genome-wide association (GWA) study sponsored through the STAMPEED program of NHLBI employed genomic DNA samples previously collected by the NFBC1966 study and stored in the DNA repository of the National Institute for Health and Welfare, Finland. This NHLBI sponsored RO1 project aimed to identify genetic variants contributing to metabolic and cardiovascular diseases (CVD). In addition to de-identified genome wide genotypic data, a selected list of phenotypic data related to CVD including weight, height, BMI, HDL, LDL, total cholesterol, triglyceride, glucose, insulin and fasting status, are also available in dbGaP. A summary of the GWAS for the NFBC1966 cardiovascular risk traits can be found in Sabatti et al., Nature Genetics 41: 35-46, 2009, PMID: <a href="http://www.ncbi.nlm.nih.gov/pubmed/19060910" target="_blank">19060910</a>.</p>