Dataset Information

All metrics are equal, but some metrics are more equal than others: A systematic search and review on the use of the term 'metric'.

ABSTRACT:

Objective

To examine the use of the term 'metric' in health and social sciences' literature, focusing on the interval scale implication of the term in Modern Test Theory (MTT).

Materials and methods

A systematic search and review on MTT studies including 'metric' or 'interval scale' was performed in the health and social sciences literature. The search was restricted to 2001-2005 and 2011-2015. A Text Mining algorithm was employed to operationalize the eligibility criteria and to explore the uses of 'metric'. The paradigm of each included article (Rasch Measurement Theory (RMT), Item Response Theory (IRT) or both), as well as its type (Theoretical, Methodological, Teaching, Application, Miscellaneous) were determined. An inductive thematic analysis on the first three types was performed.

Results

70.6% of the 1337 included articles were allocated to RMT, and 68.4% were application papers. Among the number of uses of 'metric', it was predominantly a synonym of 'scale'; as adjective, it referred to measurement or quantification. Three incompatible themes 'only RMT/all MTT/no MTT models can provide interval measures' were identified, but 'interval scale' was considerably more mentioned in RMT than in IRT.

Conclusion

'Metric' is used in many different ways, and there is no consensus on which MTT metric has interval scale properties. Nevertheless, when using the term 'metric', the authors should specify the level of the metric being used (ordinal, ordered, interval, ratio), and justify why according to them the metric is at that level.

SUBMITTER: Adroher ND

PROVIDER: S-EPMC5839589 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

All metrics are equal, but some metrics are more equal than others: A systematic search and review on the use of the term 'metric'.

Adroher Núria Duran ND Prodinger Birgit B Fellinghauer Carolina Saskia CS Tennant Alan A

PloS one 20180306 3

<h4>Objective</h4>To examine the use of the term 'metric' in health and social sciences' literature, focusing on the interval scale implication of the term in Modern Test Theory (MTT).<h4>Materials and methods</h4>A systematic search and review on MTT studies including 'metric' or 'interval scale' was performed in the health and social sciences literature. The search was restricted to 2001-2005 and 2011-2015. A Text Mining algorithm was employed to operationalize the eligibility criteria and to ...[more]

PMID: 29509813

Similar Datasets

Project description:BackgroundUse of patient portals has been associated with positive outcomes in patient engagement and satisfaction. Portal studies have also connected portal use, as well as the nature of users' interactions with portals, and the contents of their generated data to meaningful cost and quality outcomes. Incentive programs in the United States have encouraged uptake of health information technology, including patient portals, by setting standards for meaningful use of such technology. However, despite widespread interest in patient portal use and adoption, studies on patient portals differ in actual metrics used to operationalize and track utilization, leading to unsystematic and incommensurable characterizations of use. No known review has systematically assessed the measurements used to investigate patient portal utilization.ObjectiveThe objective of this study was to apply systematic review criteria to identify and compare methods for quantifying and reporting patient portal use.MethodsOriginal studies with quantifiable metrics of portal use published in English between 2014 and the search date of October 17, 2018, were obtained from PubMed using the Medical Subject Heading term "Patient Portals" and related keyword searches. The first search round included full text review of all results to confirm a priori data charting elements of interest and suggest additional categories inductively; this round was supplemented by the retrieval of works cited in systematic reviews (based on title screening of all citations). An additional search round included broader keywords identified during the full-text review of the first round. Second round results were screened at abstract level for inclusion and confirmed by at least two raters. Included studies were analyzed for metrics related to basic use/adoption, frequency of use, duration metrics, intensity of use, and stratification of users into "super user" or high utilizers. Additional categories related to provider (including care team/administrative) use of the portal were identified inductively. Additional analyses included metrics aligned with meaningful use stage 2 (MU-2) categories employed by the US Centers for Medicare and Medicaid Services and the association between the number of portal metrics examined and the number of citations and the journal impact factor.ResultsOf 315 distinct search results, 87 met the inclusion criteria. Of the a priori metrics, plus provider use, most studies included either three (26 studies, 30%) or four (23 studies, 26%) metrics. Nine studies (10%) only reported the patient use/adoption metric and only one study (1%) reported all six metrics. Of the US-based studies (n=76), 18 (24%) were explicitly motivated by MU-2 compliance; 40 studies (53%) at least mentioned these incentives, but only 6 studies (8%) presented metrics from which compliance rates could be inferred. Finally, the number of metrics examined was not associated with either the number of citations or the publishing journal's impact factor.ConclusionsPortal utilization measures in the research literature can fall below established standards for "meaningful" or they can substantively exceed those standards in the type and number of utilization properties measured. Understanding how patient portal use has been defined and operationalized may encourage more consistent, well-defined, and perhaps more meaningful standards for utilization, informing future portal development.

Project description:Previous reports identifying discordance between multiparameter tests at the individual patient level have been largely attributed to methodological shortcomings of multiple in silico studies. Comparisons between tests, when performed using actual diagnostic assays, have been predicted to demonstrate high degrees of concordance. OPTIMA prelim compared predicted risk stratification and subtype classification of different multiparameter tests performed directly on the same population. Three hundred thirteen women with early breast cancer were randomized to standard (chemotherapy and endocrine therapy) or test-directed (chemotherapy if Oncotype DX recurrence score >25) treatment. Risk stratification was also determined with Prosigna (PAM50), MammaPrint, MammaTyper, NexCourse Breast (IHC4-AQUA), and conventional IHC4 (IHC4). Subtype classification was provided by Blueprint, MammaTyper, and Prosigna. Oncotype DX predicted a higher proportion of tumors as low risk (82.1%, 95% confidence interval [CI] = 77.8% to 86.4%) than were predicted low/intermediate risk using Prosigna (65.5%, 95% CI = 60.1% to 70.9%), IHC4 (72.0%, 95% CI = 66.5% to 77.5%), MammaPrint (61.4%, 95% CI = 55.9% to 66.9%), or NexCourse Breast (61.6%, 95% CI = 55.8% to 67.4%). Strikingly, the five tests showed only modest agreement when dichotomizing results between high vs low/intermediate risk. Only 119 (39.4%) tumors were classified uniformly as either low/intermediate risk or high risk, and 183 (60.6%) were assigned to different risk categories by different tests, although 94 (31.1%) showed agreement between four of five tests. All three subtype tests assigned 59.5% to 62.4% of tumors to luminal A subtype, but only 121 (40.1%) were classified as luminal A by all three tests and only 58 (19.2%) were uniformly assigned as nonluminal A. Discordant subtyping was observed in 123 (40.7%) tumors. Existing evidence on the comparative prognostic information provided by different tests suggests that current multiparameter tests provide broadly equivalent risk information for the population of women with estrogen receptor (ER)-positive breast cancers. However, for the individual patient, tests may provide differing risk categorization and subtype information.

Dataset Information

All metrics are equal, but some metrics are more equal than others: A systematic search and review on the use of the term 'metric'.

Objective

Materials and methods

Results

Conclusion

Publications

All metrics are equal, but some metrics are more equal than others: A systematic search and review on the use of the term 'metric'.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets