Dataset Information

Transitivity Violations Undermine Rating Scales in Motivation Research.

ABSTRACT: Measures of psychological attributes, such as motivation, typically involve rating scales, assuming that an attribute can be ordered. If an attribute has an ordinal structure, its levels stand in ordinal relations to one another, and these must be transitive. We tested if transitivity is preserved when people compare different motives in terms of their importance to learning. We found transitivity violations in both strict (Study 1) and non-strict (Study 2) orderings in about half of the participants. Nevertheless, based on the distribution of such violations, we conclude that an ordinal structure of motivation can be found, but only when levels of motives differ noticeably. As the levels become subjectively similar, transitivity is not preserved, and the ordinal structure cannot be justified even in non-strict ordering. The findings question the mainstream practice of measuring psychological attributes before their structure is properly explored.

SUBMITTER: Tyumeneva Y

PROVIDER: S-EPMC8514613 | biostudies-literature | 2021

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Transitivity Violations Undermine Rating Scales in Motivation Research.

Tyumeneva Yulia Y Vergeles Kseniya K

Frontiers in psychology 20210930

Measures of psychological attributes, such as motivation, typically involve rating scales, assuming that an attribute can be ordered. If an attribute has an ordinal structure, its levels stand in ordinal relations to one another, and these must be transitive. We tested if transitivity is preserved when people compare different motives in terms of their importance to learning. We found transitivity violations in both strict (Study 1) and non-strict (Study 2) orderings in about half of the partici ...[more]

PMID: 34658985

Similar Datasets

Project description:Rating scales are popular methods for generating quantitative data directly by persons rather than automated technologies. But scholars increasingly challenge their foundations. This article contributes epistemological and methodological analyses of the processes involved in person-generated quantification. They are crucial for measurement because data analyses can reveal information about study phenomena only if relevant properties were encoded systematically in the data. The Transdisciplinary Philosophy-of-Science Paradigm for Research on Individuals (TPS-Paradigm) is applied to explore psychological and social-science concepts of measurement and quantification, including representational measurement theory, psychometric theories and their precursors in psychophysics. These are compared to theories from metrology specifying object-dependence of measurement processes and subject-independence of outcomes as key criteria, which allow tracing data to the instances measured and the ways they were quantified. Separate histories notwithstanding, the article's basic premise is that general principles of scientific measurement and quantification should apply to all sciences. It elaborates principles by which these metrological criteria can be implemented also in psychology and social sciences, while considering their research objects' peculiarities. Application of these principles is illustrated by quantifications of individual-specific behaviors ('personality'). The demands rating methods impose on data-generating persons are deconstructed and compared with the demands involved in other quantitative methods (e.g., ethological observations). These analyses highlight problematic requirements for raters. Rating methods sufficiently specify neither the empirical study phenomena nor the symbolic systems used as data nor rules of assignment between them. Instead, pronounced individual differences in raters' interpretation and use of items and scales indicate considerable subjectivity in data generation. Together with recoding scale categories into numbers, this introduces a twofold break in the traceability of rating data, compromising interpretability of findings. These insights question common reliability and validity concepts for ratings and provide novel explanations for replicability problems. Specifically, rating methods standardize only data formats but not the actual data generation. Measurement requires data generation processes to be adapted to the study phenomena's properties and the measurement-executing persons' abilities and interpretations, rather than to numerical outcome formats facilitating statistical analyses. Researchers must finally investigate how people actually generate ratings to specify the representational systems underlying rating data.

Project description:Upper and lower gastrointestinal dysautonomia symptoms (GIDS)--sialorrhea, dysphagia, and constipation are common in Parkinson's disease (PD) and often socially as well as physically disabling for patients. Available invasive quantitative measures for assessing these symptoms and their response to therapy are time-consuming, require specialized equipment, can cause patient discomfort and present patients with risk. The Movement Disorders Society commissioned a task force to assess available clinical rating scales, critique their clinimetric properties, and make recommendations regarding their clinical utility. Six clinical researchers and a biostatistician systematically searched the literature for scales of sialorrhea, dysphagia, and constipation, evaluated the scales' previous use, performance parameters, and quality of validation data (if available). A scale was designated "Recommended" if the scale was used in clinical studies beyond the group that developed it, has been specifically used in PD reports, and clinimetric studies have established that it is a valid, reliable, and sensitive. "Suggested" scales met at least part of the above criteria, but fell short of meeting all. Based on the systematic review, scales for individual symptoms of sialorrhea, dysphagia, and constipation were identified along with three global scales that include these symptoms in the context of assessing dysautonomia or nonmotor symptoms. Three sialorrhea scales met criteria for Suggested: Drooling Severity and Frequency Scale (DSFS), Drooling Rating Scale, and Sialorrhea Clinical Scale for PD (SCS-PD). Two dysphagia scales, the Swallowing Disturbance Questionnaire (SDQ) and Dysphagia-Specific Quality of Life (SWAL-QOL), met criteria for Suggested. Although Rome III constipation module is widely accepted in the gastroenterology community, and the earlier version from the Rome II criteria has been used in a single study of PD patients, neither met criteria for Suggested or Recommended. Among the global scales, the Scales for Outcomes in PD-Autonomic (SCOPA-AUT) and Nonmotor Symptoms Questionnaire for PD (NMSQuest) both met criteria for Recommended, and the Nonmotor Symptoms Scale (NMSS) met criteria for Suggested; however, none specifically focuses on the target gastrointestinal symptoms (sialorrhea, dysphagia, and constipation) of this report. A very small number of rating scales have been applied to studies of gastrointestinal-related dysautonomia in PD. Only two scales met "Recommended" criteria and neither focuses specifically on the symptoms of sialorrhea, dysphagia, and constipation. Further scale testing in PD among the scales that focus on these symptoms is warranted, and no new scales are needed until the available scales are fully tested clinimetrically.

Dataset Information

Transitivity Violations Undermine Rating Scales in Motivation Research.

Publications

Transitivity Violations Undermine Rating Scales in Motivation Research.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets