Dataset Information

A cognitive model for aggregating people's rankings.

ABSTRACT: We develop a cognitive modeling approach, motivated by classic theories of knowledge representation and judgment from psychology, for combining people's rankings of items. The model makes simple assumptions about how individual differences in knowledge lead to observed ranking data in behavioral tasks. We implement the cognitive model as a Bayesian graphical model, and use computational sampling to infer an aggregate ranking and measures of the individual expertise. Applications of the model to 23 data sets, dealing with general knowledge and prediction tasks, show that the model performs well in producing an aggregate ranking that is often close to the ground truth and, as in the "wisdom of the crowd" effect, usually performs better than most of individuals. We also present some evidence that the model outperforms the traditional statistical Borda count method, and that the model is able to infer people's relative expertise surprisingly well without knowing the ground truth. We discuss the advantages of the cognitive modeling approach to combining ranking data, and in wisdom of the crowd research generally, as well as highlighting a number of potential directions for future model development.

SUBMITTER: Lee MD

PROVIDER: S-EPMC4015936 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A cognitive model for aggregating people's rankings.

Lee Michael D MD Steyvers Mark M Miller Brent B

PloS one 20140509 5

We develop a cognitive modeling approach, motivated by classic theories of knowledge representation and judgment from psychology, for combining people's rankings of items. The model makes simple assumptions about how individual differences in knowledge lead to observed ranking data in behavioral tasks. We implement the cognitive model as a Bayesian graphical model, and use computational sampling to infer an aggregate ranking and measures of the individual expertise. Applications of the model to ...[more]

PMID: 24816733

Similar Datasets

Project description:BackgroundSelf-management abilities seem to be important for the cognitive functioning of older people, especially those who are frail. We investigated relationships between broad self-management abilities (initiative taking, investment behavior, resource variety, resource multifunctionality, self-efficacy, and positive frame of mind) and cognitive functioning among frail older people while controlling for background characteristics (sex, age, marital status, and educational level).MethodSurvey data were collected from mid-2014 to mid-2015 from community-dwelling frail older people residing in North Brabant, the Netherlands. We measured cognitive functioning with the 12-item Mini-Mental State Examination (MMSE-12) and self-management abilities with the short version of the Self-Management Ability Scale (SMAS-S).ResultsIn total, 588 of 834 potential participants were willing to participate (70.5% response rate). The mean age was 82.33 ± 5.19 and the majority (68.5%) of respondents were female. About one-third (38.4%) of respondents had low educational levels and 61.7% lived alone. Mean MMSE-12 and SMAS-S scores were 9.68 ± 2.10 and 3.70 ± 0.88, respectively. Bivariate analyses showed that all six self-management abilities were related positively to cognitive functioning. Multivariate analyses with adjustment for background characteristics (sex, age, marital status, and educational level) showed that cognitive functioning was associated positively with initiative taking (β = 0.23, p = 0.030) and investment behavior (β = 0.24, p = 0.030) among community-dwelling frail older people.ConclusionsThis study clearly showed that a repertoire of broad self-management abilities is related to cognitive functioning among community-dwelling frail older people. Initiative taking and investment behavior seem to be especially important. These findings are of interest in a time of populational aging and an increasing number of older people dealing with cognitive problems. Preventive investments in (older) people's self-management abilities are expected to be beneficial for their cognitive functioning in the long term.

Project description:ImportanceThe Centers for Medicare & Medicaid Services (CMS) Five-Star measure for nursing homes is designed with input from expert panels about the importance of multiple quality indicators. Consumers may assign different values to these indicators, creating different 5-star ratings.ObjectiveTo compare nursing homes' rankings based on the CMS Five-Star measure with rankings based on consumers' judgment about the importance of the same quality indicators.Design, setting, and participantsIn this quality improvement study, CMS Five-Star data were linked with a measure calculated from CMS quality indicators and consumer values obtained from a national survey. Data covered the last quarter of 2016 and the first three quarters of 2017. The study included 10 676 nursing homes, comprising 69.8% of those with reported Five-Star measures. The national survey included adults, either nursing home residents or their family members who reported being familiar with the quality of care their relative received. Data analysis was performed from January 2019 to February 2020.Main outcomes and measuresThe contingent valuation method was administered via the survey to obtain consumers' relative values of the quality indicators, and statistical analyses were used to create the contingent valuation measure. Agreement in nursing home rankings was assessed using the Five-Star measure, which is based on weights developed by expert panels, compared with rankings based on the contingent valuation measure.ResultsAmong 10 676 study nursing homes with a mean (SD) of 119.4 (59.4) beds, 7845 (73.5%) were for profit, 6424 (61.8%) were part of a chain, and 8009 (75.0%) were urban. The 4310 survey respondents (mean [SD] age, 39.9 [15.6] years; 1143 [26.5%] men; 3448 [80%] white) included mostly family members (3879 participants [90.0%]). The Pearson correlation coefficient (0.65) and weighted κ statistics (0.48) indicated only moderate agreement between ranking of nursing homes' performance by the 2 measures and disagreement on ranking for approximately one-half of the nursing homes.Conclusions and relevanceCurrent nursing home report cards might not reflect consumers' values and the relative importance consumers place on each of the quality indicators that compose the overall Five-Star measure. Quality report cards might be more relevant to consumers by augmenting the Five-Star measure with a measure reflecting consumers' preferences. It is unknown whether these conclusions are generalizable to other report cards, such as Hospital and Home Health Compare, without conducting similar studies for these report cards.

Dataset Information

A cognitive model for aggregating people's rankings.

Publications

A cognitive model for aggregating people's rankings.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets