Evaluating the Fit of Sequential G-DINA Model Using Limited-Information Measures.
Ontology highlight
ABSTRACT:
SUBMITTER: Ma W
PROVIDER: S-EPMC7174807 | biostudies-literature | 2020 May
REPOSITORIES: biostudies-literature
Publications
Evaluating the Fit of Sequential G-DINA Model Using Limited-Information Measures.
Applied psychological measurement 20190514 3
Limited-information fit measures appear to be promising in assessing the goodness-of-fit of dichotomous response cognitive diagnosis models (CDMs), but their performance has not been examined for polytomous response CDMs. This study investigates the performance of the <i>M</i> <sub>ord</sub> statistic and standardized root mean square residual (SRMSR) for an ordinal response CDM-the sequential generalized deterministic inputs, noisy "and" gate model. Simulation studies showed that the <i>M</i> < ...[more]
Similar Datasets
Project description:This research focuses on developing item-level fit checking procedures in the context of diagnostic classification models (DCMs), and more specifically for the "Deterministic Input; Noisy 'And' gate" (DINA) model. Although there is a growing body of literature discussing model fit checking methods for DCM, the item-level fit analysis is not adequately discussed in literature. This study intends to take an initiative to fill in this gap. Two approaches are proposed, one stems from classical goodness-of-fit test statistics coupled with the Expectation-Maximization algorithm for model estimation, and the other is the posterior predictive model checking (PPMC) method coupled with the Markov chain Monte Carlo estimation. For both approaches, the chi-square statistic and a power-divergence index are considered, along with Stone's method for considering uncertainty in latent attribute estimation. A simulation study with varying manipulated factors is carried out. Results show that both approaches are promising if Stone's method is imposed, but the classical goodness-of-fit approach has a much higher detection rate (i.e., proportion of misfit items that are correctly detected) than the PPMC method.
| S-EPMC5978514 | biostudies-literature
Project description:Fit indices are highly frequently used for assessing the goodness of fit of latent variable models. Most prominent fit indices, such as the root-mean-square error of approximation (RMSEA) or the comparative fit index (CFI), are based on a noncentrality parameter estimate derived from the model fit statistic. While a noncentrality parameter estimate is well suited for quantifying the amount of systematic error, the complex weighting function involved in its calculation makes indices derived from it challenging to interpret. Moreover, noncentrality-parameter-based fit indices yield systematically different values, depending on the indicators' level of measurement. For instance, RMSEA and CFI yield more favorable fit indices for models with categorical as compared to metric variables under otherwise identical conditions. In the present article, approaches for obtaining an approximation discrepancy estimate that is independent from any specific weighting function are considered. From these unweighted approximation error estimates, fit indices analogous to RMSEA and CFI are calculated and their finite sample properties are investigated using simulation studies. The results illustrate that the new fit indices consistently estimate their true value which, in contrast to other fit indices, is the same value for metric and categorical variables. Advantages with respect to interpretability are discussed and cutoff criteria for the new indices are considered.
| S-EPMC10188575 | biostudies-literature
Project description:Rapid development of intelligent information equipment accelerates the expansion of mobile social network. Speed of information spreading is gradually growing, there are lots of changes in the scale and mode of information spreading. But the basic communication network is not developed and not mature, when online information platforms breakdown sometimes it happens to be when important information appears. Therefore, the research is done to solve these occasion problems, help network information platform filter hot news and discuss the reason that hot news exists longer than other news in the Internet. In this paper, a multiple information propagation model incorporating both local information environment and people's limited attention is proposed based on Susceptible Infected Recovered (SIR) model. Two new concepts are introduced into the model: heat rate and popular rate, to measure the local information influence power and people's limited attention to information respectively, which are key factors determining node state transformation instead of fixed probability. In order to analyze the influence from limited attention, a situation is designed that several pieces of information are popular successively. The theoretical analysis shows that the early popular information gets more attention than the later popular information, and more attention makes it easier to spread. Besides, numerical simulation is conducted in both uniform network and scale-free network. The simulation results show that the early popular information is less vulnerable to the increase of information acceptance threshold and more sensitive to the decrease of information rejection threshold than the later popular information. Moreover, the model can also be used in the case of large amount of information transmission without adding too much complexity. Reasons are given in the research that the top hot news exists very much longer than the other ones, and latter news which have same influence as top news are hard to get the same focus. Meanwhile, results in the research can provide some ways for the other researches in the related fields. They also help related information platforms to filter and push news and referable strategies to maintain hot news.
| S-EPMC8496793 | biostudies-literature
Project description:The MC-DINA model is a cognitive diagnosis model (CDM) for multiple-choice items that was introduced by de la Torre (2009). The model extends the usual CDM in two basic ways: it allows for nominal responses instead of only dichotomous responses, and it allows skills to affect not only the choice of the correct response but also the choice of distractors. Here it is shown that the model can be re-expressed as a multinomial logit model with latent discrete predictors, that is, as a multinomial mixture model; a signal detection-like parameterization is also used. The reparameterization clarifies details about the structure and assumptions of the model, especially with respect to distractors, and helps to reveal parameter restrictions, which in turn have implications for psychological interpretations of the data and for issues with respect to statistical estimation. The approach suggests parsimonious models that are useful for practical applications, particularly for small sample sizes. The restrictions are shown to appear for items from the TIMSS 2007 fourth grade exam.
| S-EPMC11897991 | biostudies-literature
Project description:BackgroundThere are several frameworks that attempt to address the challenges of evaluation of health information systems by offering models, methods, and guidelines about what to evaluate, how to evaluate, and how to report the evaluation results. Model-based evaluation frameworks usually suggest universally applicable evaluation aspects but do not consider case-specific aspects. On the other hand, evaluation frameworks that are case specific, by eliciting user requirements, limit their output to the evaluation aspects suggested by the users in the early phases of system development. In addition, these case-specific approaches extract different sets of evaluation aspects from each case, making it challenging to collectively compare, unify, or aggregate the evaluation of a set of heterogeneous health information systems.ObjectivesThe aim of this paper is to find a method capable of suggesting evaluation aspects for a set of one or more health information systems-whether similar or heterogeneous-by organizing, unifying, and aggregating the quality attributes extracted from those systems and from an external evaluation framework.MethodsOn the basis of the available literature in semantic networks and ontologies, a method (called Unified eValuation using Ontology; UVON) was developed that can organize, unify, and aggregate the quality attributes of several health information systems into a tree-style ontology structure. The method was extended to integrate its generated ontology with the evaluation aspects suggested by model-based evaluation frameworks. An approach was developed to extract evaluation aspects from the ontology that also considers evaluation case practicalities such as the maximum number of evaluation aspects to be measured or their required degree of specificity. The method was applied and tested in Future Internet Social and Technological Alignment Research (FI-STAR), a project of 7 cloud-based eHealth applications that were developed and deployed across European Union countries.ResultsThe relevance of the evaluation aspects created by the UVON method for the FI-STAR project was validated by the corresponding stakeholders of each case. These evaluation aspects were extracted from a UVON-generated ontology structure that reflects both the internally declared required quality attributes in the 7 eHealth applications of the FI-STAR project and the evaluation aspects recommended by the Model for ASsessment of Telemedicine applications (MAST) evaluation framework. The extracted evaluation aspects were used to create questionnaires (for the corresponding patients and health professionals) to evaluate each individual case and the whole of the FI-STAR project.ConclusionsThe UVON method can provide a relevant set of evaluation aspects for a heterogeneous set of health information systems by organizing, unifying, and aggregating the quality attributes through ontological structures. Those quality attributes can be either suggested by evaluation models or elicited from the stakeholders of those systems in the form of system requirements. The method continues to be systematic, context sensitive, and relevant across a heterogeneous set of health information systems.
| S-EPMC4929349 | biostudies-literature
Project description:Given the long follow-up periods that are often required for treatment or intervention studies, the potential to use surrogate markers to decrease the required follow-up time is a very attractive goal. However, previous studies have shown that using inadequate markers or making inappropriate assumptions about the relationship between the primary outcome and surrogate marker can lead to inaccurate conclusions regarding the treatment effect. Currently available methods for identifying and validating surrogate markers tend to rely on restrictive model assumptions and/or focus on uncensored outcomes. The ability to use such methods in practice when the primary outcome of interest is a time-to-event outcome is difficult because of censoring and missing surrogate information among those who experience the primary outcome before surrogate marker measurement. In this paper, we propose a novel definition of the proportion of treatment effect explained by surrogate information collected up to a specified time in the setting of a time-to-event primary outcome. Our proposed approach accommodates a setting where individuals may experience the primary outcome before the surrogate marker is measured. We propose a robust non-parametric procedure to estimate the defined quantity using censored data and use a perturbation-resampling procedure for variance estimation. Simulation studies demonstrate that the proposed procedures perform well in finite samples. We illustrate the proposed procedures by investigating two potential surrogate markers for diabetes using data from the Diabetes Prevention Program. Copyright © 2017 John Wiley & Sons, Ltd.
| S-EPMC5413393 | biostudies-literature
Project description:This article provides a review of software packages for fitting maximum likelihood estimation of the deterministic input, noisy “and” gate (DINA) model. Six software packages—flexMIRT, Latent GOLD, mdltm, Mplus, OxEdit, and R—are considered. Each package is reviewed regarding data manipulation, statistical capabilities, output, and documentation. The results of these packages are compared using a sample data set and a Q-matrix. The article aims to give the reader a summary of the different capabilities of each package.
| S-EPMC7003186 | biostudies-literature
Project description:With the increasing demanding for precision of test feedback, cognitive diagnosis models have attracted more and more attention to fine classify students whether has mastered some skills. The purpose of this paper is to propose a highly effective Pólya-Gamma Gibbs sampling algorithm (Polson et al., 2013) based on auxiliary variables to estimate the deterministic inputs, noisy "and" gate model (DINA) model that have been widely used in cognitive diagnosis study. The new algorithm avoids the Metropolis-Hastings algorithm boring adjustment the turning parameters to achieve an appropriate acceptance probability. Four simulation studies are conducted and a detailed analysis of fraction subtraction data is carried out to further illustrate the proposed methodology.
| S-EPMC7076190 | biostudies-literature
Project description:Measurement of language atypicalities in Autism Spectrum Disorder (ASD) is cumbersome and costly. Better language outcome measures are needed. Using language transcripts, we generated Automated Language Measures (ALMs) and tested their validity. 169 participants (96 ASD, 28 TD, 45 ADHD) ages 7 to 17 were evaluated with the Autism Diagnostic Observation Schedule. Transcripts of one task were analyzed to generate seven ALMs: mean length of utterance in morphemes, number of different word roots (NDWR), um proportion, content maze proportion, unintelligible proportion, c-units per minute, and repetition proportion. With the exception of repetition proportion (p [Formula: see text]), nonparametric ANOVAs showed significant group differences (p[Formula: see text]). The TD and ADHD groups did not differ from each other in post-hoc analyses. With the exception of NDWR, the ASD group showed significantly (p[Formula: see text]) lower scores than both comparison groups. The ALMs were correlated with standardized clinical and language evaluations of ASD. In age- and IQ-adjusted logistic regression analyses, four ALMs significantly predicted ASD status with satisfactory accuracy (67.9-75.5%). When ALMs were combined together, accuracy improved to 82.4%. These ALMs offer a promising approach for generating novel outcome measures.
| S-EPMC8155086 | biostudies-literature
Project description:Selecting pretrained models for image classification often involves computationally intensive finetuning. This study addresses a research gap in the standardized evaluation of transferability scores, which could simplify model selection by ranking pretrained models without exhaustive finetuning. The motivation is to reduce the computational burden of model selection through a consistent approach that guides practitioners in balancing accuracy and efficiency across tasks. This study evaluates 14 transferability scores on 11 benchmark datasets. It includes both Convolutional Neural Network (CNN) and Vision Transformer (ViT) models and ensures consistency in experimental conditions to counter the variability in previous research. Key findings reveal significant variability in score effectiveness based on dataset characteristics (e.g., fine-grained versus coarse-grained classes) and model architectures. ViT models generally show superior transferability, especially for fine-grained datasets. While no single score is best in all cases, some scores excel in specific contexts. In addition to predictive accuracy, the study also evaluates computational efficiency and identifies scores that are suitable for resource-constrained scenarios. This research provides insights for selecting appropriate transferability scores to optimize model selection strategies to facilitate efficient deployment in practice.
| S-EPMC11618499 | biostudies-literature