Dataset Information

How do search systems impact systematic searching? A qualitative study.

ABSTRACT:

Objective

Systematic reviews and other evidence synthesis projects require systematic search methods. Search systems require several essential attributes to support systematic searching; however, many systems used in evidence synthesis fail to meet one or more of these requirements. I undertook a qualitative study to examine the effects of these limitations on systematic searching and how searchers select information sources for evidence synthesis projects.

Methods

Qualitative data were collected from interviews with twelve systematic searchers. Data were analyzed using reflexive thematic analysis.

Results

I used thematic analysis to identify two key themes relating to search systems: systems shape search processes, and systematic searching occurs within the information market. Many systems required for systematic reviews, in particular sources of unpublished studies, are not designed for systematic searching. Participants described various workarounds for the limitations they encounter in these systems. Economic factors influence searchers' selection of sources to search, as well as the degree to which vendors prioritize these users.

Conclusion

Interviews with systematic searchers suggest priorities for improving search systems, and barriers to improvement that must be overcome. Vendors must understand the unique requirements of systematic searching and recognize systematic searchers as a distinct group of users. Better interfaces and improved functionality will result in more efficient evidence synthesis.

SUBMITTER: Hickner A

PROVIDER: S-EPMC10621724 | biostudies-literature | 2023 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

How do search systems impact systematic searching? A qualitative study.

Hickner Andy A

Journal of the Medical Library Association : JMLA 20231001 4

<h4>Objective</h4>Systematic reviews and other evidence synthesis projects require systematic search methods. Search systems require several essential attributes to support systematic searching; however, many systems used in evidence synthesis fail to meet one or more of these requirements. I undertook a qualitative study to examine the effects of these limitations on systematic searching and how searchers select information sources for evidence synthesis projects.<h4>Methods</h4>Qualitative dat ...[more]

PMID: 37928121

Similar Datasets

Project description:Qualitative systematic reviews or qualitative evidence syntheses (QES) are increasingly recognised as a way to enhance the value of systematic reviews (SRs) of clinical trials. They can explain the mechanisms by which interventions, evaluated within trials, might achieve their effect. They can investigate differences in effects between different population groups. They can identify which outcomes are most important to patients, carers, health professionals and other stakeholders. QES can explore the impact of acceptance, feasibility, meaningfulness and implementation-related factors within a real world setting and thus contribute to the design and further refinement of future interventions. To produce valid, reliable and meaningful QES requires systematic identification of relevant qualitative evidence. Although the methodologies of QES, including methods for information retrieval, are well-documented, little empirical evidence exists to inform their conduct and reporting.This structured methodological overview examines papers on searching for qualitative research identified from the Cochrane Qualitative and Implementation Methods Group Methodology Register and from citation searches of 15 key papers.A single reviewer reviewed 1299 references. Papers reporting methodological guidance, use of innovative methodologies or empirical studies of retrieval methods were categorised under eight topical headings: overviews and methodological guidance, sampling, sources, structured questions, search procedures, search strategies and filters, supplementary strategies and standards.This structured overview presents a contemporaneous view of information retrieval for qualitative research and identifies a future research agenda. This review concludes that poor empirical evidence underpins current information practice in information retrieval of qualitative research. A trend towards improved transparency of search methods and further evaluation of key search procedures offers the prospect of rapid development of search methods.

Project description:BackgroundRealist methodologies are increasingly being used to evaluate complex interventions in health and social care. Programme theory (ideas and assumptions of how a particular intervention works) development is the first step in a realist evaluation or a realist synthesis, with literature reviews providing important evidence to support this. Deciding how to search for programme theories is challenging and there is limited guidance available. Using an example of identifying programme theories for a realist evaluation of Pressure Ulcer Risk Assessment Instruments in clinical practice, the authors explore and compare several different approaches to literature searching and highlight important methodological considerations for those embarking on a programme theory review.MethodsWe compared the performance of an academic database search with a simple Google search and developed an optimised search strategy for the identification primary references (i.e. documents providing the clearest examples of programme theories) associated with the use of Pressure Ulcer Risk Assessment Instruments (PU-RAIs). We identified the number of primary references and the total number of references retrieved per source. We then calculated the number needed to read (NNR) expressed as the total number of titles and abstracts screened to identify one relevant reference from each source.ResultsThe academic database search (comprising CINAHL, The Cochrane Library, EMBASE, HMIC, Medline) identified 2 /10 primary references with a NNR of 1395.The Google search identified 7/10 primary references with a NNR of 10.1. The combined NNR was 286.3. The optimised search combining Google and CINAHL identified 10/10 primary references with a NNR of 40.2.ConclusionThe striking difference between the efficiency of the review's academic database and Google searches in finding relevant references prompted an in-depth comparison of the two types of search. The findings indicate the importance of including grey literature sources such as Google in this particular programme theory search, while acknowledging the need for transparency of methods. Further research is needed to facilitate improved guidance for programme theory searches to enhance practice in the realist field and to save researcher time and therefore resource.

Project description:BACKGROUND:Breastfeeding has many benefits for newborns, mothers, and the wider society. The World Health Organization recommends mothers to feed newborns exclusively with breastmilk for the first 6 months after birth, but breastfeeding rates in many countries fail to align with the recommendations because of various barriers. Breastfeeding success is associated with a number of determinants, such as self-efficacy, intention to breastfeed, and attitudes toward breastfeeding. Information and communication technology (ICT) has been leveraged to support breastfeeding by means of improving knowledge or providing practical supports in different maternal stages. Previous reviews have examined and summarized the effectiveness and credibility of interventions; however, no review has been done from a human-computer interaction perspective that is concerned with novel interaction techniques and the perspective of end users. OBJECTIVE:The objective of this review was to provide a comprehensive overview of existing digital interventions that support breastfeeding by investigating systems' objective, technology design, validation process, and quality attributes, both in terms of clinical parameters as well as usability and user experience. METHODS:A systematic search was conducted in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines in the following libraries: PubMed, Science Direct, Association for Computing Machinery Digital Library (ACM Digital Library), and Institute of Electrical and Electronics Engineers Xplore (IEEE Xplore). RESULTS:A total of 35 papers discussing 30 interventions were included. The main goals of these interventions were organized into 4 categories: breastfeeding education (n=12), breastfeeding promotion (n=8), communication support (n=6), and daily practical support (n=4). Of the interventions, 13 target mothers in the postnatal period. Most interventions come in forms of client communication systems (n=18), which frequently leverage Web technologies, text message, and mobile apps to provide breastfeeding support. Systems predominantly focus on mothers; validation strategies were rather heterogeneous, with 12 user studies concerning usability and user experience and 18 clinical validation studies focusing on the effects of the interventions on breastfeeding determinants; 5 papers did not report results. Generally, straightforward systems (eg, communication tools or Web-based solutions) seem to be more effective than complex interventions (eg, games). CONCLUSIONS:Existing information and communication systems offer effective means of improving breastfeeding outcomes, but they do not address all relevant periods in parenthood (eg, the antenatal period) and often do not involve important stakeholders, such as partners. There is an opportunity to leverage more complex technical systems to open up avenues for the broader design of ICT to support breastfeeding; however, considering evaluation outcomes of existing support systems of higher complexity, such systems need to be designed with care.

Project description:BackgroundThe value of citation searches as part of the systematic review process is currently unknown. While the major guides to conducting systematic reviews state that citation searching should be carried out in addition to searching bibliographic databases there are still few studies in the literature that support this view. Rather than using a predefined search strategy to retrieve studies, citation searching uses known relevant papers to identify further papers.MethodsWe describe a case study about the effectiveness of using the citation sources Google Scholar, Scopus, Web of Science and OVIDSP MEDLINE to identify records for inclusion in a systematic review.We used the 40 included studies identified by traditional database searches from one systematic review of interventions for multiple risk behaviours. We searched for each of the included studies in the four citation sources to retrieve the details of all papers that have cited these studies.We carried out two analyses; the first was to examine the overlap between the four citation sources to identify which citation tool was the most useful; the second was to investigate whether the citation searches identified any relevant records in addition to those retrieved by the original database searches.ResultsThe highest number of citations was retrieved from Google Scholar (1680), followed by Scopus (1173), then Web of Science (1095) and lastly OVIDSP (213). To retrieve all the records identified by the citation tracking searching all four resources was required. Google Scholar identified the highest number of unique citations.The citation tracking identified 9 studies that met the review's inclusion criteria. Eight of these had already been identified by the traditional databases searches and identified in the screening process while the ninth was not available in any of the databases when the original searches were carried out. It would, however, have been identified by two of the database search strategies if searches had been carried out later.ConclusionsBased on the results from this investigation, citation searching as a supplementary search method for systematic reviews may not be the best use of valuable time and resources. It would be useful to verify these findings in other reviews.

Project description:Objective To evaluate the impact of searching clinical trial registries in systematic reviews.Design Methodological systematic review and reanalyses of meta-analyses.Data sources Medline was searched to identify systematic reviews of randomised controlled trials (RCTs) assessing pharmaceutical treatments published between June 2014 and January 2015. For all systematic reviews that did not report a trial registry search but reported the information to perform it, the World Health Organization International Trials Registry Platform (WHO ICTRP search portal) was searched for completed or terminated RCTs not originally included in the systematic review.Data extraction For each systematic review, two researchers independently extracted the outcomes analysed, the number of patients included, and the treatment effect estimated. For each RCT identified, two researchers independently determined whether the results were available (ie, posted, published, or available on the sponsor website) and extracted the data. When additional data were retrieved, we reanalysed meta-analyses and calculated the weight of the additional RCTs and the change in summary statistics by comparison with the original meta-analysis.Results Among 223 selected systematic reviews, 116 (52%) did not report a search of trial registries; 21 of these did not report the information to perform the search (key words, search date). A search was performed for 95 systematic reviews; for 54 (57%), no additional RCTs were found and for 41 (43%) 122 additional RCTs were identified. The search allowed for increasing the number of patients by more than 10% in 19 systematic reviews, 20% in 10, 30% in seven, and 50% in four. Moreover, 63 RCTs had results available; the results for 45 could be included in a meta-analysis. 14 systematic reviews including 45 RCTs were reanalysed. The weight of the additional RCTs in the recalculated meta-analyses ranged from 0% to 58% and was greater than 10% in five of 14 systematic reviews, 20% in three, and 50% in one. The change in summary statistics ranged from 0% to 29% and was greater than 10% for five of 14 systematic reviews and greater than 20% for two. However, none of the changes to summary effect estimates led to a qualitative change in the interpretation of the results once the new trials were added.Conclusions Trial registries are an important source for identifying additional RCTs. The additional number of RCTs and patients included if a search were performed varied across systematic reviews.

Dataset Information

How do search systems impact systematic searching? A qualitative study.

Objective

Methods

Results

Conclusion

Publications

How do search systems impact systematic searching? A qualitative study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets