Sort and evaluate your search results
CONTEXT
Being able to handle and systematically sort a quantity of results/literature on the basis of a literature search is a central academic skill.
Students must argue for their choice of sources and create transparency around their sorting/selection process as well as around the preceding information search.
In an academic context it is not enough to include a source simply because it touches on the topic. It is also central to be able to make it credible to the reader that the student has gained an overview of the literature in the field, and can state the arguments underlying the selection of the sources,
OUTCOME
The student acquires skills in systematically sorting literature from a larger set of search results, based on two phases.
The student becomes able to argue for the process around the sorting/selection of the relevant literature for the assignment. The student becomes familiar with digital tools for supporting the screening/sorting process and for depicting this process as an appendix to the assignment.
CONTENT AND STEPS OF THE ACTIVITY
The activity goes through how students can sort their results systematically using a step-by-step screening model.
The activity refers to digital tools that can ensure transparency in the selection process and can support collaboration on the screening process, as well as flowcharts that can be used to depict/document the sorting process.
BEFORE THE ACTIVITY
Students have carried out a thorough literature search on the basis of the question they want answered, including searching in the subject's relevant databases.
Students have a larger quantity of literature, possibly in the form of a RIS file from bibliographic databases, which they need to sort systematically, with a view to selecting core documents for their current project..
Students formulate criteria that form the basis for inclusion and exclusion of sources for their current assignment/study.
See examples of inclusion/exclusion i article from Sygeplejersken 2008′
THE ACTIVITY STEP BY STEP
1. The collected material/documents are counted and stated at the top of the flowchart, with an indication of the sources/databases searched see example and make your own which is to be filled in during the activity.
1.a Software can be used for collaboration, management and support (including “blinding”) of the process, e.g. using the open source program Rayyan. The records are imported into Rayyan as a file (e.g. in RIS format) from databases or a reference management program.
2. If several databases have been searched and/or documents collected from other sources, unique documents are counted by filtering out duplicates and this number is stated in the flowchart.
2.a Duplicate checking can be supported digitally using Rayyan or reference management software.
3. The documents are reviewed by reading titles and abstract/summary and a number are excluded. In the flowchart the excluded and included documents are counted – with an indication of the arguments for exclusion (possibly the same as at step 3).
3.a Using Rayyan, sets of “maybe” can be collected – where you have to discuss inclusion or exclusion with a collaborator. Here you can also specify terms you would like highlighted in the title/abstract for faster sorting. On the basis of 50 screened records the program can produce a machine rating of the remaining documents – which can support (or replace) manual screening.
4. The selected documents are reviewed in fulltext, e.g. by skimming the articles' methods section and conclusion (for books/reports, the introduction and conclusion). In the flowchart the excluded and included documents are counted – with an indication of the arguments for exclusion (possibly the same as at step 3)
4.a If Rayyan is used, this screening round can also be documented with arguments (reason) for exclusion in the program.
NB: For systematic reviews within the health field, individual studies are quality-assessed with appraisal tools for specific study designs (e.g. qualitative studies). See example
AFTER THE ACTIVITY
The flowchart is attached to the assignment as an appendix and is referred to in the methods section, or wherever transparency around the selection process is wanted.
MATERIALS AND PREPARATION
| Preparation | Materials |
|---|---|
| The teacher must familiarise themselves with a digital tool for screening literature, e.g. Rayyan, and get to know the flowchart. | Flowchart with an indication of the sources/databases searched see example |
| The teacher must be able to export files between various databases and programs. | The program Rayyan. |
| Students must have carried out a thorough literature search, including searching in the subject's relevant databases. | A reference management program such as Zotero or Mendeley. See Danish guidesr |
| Students must have a larger quantity of literature, possibly in the form of a RIS file from bibliographic databases. | Appraisal tools for specific study designs (e.g. qualitative studies). See example |
| Students must formulate clear inclusion and exclusion criteria to qualify the selection process. See e.g. i article from Sygeplejersken 2008 |
CREATED BY
Maria Viftrup Schneider, VIA Bibliotek.
Librarian.
EXTERNAL RESOURCES
Definition of inclusion and exclusion criteria.
See examples of inclusion/exclusion i article from Sygeplejersken 2008
Course in the Reference Management Tool Zotero
Crash course in Zotero (version 7)
THE STAK PROJECT
This learning element is an edited version of a learning pattern from the research project STAK (Studerendes Akademiske Digitale Kompetencer).
Through a particular method, the project developed a number of learning patterns/paths, especially within information literacy. The elements are aimed
primarily at teachers, either as inspiration or to be integrated directly into teaching.
If you are curious about the method, you can see the original research website at: https://web.archive.org/web/20251118005249/https://www.stak.dk/