Past Meetup

Systematic Review in Software Engineering

This Meetup is past

4 people went


Jorge Biolchini, Paula Gomes Mian, Ana Candida Cruz Natali and Guilherme Horta Travasssos - "Systematic Review in Software Engineering ("


The term Systematic Review (SR) is used to refer to a specific methodology of research, developed in order to gather and evaluate the available evidence pertaining to a focused topic.In contrast to the usual process of literature review, unsystematically conducted whenever one starts a particular investigation, a SR is developed, as the term denotes, in a formal and systematic way. This means that the research conduction process of a systematic type of review follows a very well defined and strict sequence of methodological steps, according to an aprioristically developed protocol. This instrument is constructed around a central issue, which represents the core of the investigation, and which is expressed by using specific concepts and terms, that must be addressed towards information related to a specific, pre-defined, focused, and structured question. The methodological steps, the strategies to retrieve the evidence, the focus of the question are explicitly defined, so that other professionals can reproduce the same protocol and also be able to judge about the adequacy of the chosen standards for the case.Synonyms of this methodology that are to be found in the literature include the following terms: overview, research review, research synthesis, research integration, systematic overview, systematic research synthesis, integrative research review, and integrative review.The type of acceptable evidence to be gathered in a systematic review is stated beforehand. The retrieved evidence is thoroughly reviewed, comparable to other types of evidence previously and elsewhere retrieved.The evidence data are normalized in such a way as to make results from different studies comparable, in terms of their magnitude of effect, even when they are presented in diverse ways but related to compatible concepts. It is then possible, e.g., to compare studies which evidence is expressed by absolute risk reduction with others where it is expressed by relative risk.Besides comparing results of individual studies, different kinds of syntheses can be done. The election mode allows the researcher to look for each study separately and counting them as “votes” about the question focus. For instance, in a specific SR conducted in the field of medicine, the researcher could find that, among 35 valid studies, 29 showed a positive result, while 5 showed no result, and one study showed a negative result. Internal comparison of studies, based on their specific parameters, can show contrasts and other kinds of differences that may elucidate distinct aspects of the question. In the same example, one could find that the negative effect must be due to a different dosage scheme, while the five studies that showed no result were conducted in subjects that had a different age distribution in comparison to the 29 positive ones.Another type of research synthesis is known as meta-analysis, where the original individual studies are treated as if they were parts of one larger study, by having their data pooled together in one single and final result that summarizes the whole evidence. By selecting studies that are compatible in their quality level, and by taking strict care with their specific details, this methodological procedure can produce evidence as well as reveal aspects that the original studies are not individually able to elucidate. For instance, meta-analysis may prove that the results are statistically significant when small studies give inconclusive results with large confidence intervals. Besides that, when conflicting results arise from different individual studies, meta-analysis may reconcile the data in a synthetic result, while each individual study can then be weighted and compared with it, so that other kinds of conclusions might be derived from these discrepancies.