EDU Graduate Research

Question: The topic that a review aims to address should be clear in the form of a concise question. This is an indication that the review was systematic and therefore eligible for inclusion in a study.

Sources: May include general purpose databases (e.g., Ebsco, Proquest), search engines (e.g., Google), conference proceedings, book chapters or books, Provide a careful selection of appropriate and multiple databases so that the risk of introducing a database bias is minimized and the yield of relevant studies is maximized. According to Rothstein et al (2005), "publication bias is the term for what occurs whenever the research that appears in the published literature is systematically unrepresentative of the population of completed studies". When appraising a review, the reader should determine whether the authors have made a serious attempt to locate unpublished studies, including conference proceedings and unpublished theses and dissertations. 

Scope: Geographic constraints refer to restrictions imposed on the search based on geographic region. Imposing geographic constraints will almost always yield a biased sample if the particular questions at hand warrants that all evidence be considered regardless of geographic origin, especially in times of globalization. There are some examples where this is acceptable. For example, a systematic review is restricted to a particular subpopulation that is only found in a certain region (e.g., American Indians), so it is therefore unlikely that studies are published elsewhere. 

Temporal restraints refer to limits imposed on a search in terms of time. For example, it may be stated that only studies dated between 2000 (starting date) and 2010 (end date) are being considered for inclusion. Readers should be looking for this information in the methods section where the procedures for locating studies are described. 

Selection principles include any kind of editorial criteria (e.g., type of design) used in accepting or rejecting studies to be reviewed other than those discussed above. As a reader, one would want to see clearly stated criteria for inclusion and exclusion along with rationales. 

Reference

Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication bias in meta-analysis. Publication bias in meta-analysis: Prevention, assessment and adjustments, 1-7.