Author: What Works Clearinghouse (ED)Publisher:
ISBN:
Category :
Languages : en
Pages : 130
Book Description
The What Works Clearinghouse (WWC) systematic review process is the basis of many of its products, enabling the WWC to use consistent, objective, and transparent standards and procedures in its reviews, while also ensuring comprehensive coverage of the relevant literature. The WWC systematic review process consists of five steps: (1) Developing the review protocol; (2) Identifying relevant literature; (3) Screening studies; (4) Reviewing studies; and (5) Reporting on findings. This "What Works Clearinghouse Standards Handbook (Version 4.0)" provides a detailed description of the standards used by the What Works Clearinghouse (WWC) to review studies (Step 4 above). Steps 1-3 and Step 5 are described in a separate "What Works Clearinghouse Procedures Handbook" (ED577035). Taken together, these two documents replace the single document used since March 2014, the "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)." Figure I.1 shows how the steps of the WWC systematic review process are divided between the two Handbooks. This "Standards Handbook" provides a detailed description of the standards used by the WWC when reviewing studies that have met eligibility screens, including using one of the following eligible designs: randomized controlled trial, quasi-experimental design, regression discontinuity design, and single-case design. Studies that use other designs are not reviewed by the WWC. The WWC refers to randomized controlled trials and quasi-experimental designs collectively as group design studies. Studies reviewed against WWC standards receive one of the following three study ratings indicating the credibility of evidence from the study: "Meets WWC Design Standards Without Reservations, Meets WWC Design Standards With Reservations", or "Does Not Meet WWC Design Standards." The substantive differences between this version of the standards (4.0) and the previous version (3.0) include the following: (1) The regression discontinuity design standards have been revised; (2) The standards for cluster-level assignment studies have been revised; (3) The standards for studies with missing data have been revised; (4) The "Standards Handbook" includes standards for randomized controlled trials that present complier average causal effects; (5) Additional methods of statistical adjustment can be used to satisfy the baseline equivalence requirement; and (6) The handbook includes additional clarification of existing standards. The remainder of the document is organized as follows. Chapter II provides standards for randomized controlled trials and quasi-experimental designs. This chapter also provides additional standards for randomized controlled trials that present complier average causal effects (with supplemental technical detail in Appendix D). Chapter III provides standards for studies that use regression discontinuity designs. Chapter IV provides information on outcome eligibility and confounding factors that applies broadly across designs. As the WWC uses and applies the standards in this "Standards Handbook", reviewers may occasionally need additional guidance. If necessary, the WWC will produce guidance documents for reviewers that provide clarification and interpretation of standards and support consistency across reviews. This WWC reviewer guidance will clarify how these standards should be implemented in situations where the current "Standards Handbook" is not sufficiently specific to ensure consistent reviews. As the WWC continues to refine and develop standards, the "Standards Handbook" will be revised to reflect these changes. [To view the previous version of this report, see ED544775.].