All trials are not created equal.
I made this tutorial video to help my students understand that assessments of the effects of teaching interventions sit on a scale of trustworthiness, and that it is important to take into account an experiment’s position on that scale when interpreting its results. This video briefly explains the principles behind research trustworthiness and walks through the process of assessing it.
Using a paper that describes three studies of SIOP (a sheltered instruction model of teaching children whose first language is not English) as a test case, I use Stephen Gorard’s “‘sieve’ to assist in the estimation of trustworthiness” to assess the first of the three studies.
Feel free to use the video if you would like to. Just let me know how you get on with it, in the comments section.
The papers used in the video are:
Gorard, S. (2014). A proposal for judging the trustworthiness of research findings. Radical Statistics, 110, 47–59.
Short, D., Echevarria, J., & Richards-Tutor, C. (2011). Research on academic literacy development in sheltered instruction classrooms. Language Teaching Research, 15(3), 1–19.