Tagged: ethics

(When) Is it OK to deny half of your pupils a promising new teaching approach?

One of the objections sometimes voiced about conducting RCTs in education is that they are unethical. This, it is often claimed, is because requiring a control group (a group of children who do not receive the teaching approach that you wish to evaluate) wilfully ‘denies’ that group access to the new teaching approach. I encountered this argument at a seminar session at BERA 2016, and explored my response to it this post. I’ve been thinking about it some more.

First, it is important to stress that there is nothing peculiar to RCTs about this allegedly unethical behaviour. Research of any design that evaluates the effects of a new teaching approach by comparing it to an alternative involves ‘denying’ some children access to the new approach. Matched pairs designs, regression discontinuity designs, non-random comparisons, multiple baseline interrupted time series, stepped wedge designs, and designs with a waitlist control, all ‘deny’ the new approach to some children, either for the duration of the evaluation or for a portion of it. Equally, all designs (experiments, quasi-experiments, and non-experimental observations) ‘deny’ access to the new approach to any children who are not in the study. Moreover, and by the same logic, children receiving the experimental approach in any of the above contexts are ‘denied’ the control approach for the duration of the study.

This last point brings us to my second, more germane, observation. Implicit in the argument that children in a control group are ‘denied’ the new approach is the assumption that new approaches are always superior to existing approaches. The argument fails to acknowledge the possibility that a new approach might be inferior to existing approaches. It fails, therefore, to acknowledge that children may be harmed by exposure to a new teaching approach. The argument that control groups are unethical is, therefore, lopsided. By the standard applied here, we must acknowledge that ‘denying’ children access to the control approach is unethical as well, as exposure to the new approach may be harmful. This does not leave us in a very informed position.

A recent tweet by Vinay Prasad (see below) explored this argument in relation to the use of sham surgical procedures in evaluations of new surgical approaches. Here, a new surgical procedure is compared to a surgical placebo. Basically, the members of the control group undergo a surgical procedure (anaesthesia, incision, stitching, etc.) but do not undergo the actual surgery. This is important because the potential harms of undergoing surgery may outweigh potential benefits of the surgery itself. For example, in the treatment of prostate cancer, surgery to remove the cancer can cause complications such as incontinence and impotence without changing the life expectancy of the patient (many men die with prostate cancer, not because of it). Without a sham procedure we are less well informed about the relative quality of life following the surgery and, therefore, we are in a less well-informed position to decide the best course of action for the patient.

Prasad illustrated this in the flow diagram below, making the point that introducing a new surgical procedure without comparing it to an alternative (sham) procedure risks harming patients for no apparent reason.

Harms in education are less easy to spot. They are rarely as dramatic as some of those found in medicine, but they do exist (see, for example, evaluations of Chatterbooks and Mate-Tricks).

It is perhaps better to think in terms of opportunity costs. For example, a child taught using an approach that is less effective than available alternatives may still make progress, but at a slower rate. Or, if a child spends time away from their mainstream classroom to receive a targeted intervention, they miss whatever is going on in their classroom during that time. Or, in cases where a new approach is no better and no worse than existing approaches, there are costs of time, money and effort associated with changing the way teachers teach. What could be done with that time, money and effort instead of implementing the new approach for no relative gain in primary outcomes?

Building on Prasad’s work, I mapped out my thoughts on the use of control groups in education. See below.

 

Ethics of Control Groups in Edcuation

For those who argue that ‘denying’ children access to a new teaching approach is unethical, I invite them to consider the question at the bottom of that diagram. My own position is that when there is uncertainty about the effects of a new teaching approach, the only ethical course of action is to evaluate it in relation to the best available alternative, which necessitates having a control group.

A final note on RCTs. As I have said, there is nothing peculiar to RCTs about ‘denying’ an approach to some children while making it available to others. What is peculiar to RCTs is the method by which children are allocated to different approaches. The single defining feature of an RCT is that children are allocated to alternatives fairly. No child (or school or classroom if you are doing a clustered RCT) stands a better or worse chance of being allocated to either the experimental or control group than any other child when the decision is a random one. That’s the whole point of randomisation.

When my brother and I were children, our dad used to toss a coin to decide which of us got lumbered with the washing up after family meals. While it never felt fair to the one who ended up wearing the Marigolds, we could hardly argue with the ethics of our dad’s method of choosing. By the same token, when we must decide who receives what in a comparison of alternative teaching approaches, I contend that random allocation is not just the most effective way of creating unbiased comparison groups, but it is the most ethical way, too.

For anyone interested, you can download a PDF version of my diagram here. Feedback welcome.

Advertisements