Does forcing Welsh children to learn both English and Welsh make them educationally ‘weaker’ than their English peers? – #AskForEvidence Update

It’s been three weeks since I asked Toby Belfield, principal at Ruthin School, Denbighshire, North Wales if he would be willing to share the evidence upon which he based his assertion that “part of the problem with forcing young people to learn both English and Welsh (arguably, both to a substandard level) is that young people in Wales will continue to be educationally weaker than their peers in England and abroad.” (see this post).

I have not had any response (not even a cursory ‘no comment’) to the two emails and three tweets I sent to him. Perhaps he is too busy to want to get involved further after his public and unreserved apology – it is exams season, after all.

In that apology though, Belfield fails to address his claim about the effects of dual language education. Instead, he engages in special pleading to recast his assertions as something they were not. His original letter stated that dual language education (“forcing young people to learn both English and Welsh”) puts Welsh children at a disadvantage. In it he also contended that the educational system in Wales should have English as its first language. He now says that what he meant was that he rejects Welsh only education, that he thinks finding suitably qualified teachers who speak Welsh would be difficult, and that he supports parental choice. He did not address any of these things in his original letter (indeed, his insisting that schools in Wales should have English as their first language rather contradicts that last point). This special pleading, in lieu of addressing what he actually said in his letter, suggests either that he is unwilling or unable to support his original position or that he is not sufficiently adept at expressing himself in a letter written for public consumption. I will leave readers to consider which of these possibilities should be the greater concern for the parents of children at his school.

But, to the question of evidence. Belfield has shown no interest in addressing the claim he made, so I felt it only right to attempt to do so on his behalf.

First, I want to be clear that I intend only to address the question of the effects of dual language or Welsh Medium education in Wales with substantive, empirical evidence. There are other claims that Belfield makes that could be assessed in the light of evidence. For example, we could look at whether it would be hard to find sufficiently qualified bilingual teachers, whether children with Welsh as their first language do find it harder to get into top universities, whether job prospects are worse for Welsh-speaking school leavers, whether those Welsh-speaking school leavers who want to find employment outside of Wales do find it more difficult, and so on. I will say no more about them, other than to invoke Hitchen’s Razor.

Belfield is quoted as saying that he based his opinion about the precursors of the poor performance of Welsh school children on the NfER report on the PISA assessments of 2012. In 2012 PISA tested the reading, science and mathematics attainment of 15 year-olds in 65 countries.  From the UK data, the NfER prepared reports for each of the constituent countries of the union. I obtained a copy of the Wales report referenced by Belfield to see if I could find anything in it that would lend support to the notion that Welsh medium or dual language education is responsible for poor performance among Welsh children.

On reading the report it is easy to sympathise with Belfield’s concern for the Welsh education system. In summary, Wales is said to be performing significantly below the mean average for OECD countries in all subjects assessed, and significantly lower than all other areas of the UK. Notwithstanding that half of the countries in the sample necessarily must fall below the mean average; so far, so concerning.

These data are derived from the averages of all 3305 Welsh students attending a total of 137 Welsh schools that took part in the assessments. So, if you want to draw conclusions about the effect language of instruction has on attainment, the sensible thing to look for is the languages of instruction of the schools that took part in the assessments. Were they Welsh Medium, English Medium, dual language, or all three?  If it was all three, is there any breakdown of results by school type that could indicate if any one type is outperforming the others?

In the appendix of the report (page 101) we learn that “The sample in Wales contained Welsh and English medium schools and bilingual schools, although language of instruction was not a stratification variable.” This means that when the 137 schools that took part in the assessments were initially chosen, no attempt was made to sample a representative number of each type of school. So, we know that Welsh and English medium schools and bilingual schools took part in the assessments, but not how many nor in what proportions.

Later in the appendix we are told that a Welsh language version of the test was made available to schools who requested it (page 103). They report that in 14 schools all students completed the Welsh language versions of the tests and that in a further 8 schools some students completed the Welsh language version. The report states that, in total, 381 students carried out the tests in Welsh. From these figures we might infer that 14 of the 137 schools were Welsh medium and that the other 8 schools might have been bilingual schools, or Welsh medium schools with students who nonetheless preferred to take the tests in English.

According to Welsh Government statistics, in 2012 there were 56 (25%) Welsh medium secondary schools, serving 41,262 (21%) children. So if my guesswork is in any way accurate, we might conclude that Welsh medium schools are underrepresented in the PISA sample as a proportion of secondary schools in Wales as a whole (approximately 10% as opposed to 25% – or approximately 16% if we include the schools where only some students took the test in Welsh), as are students taking the test in Welsh, compared to the proportion of children in total who attend Welsh medium schools (11% as opposed to 21%). But, as I say, this is guesswork, and nowhere in the report is this clarified. And nowhere in the report are the results of the assessments broken down by school type anyway. This leaves us none the wiser as to any potential associations between language of instruction and student attainment.

So, what can we conclude from this? Mainly, that Toby Belfield can’t have based his claims about the effects of Welsh language education on anything contained in the PISA report that he suggests informed them.

I wasn’t happy to leave it there though, so I contacted the authors of the report to ask if they would be willing to share the raw data so that I could do my own sub-group analysis based on language of instruction.

Bethan Burge, one of the authors of the report, was extremely helpful and explained to me that the data was owned by the British and Welsh governments and that I would need to ask them if I could see it. However, she did send me a copy of another report that she and a colleague had prepared using the PISA data, entitled Additional Analysis of Wales’ Performance in PISA 2012. In addition to the sub-group analyses you would expect (results by gender, free school meals, ethnicity, SEN and so on) it had a short analysis of the results of the PISA assessments, stratified by medium of instruction.

Here it is in all its glory:

In reading and science, the performance of learners attending Welsh medium and English medium schools is comparable, that is, score differences are not statistically significant. However, in mathematics, learners attending Welsh medium schools outperform those in English medium schools by 10 score points. This difference is statistically significant.

Burge and Lenkiet (2015:7)

In addition there is some analysis of GCSE results and language of instruction:

At the school level, there are significant associations between medium of instruction and mathematics GCSE scores […] if we look at a group of learners with similar scores in the PISA mathematics assessment and similar individual learner characteristics, those in a Welsh-medium school will, on average, have higher mathematics GCSE scores.

Burge and Lenkiet (2015:12)

and

At the school level, as is the case for mathematics, there are significant associations between medium of instruction and science GCSE scores […] This means that if we look at a group of learners with similar scores in the PISA science assessment and similar individual learner characteristics, those in a Welsh-medium school will, on average, have higher science GCSE scores.

Burge and Lenkiet (2015:12-13)

So, I can now answer my own #AskForEvidence based on the same data source that provoked Belfield to make his bold claim. Far from putting students at a disadvantage, the PISA data suggest that language of instruction makes no difference to outcomes in science and reading (and so guilt-free parental choice can be exercised) and that Welsh medium schools put students at an advantage in maths. In terms of GCSE results, all else being equal, Welsh medium schools are associated with higher achievement in GCSE maths and science.

Of course there are always caveats to any data set, some of which I have addressed in my original post on this matter, and the article linked to by one of the comments on that post is a useful read in this regard. Nonetheless, it is helpful to know that empirical evidence is available to inform assessment of the effects of different models of Welsh education. And empirical evidence is always preferable to personal opinions based, we can only assume, on no more than gut feeling.

Advertisements

8 comments

  1. Craig

    Great article mate. Unfortunately though very few political decisions (or any other decisions for that matter) are made on empirical evidence, and we pretty much all know that Belfield wrote what he wrote solely on the basis of how he FEELS about Wales, the Welsh and the language. Parents send their children to private schools because they believe theyre buying their kids entry into the sort of upper class English lifestyle they have seen in the films; punts, straw hats, cricket, Pimms, endless sunshine in the summer, snow on christmas day, cottages, a nice job in London. For the parents that have stayed in England that sort of ambition is trouble free, it matches perfectly the lives and agendas of the great and good of that country. Unfortunately though for the ones who have found themselves in Wales there is some tension. On the one hand its pretty much just like England, and the arrogance that enabled the parents to get on in life is met with even more deference here than it is across the border so there shouldnt really be a problem. But on the other hand, somehow we managed to get Welsh onto the cirriculum, somehow Welsh culture isnt as completely cowed as they had assumed, somehow not everybody here dreams of being an extra in a Richard Curtis film. When they cant have it all exactly as they want it, they cant check the arrogance, they cant see things through others’ eyes and they write badly thought out letters to papers calling for us all to grow up and stop all this Welsh nonsense. You can make an appeal based on empirical evidence but it will ammount to nothing because Belfield has the greatest of all English gifts, commonsense.

    • Hamish Chalmers

      Thanks for your comment. One of the key motivations for my post was the issue of evidence in making claims. You are right to say that it can seem that political (and other) decisions are often not made in the light of empirical evidence. This underscores the importance of the #AskForEvidence campaign in which I framed my approach to this issue.

      One of the key goals of that campaign is to show people like Belfield that it is not sufficient to allow claims that can be supported by evidence to be made in the absence of that evidence. To quote the Ask For Evidence website “Simply by asking – and telling people that you’ve asked – you’re showing that people care about evidence.” My hope is that, by increments, this will help foster a culture where people are less and less able to get away with making unsubstantiated claims, in the way that Belfield has. The question of Welsh language education in Wales is really important, not just for educational reasons but for cultural reasons and for reasons of social justice. We know that bilingual education has worked very well in the USA for latino students (see Krashen and McField 2005). There are good examples of where honouring indigenous languages has had very positive effects beyond education, even in the face of official ambivalence. See, the case of Māori schools in New Zealand (Appleby 2002). And there are, of course, examples of where revitalisation of indigenous languages and cultures has failed to produce the results hoped for. See, for example, the case of South Africa (Kwaa Prah 2007).

      Toby Belfield has made a claim that does not appear to be backed by evidence – certainly not by the document that he has been citing as the inspiration for his letter to the Denbighshire Free Press. He is entitled to say what he likes about how he ‘feels’ about Welsh language education (and shoulder the angry responses that that will provoke). But for someone in a position of influence and respect (he is the principal of a school, after all) this evidence-free behaviour seems rather unbecoming to me.

  2. Judith

    Dear Hamish

    I just read your ‘Ask for Evidence’ submission with great interest. Thank you for your excellent posts – I felt like replying, even though they are a few months old now. I am a senior lecturer in education at Exeter University with a (research) interest in language use in classrooms and also happen to have two children in Welsh medium education. We speak Dutch at home, the children are being taught through the medium of Welsh and they speak English in the wider community.

    I agree very much that the issues are very complex and that it will be difficult, if not impossible, to relate any results about school attainment to the language spoken in school and at home. I think the discussion is a bit difficult in itself – if the situation had been reversed. If Welsh happened to be world language, it would have been seen as a huge benefit for English speaking children to learn the two languages. This situation is not dissimilar to Welsh-speaking children going to an English-medium school, or Dutch children in Holland going to dual medium (i.e. English and Dutch) secondary schools, something which is highly popular. The issue, in my opinion, is thus only an issue because Welsh is a relatively ‘marginal’ language and Wales does not tend to do well in international comparisons.

    This latter phenomenon hints to another problem, which is that an unfair comparison is made when English speaking children in EM schools are compared with non-Welsh speaking children in WM schools. One of the links above mentions that children in WM education from non-Welsh speaking backgrounds are less likely to ‘overachieve’. Yet, the fact that these children are fluent in two languages is overlooked as an ‘overachievement’. To me, it seems that children who learn to speak, read and write fluently in two (or, in my own children’s case, more) languages, are by default over-achievers. It has to be kept in mind that all the tests that these multi-lingual children do are based on a mono-lingual assumption and any results will therefore be biased. Any fair comparison of Welsh-medium education system should only be with other multi-lingual countries, not with mono-lingual English education.

    I think more attention should be given to the fact that these children are learning other languages, and therefore perhaps will be slower to catch up or will not achieve as highly on standardised tests. Of course, there is a wider discussion in here, as to what the value is of education in the first place. The point that I am trying to make is also illustrated with the recent findings that children in private education tend to do better in GCSE and SATs but are less likely to achieve a First in their university degrees, than children from state-funded schools. Privately educated children thus tend to do better on standardised tests, but as soon as a wider range of capabilities are called upon, their ‘advantage’ is diminished. The value of standardised tests can thus be questioned, and the real value of a dual-language education will perhaps not be seen on standardised tests, but in other situations.

  3. Hamish Chalmers

    Thank you Judith. I really appreciate you taking the time to comment.

    There has been publicly voiced disagreement about the place of mother-tongue in prestige-language education for centuries. When I did my MA dissertation (a plain language summary of which you can see here) I found examples of people either encouraging or proscribing use of mother-tongue in education going back as far as the 1500s and continuing right up to the present day. So, your comment is, relatively speaking, bang on time being only a couple of months since I wrote the posts 😉

    Toby Bellfield was demonstrably wrong to assert that Welsh language speaking was the cause of low achievement in recent PISA tests. This is borne out by the subset analysis conducted by the NfER that detected statistically significant differences between the results of Welsh language users and English language users, in favour of users of Welsh. He went on to fudge his assertions by claiming that he had meant something else.

    These PISA data notwithstanding, the issue of dual language education and its effects on educational or quasi-educational outcomes (such as the ones you allude to) is, naturally, a little more complicated than that.

    The research on bilingual forms of education is interesting and varied. Some of the most trustworthy research (i.e. studies that use counterfactuals, and sometimes random allocation to comparison groups) was done in the 1970s and shows a modest positive effect on language acquisition (both L1 and L2) and academic achievement of attending bilingual schools, with the best results coming from programmes that aim to maintain both L1 and L2 – rather than transition from L1 to L2. There are a number of good systematic reviews that synthesise the evidence on bilingual forms of education. See Krashen and McField 2005 for an overview of these reviews. In my view, one needs to be cautious in interpreting the findings, however. One clear cautionary element is selection bias: Who chooses to send their children to bilingual schools? And, who stays in bilingual schools long enough to contribute to outcome measures such as exam results? There is always a choice in these respects. Children whose parents feel that their children won’t thrive in bilingual schools don’t send them to them, and children who are failing to thrive can be relocated to monolingual schools. This makes interpretation of the results of longitudinal studies problematic as there is likely to be a systematic difference between children in bilingual schools and those not that could influence comparisons.

    Your point about what we value in terms of educational outcomes is also a really important part of this. In a bilingual school that, on average, does no better than a monolingual school in standardised tests, the children still have an extra language when they leave. So, by any measure, children in those schools have, as you put it, over-achieved. In terms of being slower or faster to catch up, we find that, for example, vocabulary acquisition progresses at the same rate as in monolingual children – it’s just that the ‘slots’ in the brain for new words are shared out among the two (or more) languages. In the early stages of acquisition in young children this looks like a delay if you are considering only one language at a time. If a child has, say, 1000 lexical ‘slots’, 500 are taken by L1 words and 500 are taken by L2 words. So if you judge a bilingual child’s rate of vocabulary growth at that stage using only one of the languages available to them, it looks like she or he has acquired only half the number of words that would have normally been acquired. But once children start school vocabulary acquisition is so rapid and rises exponentially that this sharing of ‘space’ soon becomes a non-issue.

    My final thought on this, for now, relates to what I see as linguistic/educational bigotry. We have seen the learning of foreign languages begin at younger and younger ages. When I was at school we started with French in Year 7 which was augmented with German in Year 9. My eldest daughter has just started in EYFS and has already begun German once a week. Proficiency in an additional language is clearly sufficiently highly valued for policy makers to have taken the decision to introduce it earlier and earlier (despite, incidentally, evidence that suggests that this early introduction makes very little difference to children’s ultimate attainment in that language). And I’m willing to put money on Toby Bellfield getting the warm fuzzies when he sees the pupils at his school scoring well in their French and Latin exams. However, it seems that proficiency in more than one language is only valued for children who start off as English speakers and add another language to it. If they start of with something other than English (Polish, Urdu, Pashto, Welsh, etc.) then all but the most enlightened schools do everything they can to drum it out of them and turn them into monolingual English speakers. I can’t think of any other subject where policy makers see it as appropriate to strip away their pupils’ abilities, rather than use them as building blocks for higher achievement.

  4. S R Adams

    Dear Hamish

    I’ve just read your post, nearly a year after ‘the event’, and I do apologise for posting a response when you may have put this one ‘to bed’. I’m a parent of a pupil at Ruthin School – perhaps this would be a good time for me to run for cover! But I’m not responding to your post to defend the Head’s original letter, just to clarify perhaps a few misconceptions and even throw a few more questions into the mix.

    As you’re no doubt aware, the letter in the press was written by the Head in response to a letter that called for all schools in Wales to be Welsh-medium. I think the hasty response of the Head that followed did not adequately articulate the position of multi-lingual or bilingual education in the school and his response did indeed reflect a critical attitude towards Welsh language education.

    One of the key elements that was not clear in the Head’s response is the existing multi-lingual nature of the school. At a guess, I would say that at least 95% of the pupils are bilingual and around 50% of the pupils are multi-lingual with at least three languages. Some pupils I know there speak 8 languages. Most of these pupils have been brought up and educated in simultaneous language-learning environments and a very large percentage do not speak English when they arrive at the school – summer schools are put on prior to the start of terms to help pupils with their English language, and there is a full time ESL teacher. The school offers some lessons through the medium of Mandarin, and I have also been told of lessons being offered through the medium of French, Spanish and Russian. I know five pupils for whom Welsh is one of their languages and four of them are ‘1st language Welsh’; one pupil was a sequential Welsh speaker who attended Welsh medium education until the age of 11. Welsh was offered as part of the school curriculum but due to the number of overseas pupils there was little take-up. For those of us who wanted our children to access Welsh language learning, Welsh lessons were made available until the options stage when the pupils could choose if they wished to continue to GCSE level. I think about 5 of the staff at the school speak Welsh as a first language – I know of three teachers who do, one of whom speaks all the P-Celtic languages fluently. Multi-lingualism is certainly not discouraged in the school and in fact the opposite is true, with efforts being made to support pupils in their native language rather than use English. It’s a joy to hear the pupils chatting and switching between languages.

    From speaking to some of the parents of my son’s friends, and the parents of past pupils who chose to move their child from Welsh-medium education at Primary, their decision to move their child to the school was mainly based on the fact that their child was a sequential Welsh language learner and as monoglots or bilinguals (in other languages) but not Welsh speakers themselves, the parents were concerned about their child’s level of English and science knowledge. This didn’t seem to be a pushy parent situation – they talked about struggling with the decision but feeling they had to listen to their children because they were motivated to learn but they were not coping well in the secondary Welsh-medium system. As their children weren’t simultaneous bilinguals and probably only spoke academic Welsh in school, with little transmission beyond that, they were finding their education difficult. That of course is their very personal experience and it differs from pupil to pupil. There are also many factors that contribute to those experiences, but saying it’s because they are in Welsh-medium education is certainly not the reason for their difficulty – they may have had a poor teacher, or the pupils may have been hard on themselves, or perhaps they were unhappy etc etc. So I’ll say again that I feel the Head’s response was hasty, possibly based on some of this feedback from parents coupled with a superficial reading of the PISA assessments, and I think he may have fared better to distinguish between the benefits of simultaneous bilingualism compared to learning through the medium of Welsh if a pupil has learned it as a second language up to the age of 11, and then enters Welsh-medium secondary with total immersion. I think there are some reports and statistics around this but I’d be interested to read your take on it as a linguist.

    The aftermath of the publication of the Head’s letter was interesting but also heartbreaking. The pupils were targeted on social media – many of them young Chinese pupils who were sought out because they had the school name in their profile, and cruelly berated. It was an ugly side of those who are determined to link language with a perception of racial identity. The other side of the coin was that the school had a huge number of enquiries from Welsh parents who were not happy with their child’s education, asking about the scholarships or fees for their children. I know this second point will not be popular, but it was the situation at the time.

    I’m aware that the private education system divides people in more ways than one. Assumptions are often made about background, language, affluence and aspiration. I’m not writing to make the case for or against private education. I believe all pupils should not only have the right to an education, but an excellent education regardless of their background. We made our choice for a number of reasons that I don’t feel need to be defended. In relation to one of the responses to your post above however, I know of no parents who have sent their children to private school because of a ticket to an upper-class English lifestyle in London. I’m not saying that doesn’t happen, but it hasn’t in my experience. I went to private school in South Wales. My father was a builder, my mother an accountant for a production company. My grandparents were miners and farmers. I was an only child and because the local school was having major problems at the time, my parents saved to send me to private school at the age of 11. I know it wasn’t easy for them, and I think that is the situation of most people who choose private education. Most of my fellow pupils either went into medicine or the voluntary or international development sector, many of them working abroad – not into banking or ‘making piles of cash’ in London etc. Private school didn’t give me a ticket to anything, but it did make me very liberal and socially aware, perhaps because of the international nature of the school.

    In my lifetime I would like to see bilingualism from an early age, of course in the home but at least as soon as a child starts nursery education. I feel it would also be beneficial if all children across the UK had some knowledge of the P-Celtic languages and a system was introduced that they were introduced to basic words and phrases and had the opportunity to try the languages out as a core part of their Primary curriculum. But I also believe in choice, and if our language curriculum can be as varied as possible (perhaps tri-lingual to include a language that is symbolically written) it can only serve to give children the tools to make an informed choice.

    Again, apologies for coming to your original post so late and I hope that in the points I’ve put forward, I don’t appear to be defensive.

    • Hamish Chalmers

      Dear S.R.

      Thank you for this thoughtful comment on my post. It deserves an equally thoughtful response. If you will forgive me a few days to review my notes from the time of the original post, I look forward to doing so.

  5. Pingback: Welsh Language in Ruthin Under Threat? | L3xiphile

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s