Depende, by Anique de Bruin
“Depende”. It depends. The title of a song from my favorite Spanish band ‘Jarabe de Palo’. I had hardly heard it since it was a hit in 1999, when I was an exchange student at the Universidad Autónoma in Madrid, but it was on the radio of my rental car minutes after landing on Ibiza, last September. It depends. It is the suitable answer to any question on the effectiveness of any educational intervention.
No matter how large the evidence base, whether an intervention works depends on learning context, learning materials, and student and (if applicable) teacher characteristics. This presents a major challenge to us, educationalists. A challenge we try to ignore most of the time, or one we reserve for the Discussion section of our papers where we shove it onto others in the ‘future research’ paragraph. It is a challenge we should urgently embrace according to recent research in Nature Human Behavior. Trying to understand the heterogeneity in effectiveness of interventions by unraveling the moderator variables that change effectiveness will have theoretical and practical implications, probably larger than we can envision.
I agree. If we want effective education and training, let’s hug this heterogeneity and never let go. But how do we go about igniting this research revolution? The truth is I don’t know (it depends?), but I will start by formulating three steps forward. First, for quantitative studies; let’s increase sample sizes considerably to allow for powerful analyses of moderating variables that should be identified based on theoretical assumptions. For qualitative studies; increase contextualization as a theme both in the design of the study and in the interpretation of findings. Moreover, let’s distinguish between subjective factors (such as experienced effort, interest in the topic, trainees’ beliefs) and objective factors (such as working memory capacity, IQ, GPA, learning domain, learning task), and acknowledge that the objective factors help us understand what the learning mechanisms of the interventions are, whereas the subjective factors (also) influence how the intervention is experienced and used by learners and teachers. Heterogeneity at both levels is affecting effectiveness of interventions and may interact. Finally, let’s not fall for the assumption that every student always needs an adapted intervention and recognize that flexibility of education is not a goal in itself. It may very well be that despite large heterogeneity a general intervention works and is adopted by the great majority of students. When and why this is the case should be an explicit topic of research.
For a recent example of how we are starting to hug heterogeneity in our Self-regulated Learning (SRL) Lab at SHE, have a look at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8103204/ where we distinguished different clusters of students in how they adapted to emergency remote learning during the first lockdown of the Covid 19 pandemic.
The SRL lab in action in Maastricht. From left to right: Anique de Bruin, Luotong Hui, Felicitas Biwer, Wisnu Wiradhany (on screen), Erdem Onan, Héctor Pijeira Díaz. Not in picture: Sanne Rovers, Louise David, Tian Fan.
And enjoy Jarabe de Palo's song
Anique de Bruin, Professor Self-regulation in Higher Education. Vice-director School of Health Professions Education