In this post, Anna Wood at the School of Mathematics introduces the Framework for Interactive Learning in Lectures, a classroom observation tool she co-developed to support reflections on teaching. It often goes unsaid that reflection relies on good recall – yet memory can be, as Anna points out, unreliable for many reasons. Might detailed data on teaching, and its collection and analysis, be a useful response to this? This post is part of the Learning & Teaching Enhancement Series: Reflective Learning.
Think back to the last time you reflected on your teaching. What form did it take? Was it, like many of the lecturers I have interviewed, triggered by student evaluations at the end of a course? Or did it take place when you came to write your slides for the following year? Or perhaps you took some time after each lecture to think about what worked and what didn’t?
Either way, it is likely that you are relying on your memory of what happened (or that of your students). But memory is fallible, especially in an environment that is stressful and where we are concentrating on many different elements simultaneously.
For this reason, George Kinnear and myself have developed a way to help support teachers’ reflection that is based on concrete and personalised data about teaching practices. In this approach we use a classroom observation tool called FILL+ (Framework for Interactive Learning in Lectures) to create detailed visualisations of the activities that take place in lectures. FILL+ was developed by myself and George (Kinnear et al., 2021; Wood et al., 2016) as a way to measure the time spent, second by second, on common activities, such as the lecturer talking, students asking and answering questions, as well as active learning techniques such as students discussing in small groups.
Each code is also assigned to one of three interactivity categories: a) interactive, b) non-interactive and c) vicarious interactive. The first two are fairly self-explanatory – interactive activities involve the students discussing or thinking about a problem, whereas non-interactive activities involve listening to the lecturer talking. The vicarious interactive category refers to questions to and from the lecturer (and their respective answers), since this sort of dialogue does not belong in the non-interactive category, but neither is it fully interactive as it involves only a few students as active participants.
Two examples of time-lines from a FILL+ analysis are shown below. The first shows a simplified version with the three levels of interactivity. This is a good starting point for lecturers new to a FILL+ analysis who want to get a quick overview of how their lecturers are experienced by students. The second graphs shows the full FILL+ analysis with all 10 codes. This one is more useful for lecturers who want to take their time thinking about the detail of their teaching. This data helps the teacher to see how much time they are spending on different types of activity, to observe how activities are spaced during the lecture, and to understand how the lectures are experienced by their students.
Our data-led approach to reflection combines a FILL+ analysis of lectures with an opportunity to take part in a supportive discussion about teaching practices. In 2020 we conducted a pilot study (Wood et al., 2022) with 17 lecturers from across STEMM subjects (Science, Technology, Engineering, Mathematics, and Medicines). We found that most participants valued seeing their data and that they particularly appreciated the detail and accuracy of it. Participants also felt that it gave them reassurance that they were achieving the type of teaching that they aimed to provide.
However, we were really interested in the effect of this approach on reflection. We found that the majority of participants did find this data-led approach helped them to reflect on their teaching. One commented, “it makes you reflect a little bit about what you do and how you do it”, and others noted that they would now look back at their lecture notes to understand the data in more detail – in fact, a few did this during the discussion!
Some participants also indicated that they would think about changing their teaching as a result of seeing this data. For instance, some lecturers said they planned to increase the interactivity in their lectures and others noted that they had long periods of talking, which they would try to break up in the future by spacing out interactive elements.
One interesting finding was that while the overview of the data was not surprising, frequently participants found that the detail in the data was surprising. Finding something surprising, triggered them to reflect on the data and on their teaching.
A couple of participants however did not find any value in this approach. These were typically lecturers who had lower levels of interactivity and for whom the LT code (lecturer talking) applied for at least 90% of the lecture. A few others found the data interesting, but did not feel it would make any difference to their teaching approach. The reasons for this, and the barriers lecturers faced in changing their teaching, is a subject for a future blog post!
We are now exploring how this approach to helping STEMM lecturers reflect on their teaching can be integrated into formal academic development programmes, starting with the PgCAP and EdTA. We are now recruiting participants – so, if you are interested in having your data analysed, whether or not you are on these courses, please do get in touch.
References
Kinnear, G., Smith, S., Anderson, R., Gant, T., MacKay, J. R., Docherty, P., Rhind, S., & Galloway, R. (2021). Developing the FILL+ tool to reliably classify classroom practices using lecture recordings. Journal for STEM Education Research, 1–23.
Wood, A. K., Christie, H., MacKay, J. R., & Kinnear, G. (2022). Using data about classroom practices to stimulate significant conversations and aid reflection. International Journal for Academic Development, 1–14.
Wood, A. K., Galloway, R. K., Donnelly, R., & Hardy, J. (2016). Characterizing interactive engagement activities in a flipped introductory physics class. Physical Review Physics Education Research, 12(1), 010140.
Anna Wood
Dr Anna Wood is an education researcher within the School of Mathematics. She has a PhD in Physics and gained an MSc in E-learning (now digital education) in 2013 and combines these interests to understand how STEM students learn in large classes, the role of technology, and the impact of different types of interactions including dialogue in learning. In 2016, she developed the FILL (Framework for Interactive Learning in Lectures) to quantify the time spent on activities during lectures.