Author: Donna Hurford
As illustrated in Hattie’s (2015) acronym ‘teachers are to DIE for’, D is for diagnosing students’ learning ‘status’ at the start of a lesson, I is for the various teaching interventions the teacher can draw on if one is ineffective and E is for the evaluation of these interventions on students’ learning. Evaluations of student learning can inform effective teaching but, evaluations are not by default informative or unbiased. Yet it is standard university practice to expect students to complete an evaluation at the end of a course and to expect teachers to review and update their courses and teaching in response to this feedback.
Teachers’ skepticism about end of course evaluations, which include more highly satisfied and highly dissatisfied feedback rather than providing a more even range of feedback is supported by Grimes et al. (2016) whose study found evidence of students inflated retrospective memories. End of course evaluations rely on responders’ altruistic commitment to the next cohort of learners whilst providing a forum for the discontent and overly content. But what’s the payback for students who have already completed the course? And how relevant is one cohort’s course evaluations for another cohort’s engagement and learning in the same course?
In addition, the quality and usefulness of the responses depend on the evaluation’s purpose and how well this is aligned with the questions, its design, and the clarity and accessibility of the questions. And even when the evaluation is well designed, aligned and clear there is still the issue of student bias, which can be highly influenced by stereotyped expectations – the humourous male teacher and the personable female teacher (Doubleday et al, 2016; MacNell et al., 2014). If evaluating learning is going to be time effective for students and relevant for teachers, we need to address some issues.
What’s the purpose of a course evaluation?
- Be clear about the purpose and value of student evaluations. What will be done with them? Are they fit for purpose?
- Co-design course evaluations with teachers, survey designers and students.
What are learners best placed to evaluate?
- Focus evaluations on the course learning goals and how specific learning activities align with these goals. See Sparqs’ guidance on the Student Learning Experience.
- Train students in evaluation practices, acknowledge it as a graduate competence. See Simon Varwell’s contribution on ‘Students as Partners and Co-creators’ for more on developing students’ competences.
Evaluate during not just after
- Integrate formative evaluations during the course – one-minute papers, Student Response Systems, focus groups, Think-Pair-Share, student representatives forum
- Make the evaluation process formative for the students and the teachers, so the students and teachers get a payback from the evaluation process.
- Be explicit about what is or is not being changed in response to student feedback and provide clear pedagogic rationales.
Address Bias
- Provide students, course representatives, instructors and teachers with bias-aware training – make it compulsory and integrated into programmes, not just a one off during the introductory week of a programme. Include a graduate competence on bias-awareness and evidence of applying strategies for mitigating bias in their university studies.
Raise the status of evaluations
- Adopt a Faculty-wide commitment to bias-aware evaluations, raise the profile of evaluations and make it explicit in the evaluation policy that the Faculty/Department is committed to bias-aware pedagogic practices including course evaluations. Maintain a consistent approach so all students know this is being taken seriously. Include information in student handbooks, make it explicit.
- Support teaching staff to create “success” statements such as “Last year, the students who evaluated this course led to the following changes… now how can you help?”
- Make completing course evaluations a course requirement, link to a Learning Outcome
- Make it a research project, involve staff and students – get published and present on your university’s evaluation strategy. Finding ways forward in this area is cutting edge, few universities are addressing it head on.
References
Doubleday, A.F. and Lee, L. M. J. (2016) ‘Dissecting the voice: Health professions students' perceptions of instructor age and gender in an online environment and the impact on evaluations for faculty.’ Anatomical Sciences Education. Vol. 9 (6), 537-544.
Grimes, A., Medway, D., Foos, A., Goatman, A. (2016) ‘Impact bias in student evaluations of higher education.’ Studies in Higher Education. Vol. 42 (6), 945-962
Hattie, J. (2015). ‘The applicability of Visible Learning to higher education.’ Scholarship of Teaching and Learning in Psychology. Vol. 1 (1), 79–91.
MacNell, L., Driscoll, A. and Hunt, A. N. (2014) What’s in a Name: Exposing Gender Bias in Student Ratings of Teaching. Innovative Higher Education. Vol. 40 (4), 291-303.
Student Learning Experience resources https://www.sparqs.ac.uk/resource-item.php?item=205
Unlimited.sdu.dk Unlimited Thinking and Teaching: Insights and resources on bias aware teaching and learning
http://unlimited.sdu.dk/index.php?page=evidence-research – ‘Students Evaluations of Courses’, includes references and abstracts from relevant articles.
Alternatives to Student Evaluation Surveys – Age of Awareness – Medium
based on http://cte.rice.edu/blogarchive/2018/2/20/studentratingsupdate screen cast Student Ratings of Instruction: A Literature Review
************************************************************************************************
This work is licensed under a Creative Commons Attribution 4.0 International License.
