Making sense of your evaluation data and taking action

Interpreting the data:

• Recognising and mitigating for challenges including:
• Engagement (eg survey fatigue or motivation to engage in focus groups). This can lead to challenges in achieving a representative sample of a cohort for focus groups. For example, volunteers may reflect the most motivated students within the class or those with strong positive or negative feelings about particular approaches, presenting a one-sided view of the learning experience. For international cohorts, cultural issues may come into play, with some groups of students reluctant to offer negative feedback on the course design or level of instructional support for their learning.
• Visibility and context (data collected may not be comprehensive enough to capture the full range of learning in formal as well as informal settings)
• Halo/horns effects (eg impact of assessment outcomes for individual responses or other potentially biasing factors relating to perceptions of teaching staff)
• Triangulation of results with other data sources may help to confirm or throw into doubt the trends that have been identified.

Reflection in action

• Formative evaluation focuses on processes as well as outcomes so will allow some adjustments to be made as the module progresses (depending on the timing and nature of measures to uncover these).

Reflection on action

• After the module, reviewing the data in the light of the original goals should allow for adjustments to be made (eg design, activities, materials, instructions, feedback and support).