Identifying methods and developing a plan
With a clear view of the purpose of the evaluation, the key research questions and the principles underpinning the choice of evaluation methods or tools that you will employ, the next step is to proceed with the development of an evaluation plan.
Developing a plan will help you to consider how the evaluation methods will be applied across the life cycle of the course, identifying key actions and the stakeholders who will be involved, as well as the timing for the activities that will take place.
It will help to draw together the threads of your thinking into an action plan.
It will also ensure that you are able to inform students at the beginning of the course about their role and contribution to the evaluation effort.
When considering evaluation methods, useful possible combinations of approaches include:
- Entry surveys can be effective in measuring attitudes to learning, study methods and learning approaches as well as student understanding or perceptions of a topic or theme.
- Exit surveys can measure change in these aspects to gauge ‘distance travelled’ and support evaluation of design and delivery.
- Requests for students to consider and report on study progress at specific milestones within a course, along with ‘stop and check’ activities at regular intervals, can be effective in exploring aspects of the blended learning tasks and gauge how students are experiencing them.
- Tools such as personal learning journals, reflective diaries, video blogs, or self-assessment proformas can also be helpful for students to reflect on their learning to identify and plan for improvement. They can also provide you with valuable insights into their understanding and progress along the way, which can help you to make adjustments where needed.
- Can be conducted during the module or after its completion.
- Worth considering where surveys or other methods raise issues that require clarification, exemplification or elaboration.
- Unstructured or semi-structured questioning often works best, enabling students to direct the discussion rather than be led by the staff agenda in reflecting on the learning experience. This can help to uncover aspects you were unaware of or issues that require further attention.
Many activities involving student interaction with content, other students and staff are facilitated using tools within the VLE such as the quiz tool, discussion boards or learning journals and/or by related tools such as Padlet.
The ‘permanence’ of contributions made using these tools means that day-to-day engagement by individuals and the group can typically be monitored as the module progresses (though it is worth noting that, unlike the VLE, records from third-party tools such as Padlet are not centrally archived).
The information gleaned by staff from these tools can also be supplemented by access to usage reports showing details such as the number of times a resource has been accessed, when and for how long, and times and duration of access to the tools.
However, unless students are specifically required to conduct formative work using these tools, this data may only provide a partial view of engagement. Students may be conducting the majority of their self-study activities through unmonitored independent study.
While usage of these tools is referenced within the student contract, it is also worth discussing these activities with students, perhaps as part of early ‘expectation setting’ and ‘code of conduct’-type activities.
Once you have established the combination to be deployed, you can plan out the activities you will undertake in a delivery plan for your module.
This is likely to include timings, themes and questions, participants, methods, outputs and actions.
Example evaluation pathway
Start of module:
Entry survey or needs analysis as part of an early module session focused on socialisation.
During module:
Self and peer assessment activities carried out by students provide staff with insight into progress and also with embedded evaluation data.
This is supported by regular entries in a reflective learning journal along with statistics on access to Panopto and VLE quiz results. Focus groups as required.
End of module:
Exit survey allows for comparison with entry survey results. Focus groups as required.