Automating course evaluations

This spring, the Learning environment group initiated the process of automating course evaluations by using Survey & Report. The present way of working is resource-demanding and therefore the long-term goal is to completely automate the creation of course evaluations. All necessary data cannot be extracted from the present version of Ladok, which is an obstacle to complete automation. That information will be available in Ladok 3 which is to be introduced during 2017, and this will allow for complete automation.

The first pilot studies will be conducted during some of our summer courses, and in the course of the autumn semester we anticipate an implementation for all of LNU. Pilot study participants from the Faculty of Arts and Humanities have already been selected.

When the surveys are automated, only anonymous respondent surveys will be created.

The purpose of automation

• Staff members who make course evaluations for entire faculties will have their workload reduced as the majority of steps are automated.
• Staff members who archive paper surveys will no longer have to do this.
• Teaching staff who previously had to turn to an administrator who then handled the archiving of one or more signed course evaluations including the action plan, will now simply be able to approve these directly in the system.
• The surveys will be respondent surveys which will lead to an increased accuracy and often an increased response rate. The human factor will be minimized when it comes to handling a link (public survey) that is to be emailed or placed in a moodle room.

Who will be affected

• Staff members who are now responsible for producing course evaluations for entire institutions/faculties.
• Staff members who handle the archiving of course evaluations.
• Teaching staff

Details on the process of automation
Administrators will work in an entirely new interface in S&R. The system will list all courses that are available for evaluation. The administrators must fill in certain information that cannot currently be extracted from Ladok.

• The final week of the course
• Teaching language (Swedish or English)
• Course coordinator (and/or the person responsible for course evaluations)
• Teacher education course (flag whether the course is part of the teacher education program or not)

When all information has been filled in, a survey will be created automatically, and then an email with a link to the survey will be sent to the course teachers. Teachers will have the opportunity to add questions. If the teacher does not want to add any questions, it is not necessary to open the survey at all. Then respondents/students are extracted from Ladok and the course evaluation is sent out. A reminder will be sent out shortly after. When the survey is closed, two reports are created automatically, one with written answers and one without.

The course teacher will receive an email when the reports are available. The teacher examines both reports to be able to make comments and work out an action plan. Then the teacher opens the report without written answers, makes comments, adds the action plan, approves the survey and sends it to be archived.

This step “semi-automates” the process, fully automating it will most likely be possible when we have Ladok 3 (autumn semester 2017).

In order to clarify the process, a flowchart is available to administrators who will be involved in the automation, as well as a flowchart that clarifies the process to course coordinators.