In 2003, McGill began to transition from paper course evaluations (administered in class) to an online system. The table below summarizes changes to the course evaluation system and process at McGill from 1992 to the present. The arrow ➤ indicates "status quo."
|
YEAR →
|
1992 - 2003 Paper Evaluations |
2003 – 2019 |
2003 - 2006 Hybrid Paper & Online |
2006 - 2011 Online as Official System |
2011 - 2014 |
2014 - Present |
2019 – Present |
|---|---|---|---|---|---|---|---|
|
Method |
Paper and pencil, in-class |
➤ |
➤ |
||||
| System | In-house system built within Banner | ➤ | ➤ | ➤ | ➤ | Implementation of third-party system, Blue, by Explorance | |
|
Questionnaires |
No common questions across University |
➤ |
➤ |
➤ |
|||
| Questionnaire length |
Up to 60+ questions |
➤ |
Maximum 25 questions |
➤ |
➤ |
||
|
Timing |
1 day in last two weeks of class |
Last 2 weeks of class |
Last 3 weeks of class |
Default period until end of classes; option of extended period until end of exams |
Default period until end of exams; option of condensed period until end of classes |
||
|
Comments |
Handwritten and (usually) retyped by staff |
➤ |
➤ |
➤ |
|||
|
Reporting & Analysis |
Departmental level only; no Faculty or University analysis possible |
➤ |
Level and class size analyses regularly reported |
➤ |
|||
| Level of Precision |
Over-precision (.001) |
Appropriate precision (.1) |
➤ |
➤ |
➤ |
||
| Interpretation |
No guidelines for interpretation |
➤ |
➤ |
➤ |
|||
| Availability of results |
Delays of up to 4 months for results |
Results available on submission of grades |
➤ |
➤ |
➤ |
||
|
Data Security |
Ad hoc |
➤ |
➤ |
➤ |
|||
|
Dissemination |
Available in the libraries |
Available online with explicit permission and response threshold met |
➤ |
Course-by-course dissemination exceptions possible |
Available online if instructor does not object and response threshold met |
||
|
Environmental Impact |
±480,000 sheets of paper per year for questionnaire completion only |
Saved ±480,000 sheets of paper per year |
➤ |
➤ |
➤ |
1980
In 1980, Senate passed a set of recommendations concerning teaching evaluations. In summary, they specified among other things that:
-
there would be no uniform system of teaching evaluation,
-
results of the evaluations would be given to professors and reviewed by chairs, and
-
wide range of student opinion would be solicited in the evaluation of teaching.
1992
In 1992, Senate approved recommendations concerning course evaluations, which had become the principal mechanism for the evaluation of teaching. These recommendations included:
-
Course evaluations are comprised of two parts: Part I with course and instructor related questions, and Part II with specific questions concerning the course and the instructor, anecdotal data and verbatim comments.
-
Data from Part II is always regarded as confidential.
-
Questions in Part I are answered on a scale of 1-5
-
Where no formal change in content or instructor has occurred, students have access to evaluations from previous terms.
-
The department/school/faculty (academic unit) is responsible for the administration of course evaluations, including the placing of the evaluations in the relevant libraries.
Why did McGill move to an online system?
The paper-based method for conducting end-of-term student evaluations of courses, including preparation of questionnaires, data collection, analysis and communication of results to professors was time consuming and labor-intensive. Mercury provides several important advantages, most notably:
-
Providing students the opportunity to reflect on their responses rather than give on-the-spot answers in a time-pressured class environment;
-
Providing students who miss the class when the evaluations are completed the opportunity to express their views;
-
Spreading out the evaluation process over three to six weeks near the end of term, allowing students the opportunity to complete the evaluations at personally convenient times and avoiding the “evaluation overload” phenomenon that can result when students are asked to evaluate five courses in one week;
-
Eliminating the need to retype student comments to preserve anonymity;
-
Easily allowing for customization of the course evaluation form to reflect the characteristics of different courses;
-
Improved turn-around time for results;
-
Eliminating the paper required for data collection, analysis and communication of results; conservative estimates indicate that Mercury eliminates the need for approximately 480,000 sheets of paper (96 boxes) annually.
