Interpretation of Course Evaluation Results

This page contains a summary of the Guidelines and Recommendations for the Interpretation of End-of-Course Evaluation Results [.pdf].

As a Teaching Assistant you are at the beginning of your teaching career and so stand to benefit significantly from feedback from students. These guidelines, especially on interpreting written comments, will help you benefit from the feedback; consultation with the instructor, academic unit head or trusted peers is strongly encouraged.


Definitions Used in the Course Evaluation System

Display of results for TAsDisplay of Mercury Graph Results for Teaching Assistants

Response Rate

The sum of the “Total Number of Completed Evaluations” and the "Total Number of Declined Evaluations," divided by the “Total Enrollment in the Course or Course Section," expressed as a percentage.

Valid responses

The number of students who selected one of the options provided for the question.

Blank responses

The number of students who did not select an option for the question. Blank responses are NOT included in any calculations.

Comments

The number of students who entered a comment for the question.

Mean

The sum of all “valid responses” divided by the number of “valid responses”. Note: The closer the mean is to 5, the closer it is to “strongly agree/ excellent."

Return to top.


Recommendations for Interpreting Numerical Results

When using course evaluations to improve and enhance your teaching, numerical results are most useful for identifying strengths and weaknesses while comments provide insights for reflection. When looking at the results it is important to consider the following:

Ratings of global items are the most useful as indicators of overall instructional effectiveness.

(e.g., “Overall  this instructor is an excellent teacher”; “I learned a great deal from this course”). Responses to these questions are found to correlate most consistently with measures of actual student achievement. Generally, results below 3.5 should be of concern, while 3.5 to 4 represent solid results, and mean scores over 4 are considered strong. As well, it is advisable to follow-up on any result that is more than .5 below or above the comparison mean (department, Faculty by level or class size).

Rating Meaning (in general)
< 3.5 Of concern
3.5 - 4 Solid
> 4 Strong

The mean is not sufficient to provide a picture of the distribution of responses.

When interpreting the numerical results, consider information such as the distribution of responses by item as well as the variation in responses. To understand the range of opinion, one should interpret the mean in conjunction with the shape and frequency of responses along the scale. Generally, differences that are less than .5 above or below the mean should be regarded as functionally equivalent.

Mercury results are reported to only 1 decimal place to avoid overemphasis on differences that are not meaningful.

If follow-up analyses are carried out on the data, do not look beyond 1 decimal place. Factors that have a statistically significant impact on course ratings do not usually result in meaningful differences.

Written comments provide the most useful information for teaching improvement.

This is because they can provide insight into why some students had difficulty learning or, conversely, why others succeeded. Written comments often help clarify and illuminate some of the observed numerical response patterns. (See the section below on Interpreting Students’ Written Comments.)

Course ratings are most useful in improving teaching effectiveness when they are coupled with appropriate consultation.

To help derive the most benefit from your results, we encourage you to discuss them with the course instructor, a trusted colleague, your academic unit head or someone from Teaching and Learning Services (TLS).

 

Sample Tables for Reporting of Results (.xlsx)

 

Use this template to report on a specific course or course over time; simply download the file and enter the data in the appropriate cells. Note that there are multiple sheets (tabs) in this document. View the Guidelines for Interpretation for additional recommendations on how to report numerical results (specific course, specific term, and patterns over time).

Return to top.


Recommendations for Interpreting Written Comments

Making sense of students’ written comments can be a daunting task; however, comments provide the insights and richness that help in understanding and interpreting student feedback. Approaching the comments in a systematic way can make the process easier and more meaningful.

Considerations

  • When reading students’ written comments, always balance them against the mean rating to keep them in context. Otherwise, negative comments may be given more weight than is appropriate.
  • In general, student comments are strongly correlated with all quantitative measures on the course evaluation.
  • Look for repeated patterns in the comments which can be useful to identify issues that are of importance to students as a whole. However, do not dismiss a comment out of hand if it comes from only one student.
  • Positive comments tend to be more general in nature, whereas negative comments tend to focus on a particular aspect of a course.
  • Comments on items such as scheduling, class length, timing and frequency or class composition tend to be critical. These items should be discussed with the academic unit head.
  • After reading through your students’ written comments, make an initial assessment. Overall, were the comments positive or negative regarding the course or your instruction?
  • Comments other than general ones usually fall into one of four main categories of topics:

    1. Instructor influence

    Comments concerning clarity and difficulty, teaching strategies, course activities, assignments and assessments address areas that the instructor has the most opportunity to adjust and improve. For example, the comment: “He should present the material in a more structured and organized manner,” refers to course design and organization, aspects that the instructor can address.

    2. Shared instructor/students

    Comments that fall in this area often reflect a shared responsibility between the instructor and the students and include topics such as interest and communication. For example a student’s course experience that elicits the comment: “Some students were just not interested in learning the material of this course” may have just as much to do with their motivation to learn in this area as it has to do with the instructor’s ability to facilitate an interesting course. An open discussion with students can help identify how these areas may be improved.

    3. Shared instructor/administration

    Comments related to organization and structure as well as the physical environment often require a concerted reaction from the instructor and the program administrator (i.e., Dean, Chair or Director). For example, the comment:  “This course should be open to students in their first year” should be brought to the attention of the administration for consideration in program design.

    4. Personal traits of the instructor

    Comments about the personal traits (for example, accent or apparent unfriendliness) of the instructor understandably often elicit strong emotions. These should be reflected upon and where appropriate and possible, potential strategies should be discussed with a trusted colleague or teaching support specialist.

 

Comments Analysis Worksheet (.docx)

 

Use this document to make sense of student comments. Often multiple comments are related to the same category (see comment categories below); for example, 10 students may all make comments about the assignments being unclear. This is not really 10 different comments but rather one comment 10 times. The multiple mentions give it weight, but it is only one area that needs to be addressed for improvement.

Comment categories:

  • Overall (course or instructor)
  • Clarity & difficulty
  • Organization & structure
  • Interest
  • Teaching Strategies
  • Assessment & feedback
  • Outside of classroom communications
  • Personal traits
  • Physical environment

Return to top.


Other Resources

IDEA Education - Linking Questions to Teaching and Learning

An extremely useful series of articles written by respected experts in teaching and learning in higher education, specifically linking questionnaire items to relevant aspects of teaching and learning.

The IDEA Center is a nonprofit organization which provides assessment, development and other resources to colleges and universities worldwide. It has produced, in collaboration with POD (The Professional and Organizational Development Network in Higher Education), an extremely useful series of articles written by respected experts in teaching and learning in higher education, specifically linking questionnaire items to relevant aspects of teaching and learning.

 

For more information on interpreting course evaluation results, please consult with an Academic Associate at Teaching and Learning Services.

Return to top.

McGill University is located on land which has long served as a site of meeting and exchange amongst Indigenous peoples, including the Haudenosaunee and Anishinabeg nations. McGill honours, recognizes and respects these nations as the traditional stewards of the lands and waters on which we meet today.