RateMyProfessors.com is more accurate than you might think

http://www.insidehighered.com/news/2008/04/25/rmp

April 25

Validation for RateMyProfessors.com?

“You’ve heard the reasons why professors don’t trust RateMyProfessors.com, the Web site to which students flock. Students who don’t do the work have equal say with those who do. The best way to get good ratings is to be relatively easy on grades, good looking or both, and so forth… But what if the much derided Web site’s rankings have a high correlation with markers that are more widely accepted as measures of faculty performance? Last year, a scholarly study found a high correlation between RateMyProfessors.com and a university’s own system of student evaluations. Now, a new study is finding a high correlation between RateMyProfessors and a student evaluation system used nationally. A new study is about to appear in the journal Assessment & Evaluation in Higher Education and it will argue that there are similarities in the rankings in RateMyProfessors.com and IDEA, a student evaluation system used at about 275 colleges nationally and run by a nonprofit group affiliated with Kansas State University.  What is notable is that while RateMyProfessors.com gives power to students, IDEA gives a lot of control over the process to faculty members. Professors identify the teaching objectives that are important to the class, and those are the measures that count the most. In addition, weighting is used so that adjustments are made for factors beyond professors’ control, such as class size, student work habits and so forth — all variables that RateMyProfessors doesn’t really account for (or try to account for).  The study looked at the rankings of 126 professors at Lander University, in South Carolina, and compared the two ratings systems. The findings:

• Student rankings on the ease of courses were consistent in both systems and correlated with grades.
• Professors’ rankings for “clarity” and “helpfulness” on RateMyProfessors.com correlated with overall rankings for course excellence on IDEA.
• The similarities were such that, the journal article says, they offer “preliminary support for the validity of the evaluations on RateMyProfessors.com.”