Data from 1,145 teacher evaluation forms, representing student responses in fifty-seven undergraduate courses are used to examine the role of instructor likeability on student perceptions of learning and ratings of overall teaching ability. The results suggest that students who rate their instructor high in likeability reward that instructor with high ratings in overall teaching ability. However, high likeability ratings are not associated with an increase in student perceptions of learning. The failure of likeability to effect perceived learning, juxtaposed with its positive impact on global teacher ratings, gives some credence to recent critiques of student consumerism in higher education. Namely, when students approach college as customers, they expect to be entertained and served only in ways they find pleasing. Suggestions for the more appropriate use of student evaluations of teaching (SETS) and their impact on how faculty define teaching effectiveness are discussed.
Faculty are acutely aware of the importance accorded to student evaluations of teaching (SETS) in tenure and promotion decisions. Most of us await student reactions to our courses with both eagerness and trepidation. We are relieved when our numerical ratings are at or above the institutional average, and gratified by comments such as “I learned a lot in this course,” “made difficult material understandable,” or “this class gave me a whole new understanding of myself.” We are confused or concerned, however, when we are described as “sweet” (this sounds vaguely unprofessional), and “nonjudgmental” (shouldn’t teaching involve some judgment?). When we suspect our ratings improve because, despite maintaining the same instrumental techniques and academic standards, we teach in a “kinder, gentler” manner (i.e., we are empathetic when explaining why “so much” reading is required), we begin to question the validity of our SETS. Are we being rated as effective teachers because we promote student learning, or because we have become more “likeable?”
Our concerns are not new to the discussion about the validity of student evaluations of teaching (see Greenwald, 1997, for a review). However, recent work has renewed the debate over the importance of rapport (i.e., likeability), effort, instructional behaviors, and learning on ratings of teaching effectiveness (d’Apollonia and Abrami, 1997a, 1997b; Greewald and Gillmore, 1997; Marsh and Roche, 1997; McKeachie, 1997; Williams and Ceci, 1997; Trout, 1997). Certainly, both likeability and students’ perceptions of learning contribute to an instructor’s overall teaching effectiveness, but what is the relative impact of each factor?
Both theory and research indicate that teaching is a multi-dimensional activity (d’Apolliana and Abrami, 1997a; Marsh and Roche, 1997), involving personal attributes of the instructor as well as qualities more traditionally considered hallmarks of effective pedagogy, such as expertise and clarity in presentation. Marsh and Roche (1997) note that repeated analyses of items on the Students’ Evaluation of Educational Quality (SEEQ) form identify nine teaching factors, including Individual Rapport and Instructor Enthusiasm. Ironically, within the SET literature, variables that theoretically could form the bases for “likeability” fall into two, often overlapping, categories: those studied as potential threats to SET validity, and those identified as important components of effective teaching.
The first category includes factors such as instructor expressiveness or enthusiasm, referred to as the “Dr. Fox” effect (Abrami, Leventhal and Perry, 1982). Studies using the “Dr. Fox” paradigm, in which instructor enthusiasm is systematically varied in either experimental or natural settings, have yielded competing interpretations. Abrami et al. (1982) completed a meta-analysis of experimental studies in which lecture content and instructor expressiveness were varied, and concluded that expressiveness had strong effects on overall SETS and weak effects on achievement, while content had the reverse pattern of effects. However, Marsh and Ware (1982) analyzed the same data and reported that when subjects were given incentives to perform (as they would be in a actual classroom), manipulations of expressiveness only affected their ratings on one SET variable: Instructor Enthusiasm. More recently, Williams and Ceci (1997) compared SETS from the fall and spring semester sections of the same course, taught by the same instructor using the same methods and content, who, based on skills learned from a media consultant, employed a “more enthusiastic presentation style” (p. 16). The instructor received significantly more favorable ratings from students in the spring section, even on aspects of the course unrelated to the manipulated variable (e.g., the textbook and grading policies).
Despite the debate over whether instructor enthusiasm constitutes evidence for or against the construct validity of SETS, variables reflective of instructor likeability remain on student rating forms, and continue to be discussed as components of effective teaching. The student rating form at our institution asks students to rate “how much you like the instructor and his/her style of presenting material” (italics in original). As noted above, Marsh and Roche (1997) identify rapport and enthusiasm as two of the nine factors responsible for teaching effectiveness. In a factor analysis of 458 items on 17 different student rating forms, d’Apollonia and Abrami (1997a) identified one of four factors of effective teaching to consist of behaviors related to concern for students, respect, and tolerance. Effective teachers, it is argued, exhibit concern for students, respect, and tolerance (d’Apollonia and Abrami, 1997a), provide their students with interpersonal support (DeZure, 1997), and are stimulating in style (Plater, 1997). In short, they are interesting and likeable.
While instructor likeability and student perceptions of learning may represent separate indicators of pedagogical effectiveness, the two may also interact to effect SETS. For example, students may like or develop rapport with a teacher because s/he promotes student learning, via effective delivery of information, pertinent assignments, fair grading, and so forth. Alternately, students who initially perceive a teacher as likeable, in contrast to those who do not, may be more attentive to the information that teacher delivers, work harder on assignments, be more receptive to methods of grading, and thus learn more. Presumably, the outcome of both scenarios would be more favorable SETS. Such situations, which benefit both student and teacher, are not the focus of the present study. Rather, our research question is: Can instructor likeability, relative to student perceptions of learning, exert a disproportionately strong effect on overall SETS?
This question is reflected in recent scholarship on the implications of consumerism and student disengagement in higher education (Bellah, 1999; Flacks and Thomas 1998; Long and Lake 1996; O’Brien and Howard 1996). Those concerned with these issues argue that student evaluations of teaching result in faculty reluctance to judge either students or ideas and have contributed to a culture of student consumerism (Delucchi and Smith, 1997; Sacks, 1996; Trout, 2000). Within this academic milieux, students do not expect a higher education to involve effort, challenge, or negative evaluation; rather, they expect to be entertained and amused (Edmundsen, 1997; Shepperd, 1997), to feel comfortable and to put forth little effort (Trout, 1997), to be rewarded liberally for self-disclosure, whatever its quality or form (Perillo, 1997), and to be given high grades in return for paying tuition and showing up (Delucchi and Korgen, 2000; Gose, 1997).
If these characterizations are at all accurate, then student evaluations of an instructor’s overall teaching ability may have less to do with learning, and much more to do with likeability. There is some empirical support for this perspective. Abrami et al. (1982) found a strong correlation between a potential likeability factor (i.e., instructor enthusiasm) and overall SETS. More recently, Williams and Ceci (1997) found that likeability in the form of enthusiasm contributed substantially to both overall ratings and student perceptions of learning, but not to an objective indicator of achievement.
Two important questions about the role of liking and learning in the college classroom have yet to be adequately answered. What is the relationship between instructor likeability and student perceptions of learning? What are the relative contributions of each to students’ overall ratings of teaching ability? In our investigation, we examine relationships among instructor likeability, student perceptions of learning, and ratings of teaching effectiveness. First, we construct indices to measure four factors (likeability, perceptions of learning, effort, and teaching behaviors) identified in a principle-components factor analysis of items on our institution’s student evaluation form. Second, we use multiple regression to assess the impact of likeability on perceptions of learning while controlling for the effects of effort and teaching behaviors. Finally, we employ regression to evaluate the relative effects of likeability and perceptions of learning (together with several other independent variables) on students’ ratings of their instructor’s overall teaching ability.
We obtained our sample from a state-supported baccalaureate degree-granting university. The student population at the institution is ethnically diverse (59% ethnic minorities) and comprised predominantly of nontraditional students (e.g., 79% are 25 years of age or older). Data derive from student responses to a 34-item course evaluation form administered at the end of the fall semester of 1996. The instrument asks students to rate their instructor in a variety of areas, including preparation, rapport, enthusiasm, workload, and overall teaching ability. We analyze data from 1,145 evaluations, representing student responses in 57 courses, in the present study.
The evaluation form does not include information on student demographic characteristics, an omission designed to ensure student anonymity. Consequently, our sample and the student population cannot be compared on demographic characteristics. However, course specific information indicates that our sample and the campus population are comprised of similar proportions of students enrolled in each discipline area (social sciences: psychology, sociology, etc.; humanities: history, literature, etc.; and professional studies: business, public administration, etc.).
Our primary dependent variable is derived from a question on the course evaluation form that asks students to rate their instructor’s overall teaching ability. We chose this item for three reasons. First, our study’s principal objective is to assess how students’ global ratings of their instructor’s teaching effectiveness are impacted by likeability and perceptions of learning. Second, our professional experiences indicate that global ratings are disproportionately scrutinized (relative to other evaluation form items) by faculty review committees when assessing the teaching effectiveness of colleagues. Third, factor analytic studies suggest that most SETS are dominated by a large global factor (d’Apollonia and Abrami, 1997a). Therefore, students’ evaluation of their instructor’s overall teaching ability, is measured from the “low” (= 1) to “high” (= 5) responses to the following question: “How would you rate the instructor’s overall teaching ability?”
A principle-components factor analysis of the 34-item course evaluation form resulted in the identification of four factors: instructor likeability, student perceptions of learning, teaching behaviors, and effort. Subsequently, we developed indices to measure each of the four factors. In addition, we retained six other items that failed to load on the four factors, but were deemed pertinent as potential predictors of student perceptions of learning and overall teaching ability. These serve as control variables. The four indices and six control items were the independent variables in our regression model for students’ overall rating of their instructor’s teaching ability. Descriptive statistics and coding information for all items used in our analyses appear in Table 1.
Variable Indicator Mean S.D. --------------------------------------------------------------------- Ability How would you rate the 4.55 0.84 instructor's overall teaching ability? "low" (= 1) to "high" (= 5) Likeability(LI) Index of instructor likeability: 4.44 1.14 "low" (= 0) to "high" (= 5) Learning(PLI) Index of student perceptions of 4.24 0.74 learning: "low" (= 1) to "high" (= 5) Teaching(TI) Index f teaching behaviors: 4.48 0.74 "low" (= 1) to "high" (= 5) Effort(EI) Index of student effort: 3.01 1.65 "low" (= 0) to "high" (= 5) Knowledge Instructor had adequate 4.83 0.47 knowledge of subject matter: "hardly ever" (= 1) to "almost always" (= 5) Goals Instructor clearly stated goals 4.57 0.83 of the course: "hardly ever" (= 1) to "almost always" (= 5) Consultation Instructor available for 4.51 0.79 consultation outside of class hours: "hardly ever" (= 1) to "almost always" (= 5) Assignments Instructor gave outside-of- 4.48 0.85 class assignments which were reasonable: "hardly ever" (= 1) to "almost always" (= 5) Positive As a result of taking this 0.77 0.42 course, I have more positive feelings toward this field: "false/in between" (= 0); "true" (= 1) Attention Student rating of their 4.36 0.84 attention during class sessions: "low" (= 1) to "high" (= 5)
Our Likeability Index (LI) is comprised of five evaluation form items. First, we coded “hardly ever/occasionally/sometimes” (= 0) and “frequently/almost always” (= 1) for the following items: 1) Had good rapport with students; easy to talk to; 2) Seemed enthusiastic about the subject matter; and 3) Created a feeling of community among students. Second, we coded “false/in between” (= 0) and “true” (= 1) for the item: I would like to take another course from this instructor. Third, we coded the “low/average” (= 0) to “high” (= 1) for the following question: How much did you like the instructor’s classroom manner and his/her way of presenting the course material? Likeability is measured by a summation of the five items. Exploratory factor analysis indicates that each item contributes to reliability. The index ranges from 0 to 5, and was judged to be reliable (Cronbach’s alpha = .86).
The Perceived Learning Index (PLI) is a summated measure of the “low” (= 1) to “high” (= 5) student responses to the following six course evaluation items: 1) Gaining in factual knowledge; 2) Developing skill in expressing myself orally and in writing; 3) Learning to apply course material to improve rational thinking, problem-solving and decision-making; 4) Learning how professionals in this field go about the process of gaining new knowledge; 5) Learning fundamental principles, generalizations, and theories; and 6) Developing the ability to function more creatively. The items were added together and divided by six to produce index total scores ranging from 1 through 5. Exploratory factor analysis indicated that each item contributed to reliability, and the PLI was deemed reliable (Cronbach’s alpha = .92).
The Teaching Behaviors Index (TI) is a summated measure of the “hardly ever” (= 1) to “almost always” (= 5) responses, divided by six, to the following course evaluation items: 1) Was able to explain difficult concepts; gave easy-to understand examples; 2) Created a desire to learn; 3) Made presentations which were interesting and informative; 4) Related material to real life situations; 5) Summarized materials in a manner which aided retention; and 6) Introduced stimulating ideas about the subject. Total scores on the TI range in value from 1 through 5. Exploratory factor analysis of the six items indicated one factor, and the index produced high reliability (Cronbach’s alpha = .94).
Student effort (EI) is operationalized as an index from a summation of five items. We coded “false/in between” (= 0) and “true” (= 1) for the question: I worked harder on this course than on most college courses I’ve taken. Scores on this item were added to the “low/average” (= 0) to “high”
(= 1) responses for the following items: 1) Amount of reading; 2) Amount of work in other (non-reading) outside assignments; 3) Difficulty of subject matter; and 4) Degree of intellectual exertion to meet course requirements. Index total scores range from 0 through 5. Exploratory factor analysis indicated that each item contributed to reliability, and the Effort Index (EI) was judged to have adequate reliability (Cronbach’s alpha = .79).
Multiple regression techniques illuminate the data. Regression coefficients represent the effect of a one unit change in the independent variable (X) on the dependent variable (Y), while controlling for the effects of all other independent variables. We use ordinary least squares (OLS) regression to: 1) appraise the impact of likeability on student perceptions of learning; and 2) evaluate the net effect of likeability and perceptions of learning on students’ overall ratings of their instructor’s teaching ability. Standardized regression coefficients were computed to assess the relative importance of each independent variable on the dependent variable.
Our analysis begins with an examination of the impact of likeability on perceptions of learning. In Table 2, perceived learning (PLI) serves as the dependent variable in a regression model that includes likeability (LI), teaching behaviors (TI), effort (EI), and six other variables. The model’s nine independent variables account for 64 percent of the variance in PLI.
Unstandardized Standardized Variable Coefficients SE Coefficients Likeability(LI) -.025 .020 -.037 Teaching(TI) .513*** .042 .511 Effort(EI) .060*** .009 .134 Knowledge .001 .036 .001 Goals -.021 .024 -.024 Consultation .056* .022 .059 Assignments .054* .022 .061 Positive .364*** .039 .207 Attention .127*** .020 .143 Intercept .646*** R-square .64 Note: N = 1145 * P < .05, two-tailed test of significance ** p < .01, two-tailed test of significance *** p < .001, two-tailed test of significance
The results in Table 2 reveal that likeability (LI) does not have a statistically significant effect on PLI. The regression coefficient for teaching behaviors (TI) has a positive and statistically significant impact on PLI. Specifically, holding constant the effects of all other independent variables, a one unit increase in (TI) produces a .51 increase in perceived learning (PLI). An examination of the standardized coefficients indicates that TI is the strongest determinant (Beta = .51) of PLI. The effort index (EI) also has a positive and significant effect on PLI, producing a .06 increase in perceived learning (PLI). Four control variables (i.e., availability for consultation, reasonableness of outside-of-class assignments, positive feelings toward field as result of the class, and attention during class) have significant positive effects on perceived learning.
The dependent variable in Table 3 is students’ overall rating of their instructors’ teaching ability. The results of multiple regression indicate that the four indices (likeability, perceived learning, teaching behaviors, and effort) and six control variables account for 76 percent of the variance in ratings.
Unstandardized Standardized Variable Coefficients SE Coefficients Likeability(LI) .247*** .019 .337 Learning(PLI) .100*** .028 .086 Teaching(TI) .345*** .041 .303 Effort(EI) .004 .008 .007 Knowledge .169*** .034 .088 Goals .121*** .022 .119 Consultation -.071** .020 -.067 Assignments .008 .020 .008 Positive .011 .037 .006 Attention .138*** .019 .137 Intercept -.163 R-square .76 Note: N = 1145 * P < .05, two-tailed test of significance ** p < .01, two-tailed test of significance *** p < .001, two-tailed test of significance
The coefficient for the likeability index (LI) has a statistically significant and positive effect on overall ratings. Holding constant the effects of all other independent variables, a one point increase in LI produces a .25 increase in students’ rating of their instructor’s teaching ability. Furthermore, the standardized coefficient (Beta = .34) for likeability is the largest among all independent variables. Perceived learning (PLI) also has a positive effect on the dependent variable, producing a .10 increase in overall ratings of teaching ability. However, an examination of the standardized coefficient (Beta = .09) reveals that the impact of PLI on ratings is less than one third the magnitude of likeability.
The coefficient for teaching behaviors (TI) has a positive and statistically significant effect on overall ratings. The standardized coefficient (Beta = .30) for teaching behaviors is slightly smaller than that for likeability (LI). The effort index (EI) does not have a significant effect on teacher ratings. Finally, four control variables have significant effects on overall ratings. Knowledge of subject matter, stated goals of course, and reasonableness of outside-of-class assignments increases overall ratings, while availability for consultation has a negative effect on the dependent variable.
Perceived learning (PLI) was predicted by indices of teaching behaviors (TI) and effort (EI), and four control variables (attention, positive feelings about field due to course, consultation availability, and reasonableness of assignments); the likeability index (LI) was not a significant predictor of PLI. However, of the four indices, likeability (LI) was the strongest predictor of students’ overall ratings of teaching ability, followed by teaching behaviors (TI) and perceived learning (PLI). Four of the six control variables (attention, instructor knowledge, stated goals, consultation availability) also predicted overall ratings. The effort index (EI) was not a significant predictor of students’ overall ratings of teaching ability. Our data reveal no relationship between instructor likeability and perceived learning, but indicate a strong association between likeability and overall ratings of teaching ability.
Our results both support and challenge the validity of student evaluations of teaching effectiveness. Salient teaching behaviors (TI) account for substantial variation in both students’ perceptions of learning (PLI) and in their overall ratings of teaching ability. This is reassuring, for it implies that instructors who clearly explain concepts, make interesting and informative presentations, and summarize course content will increase students’ perceptions of learning, and be evaluated as effective teachers. Furthermore, students’ ratings of their own effort in a course (EI) account for variation in perceived learning (PLI), and PLI is a significant predictor of student ratings of overall teaching ability. This is logical. Students should perceive that they learn more in courses in which they work harder, and we would expect that increased perceptions of learning would result in greater overall ratings of the instructor.
The fact that instructor likeability (LI) does not predict perceived learning (PLI) might also be reassuring, except when considered in light of likeability’s effect on student ratings of overall teaching ability. Arguably, perceived learning should primarily be a function of teaching behavior and student effort. Students should feel they learn because we work hard and they work hard, rather than because they like us. And accordingly, salient teaching behaviors, effort, and perceptions of learning should be the most important predictors of ratings of overall teaching ability. This, however, is not the case in the present study. Instructor likeability, while exerting no appreciable effect on perceived learning, has a substantial positive impact on overall ratings. The magnitude of the likeability effect is cause for concern. As a predictor of students’ overall ratings, likeability has three times the effect of PLI. Moreover, effort (EI) failed to emerge as a significant predictor. The results suggest that students who give their instructor high likeability ratings reward that instructor with high ratings in overall teaching ability. Yet high likeability ratings are not associated with any increase in perceptions of learning.
Could it be that despite their perception that they learn more, students are unwilling to give high global ratings to instructors who make them work harder than students believe is appropriate? Even if we assume that likeability has a valid role in effective pedagogy, the implication is disturbing: instructors who emphasize likeability can produce higher overall ratings than those who stress effort and enhance perceptions of learning. And of course, students may like an instructor for behaviors other than his/her warmth and rapport, for actions actually inimical to effective teaching and learning, such as lenient grading and superficial assignments (see Greenwald,1997,and Greenwald and Gillmore, 1997, for review and discussion of the grades-SETS relationship).
Our findings must be qualified by the limitations of the study. Our data were collected at a single institution, which may preclude generalizations to other colleges and universities. The use of a questionnaire that was not designed a priori to discriminate between likeability and perceived learning may have limited the precision of our measured constructs. While the factors we identified were similar to those reported in other studies of SETS, there is subjectivity in any interpretation of a principle-components factor analysis. Nevertheless, our results support recent work indicating potential biases in student course evaluations, particularly those involving global ratings of teaching effectiveness (Marche and Roche, 1997; McKeachie, 1997). Rather than renewing the debate over the validity of SETS, we discuss what our data imply for 1) how faculty define teaching effectiveness, 2) how faculty definition(s) may differ from those of our students, and from those of administrators and peers, and 3) how institutions can address potential validity problems when devising evaluation methods, and when applying these in professional development and personnel decisions. We also suggest ways to further evaluate the liking-learning relationship in students’ responses to their instructors.
While cogent and compelling arguments have been made for viewing effective teaching as a multi-dimensional activity (e.g., Marsh and Roche, 1997), we contend that, ultimately, it must be defined as promotion of student learning. Multiple teaching dimensions may contribute to learning, and among these may be factors related to likeability (e.g., expressiveness, rapport) as well as those related to more teaching-specific behaviors (e.g., skill in presentation, use of examples). Student learning is also a function of variables unrelated to the teacher, such as student ability and prior knowledge of subject. Nevertheless, if we continue to rely on students as evaluators, we should not limit our questions to how much they like us, how interesting our courses are, or even how skilled we are as presenters. Our questions should center on how much we help them to learn. Assuming learning is the primary goal of teaching, such questions provide a more direct measure of teaching effectiveness (McKeachie, 1997).
Our research indicates students do not define teaching effectiveness primarily in terms of their own learning experience. The unimportance of likeability in predicting learning, juxtaposed with its significance in predicting global SETS, gives credence to concerns about student consumerism in higher education (Delucchi and Smith, 1997). McKeachie (1997) suggests that students have misconceptions about what constitutes effective teaching, and may benefit from training in how to evaluate their own learning. This may be especially salient if, as critics of student consumerism contend, students expect to be entertained and protected, rather than challenged to achieve and grow (Edmundsen, 1997; Trout, 1997). Consequently, even students with a sincere desire to learn may give more weight to how much they like the course or instructor, particularly when confronted with items that encourage such a focus. We recommend that, in addition to revising SET measures to include items that assess student learning as well as teaching style, we elicit our students’ expectations about teaching and learning, present our own, and make explicit the areas of convergence and divergence. Such a dialogue can be revisited at appropriate points during the course (e.g., when introducing the syllabus, when giving the rationale for assignments or grades) and may provide a context conducive to more relevant SETS.
We must also recognize that administrators and faculty who make use of SETS in personnel decisions may do so without awareness of their limitations, and with conflicting and perhaps inappropriate definitions of teaching effectiveness. This may result in committees misinterpreting evaluations, e.g., concluding that numeric ratings that fall within one standard deviation of, but also one tenth of a point below, college-wide means are evidence of poor teaching (McKeachie, 1997). In response to decreasing enrollments, institutions may overvalue instructors who have full courses and high global ratings because of grading leniency (Greenwald and Gillmore, 1997) or theatrical style (Abrami et al., 1982), but who contribute little to student learning. Consequently, faculty who fail to conform to student perceptions (e.g., likeable) of effective teachers may be unfairly penalized (McKeachie, 1997).
Other methods of ensuring more appropriate use of SETS include hiring outside consultants to interpret ratings (McKeachie, 1997), statistically adjusting ratings to correct for grading leniency (Greenwald and Gillmore, 1997), and requiring that SETS only be considered in the context of other measures of teaching effectiveness, such as peer evaluations, instructors’ statements of their own teaching philosophies and goals, teaching portfolios, and data on grade distributions. A recent proposal at our institution to provide instructors data that would enable them to compare their course grades with college-wide grade distributions was met with strenuous objections. This resistance remains despite repeated assurances that such data would not be included in personnel files or used in tenure and promotion decisions. At the very least, faculty and administrators must engage in open discussion of potential problems in the use of SETS. This may be best achieved in the context of training seminars that disseminate current information on the use and misuse of SETS.
In addition to indicating a need for continued revision of SETS and caution in their use, our study suggests several directions for future research. First, it may be valuable to conduct both within and between-class comparisons of SETS designed specifically to assess perceived learning with responses to alternative instruments designed to evaluate instructor likeability. Such measures may stimulate very different responses when students are asked for global ratings of teaching effectiveness. Second, we agree with McKeachie’s (1997) contention that we need qualitative data on criteria students use when completing course evaluations. The relationship, or lack thereof, between likeability and learning in the present study indicates that students make a distinction between these variables when evaluating their instructors. Asking them about this distinction and how it influences their overall course evaluations might make for better design and use of SETS. Third, we urge educators not to abandon efforts to develop and implement objective indices of student achievement, such as pre- and post-course examinations. Such data, in combination with SETS and measures of student effort and relevant pre-course variables (e.g., ability, interest, motivation), would increase our ability to identify instructor contributions to student learning.
Abrami, Philip C., L. Leventhal, and R.P. Perry (1982). Educational Seduction. Review of Educational Research 52:446-464.
Bellah, Robert N. (1999). Freedom, Coercion, & Authority. Academe, January/February, pp. 16-21
d’ Apollonia, Sylvia and Philip C. Abrami (1997a). Navigating Student Ratings of Instruction. American Psychologist 52:1198-1208.
d’ Apollonia, Sylvia and Philip C. Abrami (1997b). In Response to Trout, P.: What the Numbers Mean. Change, September/October, pp. 18-19.
Delucchi, Michael and William L. Smith (1997). A Postmodern Explanation of Student Consumerism in Higher Education. Teaching Sociology 25:322-327.
Delucchi, Michael and Kathleen Korgen (2000). We’re The Customer--We Pay The Tuition: Student Consumerism Among Undergraduate Sociology Majors. Division of Social Sciences, University of Hawaii-West Oahu, Pearl City, Hi. Unpublished manuscript.
DeZure, Deborah (1997). In Response to Trout, P.: What the Numbers Mean. Change, September/October, pp. 26-27.
Edmundsen, Mark (1997). As Lite Entertainment for Bored College Students. Harper’s Magazine, September, pp. 39-49.
Flacks, Richard and Scott L. Thomas (1998). Among Affluent Students, a Culture of Disengagement. Chronicle of Higher Education, November 27, p. A48.
Gose, Ben (1997). Efforts to Curb Grade Inflation Get an F From Many Critics. Chronicle of Higher Education, July 25, pp. A41-A42.
Greenwald, Anthony G. (1997). Validity Concerns and Usefulness of Student Ratings of Course Instruction. American Psychologist 52:1182-1186.
Greenwald, Anthony G. and Gerald M. Gillmore (1997). Grading Leniency is a Removable Contaminant of Student Ratings. American Psychologist 52:1209-1217.
Long, Gary L. and Elise S. Lake (1996). A Precondition for Ethical Teaching: Clarity About Role and Inequality. Teaching Sociology 24:111-116
Marsh, Herbert W. and Lawrence A. Roche (1997). Making Students’ Evaluations of Teaching Effectiveness Effective: Critical Issues of Validity, Bias, and Utility. American Psychologist 52:1187-1197.
Marsh, Herbert W. and J.E. Ware (1982). Effects of Expressiveness, Content Coverage, and Incentive on Multidimensional Student Rating Scales: New Interpretations of the Dr. Fox Effect. Journal of Educational Psychology 74:126-134.
McKeachie, Wilbert J. (1997). Student Ratings: The Validity of Use. American Psychologist 52:1218-1225.
O’Brien, Jodi and Judith A. Howard (1996). To Be or Not To Be: The Paradox of Value-Neutrality and Responsible Authority. Teaching Sociology 24:326-330.
Plater, William M. (1997). In response to Trout, P.: What the Numbers Mean. Change, September/October, p. 16.
Perillo, Lucia (1997). When the Classroom Becomes a Confessional. Chronicle of Higher Education, November 28, p. A50.
Sacks, Peter (1997). In response to Trout, P.: What the Numbers Mean. Change, September/October, p. 29.
Shepperd, Jerry C. (1997). Relevance and Responsibility: A Postmodern Response. Teaching Sociology 25:333-335.
Trout, Paul M. (1997). What the numbers mean: Providing a Context for Numerical Student Evaluations of Courses. Change, September/October, pp. 24-30.
Trout, Paul M. (2000). Flunking the Test: The Dismal Record of Student Evaluations. Academe, July/August. pp. 58-61.
Williams, Wendy M. and Stephen J. Ceci (1997). How’m I Doing? Problems with Student Ratings of Instructors and Courses. Change, September/October, pp. 13-23.
© Radical Pedagogy