Using comments from end-of-course evaluations

Topics/tags: Rants, Grinnell

Last year, while I was conducting a faculty review, I encountered the following text while reading one of the sets of instructions [1].

[For interim reviews] The dossier usually consists simply of this departmental letter, the most recent Faculty Activity Report, and the curriculum vitae of the faculty member, along with course lists, enrollments, and summary quantitative data, as well as qualitative comments from end-of-course evaluations provided by the Dean’s office. [2,3]

[For complete reviews] Summaries of numeric data, as well as qualitative comments from end-of-course evaluations are provided to the review chair and Personnel Committee for their use. [4]

I was surprised, to say the least. Why? Because when the faculty approved the use of campus-wide end-of-course evaluations, we explicitly voted not to provide those comments to the Personnel Committee [5]. In addition, the instructions we read aloud when distributing end-of-course evaluations include the following statement.

Instructors receive the completed, original forms only after grades have been submitted to the Registrar. Department chairs receive copies of the completed forms. These are the only people who may read your comments unless the faculty member or the department chair chooses to include anonymous quotes in documents prepared for the faculty member’s next review. [6,7]

So I did what you’d expect me to do. I visited the College archives to find the original legislation [8], I dug out the instructions from the end-of-course evaluations, and, using the documents that I had dug out, I verified my recollections. Then I wrote a letter to the Dean expressing my concerns. I also talked to a few senior colleagues, who were as surprised as I was. Some were surprised for other reasons; I recall that one of them said something like, Personnel already has so much to read; I can’t imagine them wanting to read all of the end-of-course evaluation comments, too [9,10]. I heard back from the Dean that he was surprised to see all this, but that he understood my concerns. He even reaffirmed my interpretation at the faculty meeting of 15 October 2018.

As a follow up, Dean Latham acknowledged that our practice of ensuring that textual comments in student end-of-course evaluations will continue to be shared solely with the faculty member and chair. [11]

I thought things were settled.

It turns out that I was wrong.

This fall, a new set of guidelines appeared [12]. Although the name at the top of the document is different, the content is similar. The statements about qualitative comments are the same. I was concerned, to say the least. So I sent a letter of concern to Dean Harris and some members of Council. A few weeks later, I got a response. There were two parts to the response.

First, I was told that a recent EC/Personnel Committee meeting expanded the use of EOCEs to include qualitative data. Second, I was told that the faculty vote on EOCEs again restricted that use to department chairs (or designees) who were reviewing outliers.

The first response is absurd. Council and Personnel originally proposed the use of EOCEs to the full faculty. In response, the faculty explicitly limited the use. It should not be within the purview of Council and Personnel to again expand that use without the approval of the faculty as a whole [15]. It’s actions like this that make me worry about what I consider a dangerous shift of how faculty governance is interpreted at Grinnell; under President Kington, it seems to be much more representative governance, along with a view that the representatives can make decisions without consulting the faculty as a whole, rather than an understanding that there are benefits from involving the whole faculty.

The second response is puzzling. I don’t recall an explicit discussion of the use of EOCE comments in the broader discussion of EOCEs. It’s certainly within the purview of the Department or Review Chair to use end-of-course-evaluation comments to use those comments to provide additional information on outliers. But it’s also within the purview of the Department or Review Chair to choose to use comments for other reasons. I had thought things had stayed the same. I should also admit that I voted against the use of end-of-course evaluations during the two-year study period that we just began.

Once again, I also had a conversation with a more senior faculty member. They noted that it could be argued that the Personnel Committee implicitly has access to the comments since the Faculty Handbook explicitly states that they may seek other information. I checked, and I think this is what they were referring to.

The Dean or the Faculty Personnel Committee may return the departmental report to the review chair for additional information or evaluation. The Faculty Personnel Committee also may seek information on its own initiative. [16]

I understand how some might read that as giving access to the EOCE comments, but I don’t think it does. The faculty voted not to give access to those comments, and the instructions explicitly say that they are not made broadly available, so I don’t think PC’s initiative should override that. Personnel can’t look at my tax returns, my advisee folders, whatever documents I keep in my lower-right-hand desk drawer, and such. They can seek that information, but I don’t think others should provide it to them.

As you can likely tell, all of this frustrates me. But it also puzzles me. Why am I the only one who seems to have noticed that this had happened? Why am I the only one who seems to have complained? If I’m the only one who notices or cares, does any of this matter? And, given the potential legal issues surrounding faculty reviews, why do administrators allow these kinds of things to happen?

I have no answer to any of those questions. I wish I did.

For now, my biggest concerns are what I see as a change in our model of faculty governance and the question of whether I should stop caring about these kinds of things.

Postscript: I’m not sure whether or not we should share end-of-course evaluation comments with the Personnel Committee. On the one hand, being able to read the comments would give PC better insight into the bias in evaluations. On the other hand, it’s not clear that the members of PC have the training to understand the possible biases [17]. I would not give the comments to the Council (more precisely, the subset of Council that forms the Faculty Salary Committee). Their work is already busy enough; having to read hundreds of EOCEs for each of the fifty-or-so faculty they review each year would be nearly impossible.

However, I do think we should share comments from end-of-course evaluations with the Task Force that is studying EOCEs and making recommendations for the future of EOCEs. I would recommend that the faculty vote to give that access to the Task Force. I assume the Task Force would have to ask, first.

Postscript: As I read about other institutions around the country, especially places in which the Board installs Provosts and Presidents without consulting the faculty, or in opposition to faculty voices, I feel quite happy about faculty governance at Grinnell. But I’ll still keep trying to make it better.

Postscript: I had worried that the inappropriate policies had been in effect for a few years before I noticed the text. It appears that it was a reasonable concern. I was able to dig out the 2017-2018 guidelines, and the document entitled Reviews of Faculty has the same text. However, the checklist for the dossiers in the documented entitled Guidelines for Faculty Reviews does not. That’s probably why I missed it; I had read only the longer document, and not the shorter one. I wonder if that’s why others have also missed it.

Postscript: I am not on the Faculty Evaluation Task Force. If I weren’t on leave, I would have considered volunteering. In the past, I might have volunteered while on leave; I certainly did way too much on campus during my prior leaves [18]. This time, I’m doing my best to protect my time.

[1] There are normally at least two sets of instructions. One is called Reviews of Faculty and the other is called Guidelines for Faculty Reviews.

[2] Latham, Michael. Reviews of Faculty - Grinnell College 2018-2019. Online document, no longer available.

[3] Emphasis mine.

[4] Same document as above. Emphasis also mine.

[5] You can read my history of EOCEs for more details.

[6] Grinnell College Office of Analytic Support and Institutional Research. n.d. End-of-Course Student Ratings of Instructions. Printed documentation that accompanied written end-of-course evaluations for at least the past decade.

[7] Emphasis mine, but you knew that.

[8] I also found a followup proposal I wrote while I was on Council.

[9] Sorry, I’m not going to give the source of that (approximate) quotation.

[10] I’ve never served on the College Personnel Committee. But I have served on the Science Division Personnel Committee. I agree that there’s a lot to read. And I don’t ever recall having received the comments from people’s end-of-course evaluations.

[11] Lof, Laura. 2018. Minutes of the Faculty Meeting of October 15, 2018. Available online at https://grinco.sharepoint.com/sites/dean/_layouts/15/Doc.aspx?sourcedoc=%7B673ACCF3-CE06-44D4-BCE5-D82563F551A9%7D&file=FacMtgMin_10_15_18.docx&action=default&mobileredirect=true&CT=1570229420224&OR=ItemsView.

[12] Harris, Anne. 2019. Reviews of Faculty - Grinnell College, 2019-2020. Online document, available at https://grinco.sharepoint.com/sites/dean/Fac_Resources/Faculty_Review_Summary.pdf [14].

[14] That document is behind a password wall and is available only to people with an account on GrinCo.

[15] When I was on Council, and we wanted access to summary data on only two questions, we brought the question to the faculty as a whole. We also promised to do appropriate comparative evaluations of those data, which we did. Unfortunately, I’ve been told that our successors did not.

[16] Grinnell College. 2019. Faculty Handbook. Last reviewed on 5/24/19. Approved June 2019. p. 39.

[17] Yes, I realize that that’s also an issue with the Dean’s Survey.

[18] Including, but not limited to, chairing the department, teaching experimental classes, and taking classes.

Version 1.0 of 2019-10-06.