Reading AP CS Principles, Take 4

People have been asking for more comments on the reading. I drafted this musing on my last day of CS principles. It seemed like a good idea to take a break before finishing it and posting it. I think I’m ready.

I have now survived a week and two days of serving as a Table Leader at the reading of AP CS Principles. It is an appropriate time to reflect back on my time reading and leading.

A high point is that I got to work with some really awesome people. I was fortunate to have been assigned a fun and thoughtful group of eight readers. The two other TLs in the room and I formed a productive team, sharing responsibilities well [1]. Their readers were also pretty awesome, which made us a fun, thoughtful, and relatively cohesive room. The broader group of TLs were an interesting, social, and supportive group. The Question Leaders and the Chief Reader had to deal with repetitive questions and complaints from me, and did so with good nature. I appreciate that! I also liked working with most of the folks from College board and ETS [2], who were clearly committed to making this a successful reading [3]. I’ve been told that one of the primary benefits of reading APs is meeting a great group of people, and that was my experience. I’m not sure why the ETS folks think that people will be willing to grade the exam remotely [4].

When I started this series, I noted that I am generally suspicious of rubrics and that I was looking forward to seeing the rubrics that people with expertise develop and the process by which you get a large group of people on rubric. I remain skeptical of rubrics. As is my typical experience with rubrics, I found things that I considered high-quality work receiving low scores and things that I considered low-quality work receiving high scores [5]. That said, I am confident that students who do high-quality work could write things that will get high scores, provided that they pay attention to the rubric [6]. I also understand that this was the first year that the rubric and scoring notes were used at scale and that I expect [7] that they will get better.

While I did not agree with many interpretations of the rubric [8], I did find that we were able to get very much on rubric. I regularly re-scored the materials my readers were scoring [9], and found that our scores matched almost perfectly. I also chatted regularly with the other TLs in the room and we were also in agreement. I know that ETS does some broader comparison. We don’t receive the details, but I would expect that we were on target across the rubric [10].

People who talked to me throughout the reading know that early on in the reading, I was enough in conflict with the rankings I saw online and with the higher-ups at ETS and the College Board that I was saying something like

I support the goals of AP CSP, and I argued for Grinnell to accept credit for AP CSP. However, after looking at the examples online and hearing how we are scoring these submissions, I am not comfortable counting this as worthy of college credit.

Now that we’ve reached the end of the reading, I remain uncomfortable both with the examples posted online and with the way we scored the rubric [11]. Does that mean that Grinnell will no longer credit AP CSP? No.

Every high-school teacher I spoke with is teaching an AP CSP course that appears worthy of college credit. Hence, I am reserving judgement until the College Board updates the examples online and perhaps even until I see what happens at next year’s reading [12].

Am I done writing about my experience reading CSP? Almost certainly not. I look forward to explaining why I dislike some of the examples that were posted online at AP Central [14]. I may comment on the example that disappeared from the site [15]. I’m tempted to write templates for the two performance tasks [16], partially to help myself think through them more, partially to help any AP CSP students at GHS (or elsewhere). None of those should conflict with my contract with ETS, since they deal with public materials and do not rely on anything specific I learned while at the reading. Expect more fun over the coming weeks.

[1] From what I can tell, every room full of TLs formed a good team.

[2] I’m never sure who is who.

[3] I will say that I was less happy to find that there were higher-ups at College Board and ETS whose attitudes and opinions seem to be diametrically opposed to mine.

[4] That’s not quite true. Some folks cannot easily leave their families and would therefore prefer to read remotely.

[5] I did not keep track of the two-by-two grid of low/high quality and low/high ratings. Perhaps I should have.

[6] And provided that the rubric and the information that the College Board provides on how the rubric is scored are clear. This year, they were not.

[7] Or at least I hope.

[8] Sorry, I am not permitted to discuss my disagreements.

[9] Yes, a big part of my job was re-scoring exams.

[10] I am not permitted to post to social media comments on which of the two performance tasks I read, but my impression is that our PT group performed very consistently.

[11] Am I allowed to say that in public? I think so. I didn’t give any details of the interpretation of the rubric or why I’m uncomfortable with it.

[12] That assumes, of course, that they’ll let me come back and that it will be in a format that makes it worth coming back.

[14] Those seem hard to find. I expect they are replacing them. But I still think it’s worth commenting on what used to be there, since it served as a guidelines for many students and teachers.

[15] It’s been listed as unavailable for as long as I’ve been getting ready to read and reading. But I was interested to see why it was taken down and checked the wayback machine for what had been posted.

[16] I heard enough teachers talking about writing templates for their students that I feel comfortable writing and posting my own.

Version 1.1 of 2016-06-27.