Reading the AP CS Principles exam

Topics/tags: Miscellaneous, things I was writing anyway

A few years ago, a new AP exam for computer science was created [1,2]. The AP CS Principles (CSP) exam incorporates two important approaches: It is supposed to be a broad-based introduction to CS, not one that just emphasizes programming and it does its best to be language neutral. How do you make an exam that is language neutral? You ask students to provide programming artifacts with an accompanying narrative and then you get a bunch of CS teachers [3] to spend a week grading [4] them, using a rubric.

I volunteered to do CSP reading the first full year it was offered [5]. For reasons known only to the Chief Reader, I ended up serving as a Table Leader. That meant that I showed up early, learned the rubric well, and then guided a group of eight people in their scoring. That year, I worked on the explore task, in which students were to create a computational artifact in which they discussed the effect of some technology on society.

I had planned to do the CSP reading the next summer, but a family event conflicted, and family takes precedence [6].

In the next summer, I was able to return. This time I got to be a table leader for the create task, in which students are expected to write a program in a language of their choice and then write about it. The rubric asks the scorers to look at issues like the use of multiple algorithmic constructs and the students’ understanding of key concepts. I will admit that I found it a frustrating experience; perhaps I’ll explain why at another time. One thing that made it less fun was that most of the graders were remote, rather than in-person. That makes it easier for some teachers, such as those with family obligations or whose schools are still in session.

In spite of these frustrations, I still hope to be able to go back next summer [7]. Why? That’s the real topic of this musing, or at least related to the real topic of this musing.

As I noted earlier, the vast majority of the scorers are high-school teachers. It sounds like ETS and the College Board would like to get more college and university faculty to score the exams. Why? Part of the issue is likely that AP courses are supposed to be college-level courses, and having a cadre of college-level faculty scoring would help make sure that the exam [8] is of the appropriate level. But it’s also a marketing thing. The more higher-ed folks who know about the CSP exam, the more likely it is that departments will award credit for that exam [9].

Back when I was scoring [10], someone from ETS asked us how they might recruit more faculty. I cleverly volunteered to draft a message to the SIGCSE listserv in which I explained why I find it worth the time to grade AP CS Principles. This musing serves as my draft. It also explains why I’m tempted to go back.

Dear Colleagues,

I’ve heard from ETS and the College Board that they’d like to see more college and university faculty participate in the scoring of the AP CS Principles exam. Hence, I am writing to encourage you to consider signing up as one of the scorers for that exam. I’m not one of the core folks on CS Principles; I even have some objections to the exam and the scoring. I also realize that the idea of spending a week grading sounds horrific. Nonetheless, I’ve found the experience of scoring the exam useful, and I hope that you’ll consider spending the week of June 11-17 in Kansas City scoring the exam.

Here are some reasons that I find the experience worthwhile.

• You get a much deeper understanding of the exam and the way it is scored. Since an increasing number of students are taking AP CSP, that knowledge will help you as you place students in classes and as you consider whether or not to give credit for the exam.

• You get to meet a large cadre of thoughtful, creative, dedicated, and fun high-school CS teachers. I’ve found my perspective on teaching CS expands as I talk to these colleagues.

• You also get to work with some wonderful higher-ed folks who are involved in the exam. As I understand it, you’ll have the opportunity to work with both Paul Tymann (outgoing Chief Reader) and Tom Cortina (incoming Chief Reader).

• You can potentially have an impact on the exam. I’ve found that the CSP team listens seriously to concerns. Last year, the College Board team invited the higher-ed folks for a more detailed discussion of the exam.

• You get deeper experience with rubric-based grading and with techniques to ensure consistent grading across a host of people with different backgrounds.

• Once in a while, you get to see really remarkable student work. Unfortunately, you can’t then use that knowledge to recruit those students. (You won’t even know their names.)

• You get to explore Kansas City. Each day ends at 5 pm, giving you time to go out and do things. The last time I visited, I was able to see live jazz and spend a significant amount of time visiting two excellent art museums. I also had fun socializing with the folks mentioned above.

• You not only get housed and fed, you also get paid!

I believe it is also possible to do remote scoring of the CS Principles Exam. You lose most of the social aspects, but you do get a deeper understanding of the exam (and you get paid).

More information and an application form are available at https://apcentral.collegeboard.org/professional-development/become-an-ap-reader. I’m sure that Tom and Paul would also be happy to answer questions.

Would that convince you? Would that convince me? The jazz club and the art museums are certainly hooks. And I really do value the experience of having worked with a wide variety of awesome high-school teachers.

In any case, the draft is going off to some folks for commentary. After that, it will likely go out to the SIGCSE-members list. I hope it has some effect.

Postscript: I’ve also been really impressed by how much the high-school teachers care about diversity issues. I should consider ways to incorporate that in the note.

[1] Passive voice intentional.

[2] I am not going to give a history of that exam, nor will I give a discussion of the associated politics of the exam.

[3] Primarily high-school AP teachers, but also a few college and university faculty.

[4] I’m fairly sure that the official term is scoring, but the practice is not substantially different from grading.

[5] There was a prior, smaller-scale, experimental year.

[6] I’m getting better all the time.

[7] A number of factors will likely come into play. I caused a bit of commotion in my pushback on some of the scoring samples, so I may not be welcome back. I also have family obligations and my research students to worry about.

[8] Or at least the part that gets scored by human beings.

[9] At least that’s the hope.

[10] Table leading.

Version 1.0 of 2019-10-09.