Am I tempted to include one question on my test to send the message “HEY! We spent a day on this in class and we had a homework assignment on it, so you better do it because I said it would be on the test!” Yah, I’d probably do this. But the real message it sends is “I use my tests to reinforce that you should be doing my homework for arbitrary reasons and to punish you when you don’t”What the author realized can be summed up in the title of this post: questions are expensive!
In my work on a test development committee, one of the first things I learned was that a standardized test ("item") that appears on the ACT and SAT represents a substantial investment of cash and time. It costs upwards of $1,000 a question (in fact, by some estimates, more than $10,000) to develop, vet, and pre-test a single item of the hundred-plus items that appear on a typical SAT. In the context of test creation and administration, this fact makes test editors somewhat conservative: deciding to change (and re-vet) or throw out a test item in the late stages is actually a major financial commitment. But what I'm suggesting here is that, as teachers, we should all be somewhat conservative about what we put on tests, because every test (or quiz, or homework, or project) item is expensive.
"Expensive how?" you ask. The costs abound. It takes time to write, proofread, and format the test. It takes time for your students to do the test, time that could be spent in doing other questions or assignments or just (imagine!) having fun. It takes time to check and grade the test, and then you have to figure out what to do with the scores and information about student performance. Most important, every item you include on a test represents a decision not to include something else: you can't give a class of fifth-graders a six-hour math exam. Unless you teach the most boring class ever, the chances are that in the course of a single unit, you've had your students work on many different skills in literally dozens of tasks and contexts. You can't rehash all of that on a test, so every item that gets on the test has pushed four or five or six more off.
So you need to be a little conservative. By that I don't mean that you can only assign items that you've already reviewed in class, or that you should never change a test--quite the opposite. You do need to choose your items carefully, thinking about them more as an incredibly expensive data sample -- or a trip to a very expensive gym or tourist destination -- rather than as simply a collection of objects that more-or-less mimics some of the things you've done in class. You need to ask:
- What will I learn from doing this about what my students know? What skills and concepts does this item assess? How are my students likely to respond to the item, and what will I learn about my students from those responses?
- What will my students learn from doing this? Is this an opportunity for them to grow and stretch in some interesting ways, or just a check that they can spit out what we've put in? Will students come away from the experience with a better sense of what they themselves know and can do? And will they come away with a better sense of what it is I'm trying to teach them?
My friend and mentor Diane Herrmann speaks sarcastically about the "sponge theory" of teaching: you start the term with a dry sponge and spend the term pouring water into it. At the end of the semester, you squeeze out the sponge into a measuring cup (graduated cylinder, whatever): the student's grade is the percentage of the poured-in water that you can successfully squeeze out. I think that theory drives a lot of the garbage-y tests kids wind up taking--tests with 50 or 75 or 100 items to be done in 45 minutes, tests that ask similar questions again and again. By contrast, if you think about teaching as developing a kind of mental fitness--with certain types of habits, strengths, and skills--then you realize that a test is not just a way to find out what a kid can do, it's a chance for you to provide the kid another growth experience. And that's the real value added.
