Are You Testing Me? Part 1

This is part 1 of a 3-part series on assessment.

As a classroom teacher, I was always looking for ways to effectively assess my students’ learning. I came up with some great ways to differentiate through product, but sometimes, I just had to use a traditional assessment. I always thought I was pretty good at creating those assessments, but once I got my current job working for Assessment and Accountability, I realized I’d been doing lots of things that are not best practices when it comes to traditional assessments. I’ve decided to share some biggies with you in the hopes that your classroom assessments can be more valid, effective, and help you inform your instruction.

1. Make sure your questions are just that: questions. While it is acceptable to craft an item so that the answer completes a sentence (this is popular in college entrance exams and AP exams), classroom assessments tend to be more valid and effective (and you learn a bit more about your students’ comprehension) if your items are all worded as questions. Here are some examples.

5-21b

– “a” is the preferred item style.

– “b” is acceptable, but not ideal. Note: If you are going to use this format, be sure the “blank” students have to fill in is at the end of the stem (so, not “____ was the first President of the US.”).

– “c” is unacceptable; it isn’t a question, and students might be confused because they’re not entirely sure what you’re wanting them to do. Plus, it’s not even grammatically correct.

2. Use arbitrary order in your distractors (answer choices). We have human bias when we create answer choices. Statistically, we choose B or C as the correct answer more often than A or D (or E, if using 5 answer choices). This is a problem because if students figure out your bias, they can guess with higher rates of accuracy, which defeats the purpose of assessing what concepts or skills they actually know. If you use arbitrary order, this eliminates bias (or lessens it). Here are the common ways to arbitrarily order answer choices:

a. Alphabetically (when answer choices are one word)

b. By length (when answer choices are more than one word)

c. By the order in which the answers appear in the text (for example, line numbers in poetry or quotations from various paragraphs)

d. Chronologically

e. Smallest to largest (numerical values)

3. Number 2 depends on one thing, though: your answer choices should be roughly the same length. We like to make the right answers either the shortest or longest. It takes some practice and skill to make all your distractors the same length. Students pick up on this bias easily and will often guess an answer choice that is considerably longer or shorter than the others.

4. Punctuate your answer choices correctly. A sentence contains a subject and a verb. Sometimes the subject is what we call “understood.” For example, an imperative sentence (an order), “Clean up this mess!” is a sentence because the subject is understood to be “you” – as in, “You, clean up this mess!” It also has a verb: “clean.” There are also one-word sentences like “No.” and “Yes.” And “Stop.” If an answer choice has a subject and a verb, it should be punctuated as such: capitalize the first word and put an end mark (most likely a period) at the end. If it is NOT a sentence, do NOT capitalize the first word and do NOT put a period at the end. And in this vein, make sure all your answer choices (for a specific item) are the same, grammatically: either all are sentences or all are not. Additionally, make sure they are all consistent in terms of parts of speech, verb tense, point of view, etc. Here are some examples:

5-21a

– “a” is problematic because while “i” and “iv” are correctly punctuated sentences, “ii” is not a sentence, and should not be capitalized and ‘iii” is not a sentence and should not be capitalized or contain a period. Also, “iv” is much longer than the other 3 choices. Furthermore, all 4 answer choices are inconsistent.

– “b” and “c” are both equally acceptable. “b” uses complete sentences for all the answer choices, punctuates them correctly, and they are all roughly the same length. They also use the same verb tense (present). “c” uses a single word or phrase for all 4 answer choices; none of them are capitalized (nor should they be); none of them have end marks (nor should they). Additionally, all 4 are the same (and correct) part of speech (noun).

– “d” is problematic because although they are not sentences and are correctly (un)punctuated, they are not all the same (or appropriate) part of speech. “i”, “ii”, and “iv” are adjectives but “iii” is a noun.

5. Having 5 answer choices isn’t statistically different from having 4 answer choices, in terms of what you learn about your students’ knowledge. Anything less than 4 choices, however, is problematic because it doesn’t allow for differentiation among student mastery levels. Lesson: if you’re doing a multiple-choice assessment, go with 4 answer choices. 5 if you feel you absolutely must, but never less than 4. Side note: if you are going to follow the trend where there are multiple answer choices, you must follow this ratio: for 2 answer choices, you must have 5 or 6 options; for 3 answer choices, you must have at least 6 but not more than 8 options. You should not design a question where there are more than 3 correct answer choices.

6. Make your answer choices reasonable. You won’t learn anything about your students’ mastery if one or two of the answer choices are so outlandish that only a monkey would choose it. And don’t try to trick your students. Think about what you want to learn from their answers. Make your distractors (the incorrect answer choices) things that your students would choose if they had certain misconceptions. For example, if your question is (-2) + 3(8 x 3) / 4(5-7), then make your answer choices options that students could get if they dropped a negative, didn’t follow order of operations, etc. This way, you’ll actually gain useful information from students who get the wrong answer. You’ll know why they got it wrong and it can help you target re-teaching.

Stay tuned for part 2 of this series, which will be posted next week. I would love to hear from you! Please comment or leave a question about this blog entry using the form below!

Advertisements

Join the conversation!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s