Are You Testing Me? Part 2

As a classroom teacher, I was always looking for ways to effectively assess my students’ learning. I came up with some great ways to differentiate through product, but sometimes, I just had to use a traditional assessment. I always thought I was pretty good at creating those assessments, but once I got my current job working for Assessment and Accountability, I realized I’d been doing lots of things that are not best practices when it comes to traditional assessments. I’ve decided to share some biggies with you in the hopes that your classroom assessments can be more valid, effective, and help you inform your instruction.

  1. Only ask one question per item (for multiple-choice items; obviously, extended response items can ask multiple questions). If you need to break it apart into two items, then do that. For example, “Who was the first President of the United States and when was he elected?” is inappropriate because it asks two things in one item.
  2. Do NOT use “all of the above” as an answer choice. This decreases the discrimination value of the item because it essentially turns the question into a 3-choice question instead of a 4 or 5-choice question. You can use “none of the above,” but use it sparingly. Additionally, use questions with “not” and “except” sparingly. These tend to confuse students and you don’t really learn if they misunderstand the concept or your wording of the question. And when you do use “not” or “except” in a question, you should capitalize and bold it so it stands out to students. (Side note: you can capitalize and bold words like “most,” “least,” and “best” if you like.)
  3. Keep in mind that your students take roughly double or triple the amount of time to answer a question as you do. For example, if you create a quiz and take it yourself and it took you 10 minutes, it will take your students between 20-30 minutes. Obviously, if you have students with special needs (ESE, ESOL, etc.), this will increase the time even more. Have reasonable, developmentally appropriate expectations for your students.
  4. Be prepared to throw out poorly performing items. If more than half your students get a question wrong, chances are, there is an issue with the item. That being said, if more than half your items end up being missed by more than half your students, that’s a pretty stellar indication that a) they did not comprehend the material at mastery level and/or b) the questions had major problems in construction. Your options: throw the test out or curve it (and by “curve” I mean use a scale to norm the distribution of grades). Creating a bad test doesn’t make you a bad teacher, it just means you need more practice and guidance in test creation.
  5. Base your items on instructional objectives and/or standards. Don’t test trivial knowledge. Make sure you are assessing things that matter in terms of the scope and sequence of your course. The BEST option is to create the assessment BEFORE you teach the lesson/unit/concept. Begin with the end in mind. Your assessment is like a blueprint for your lesson. It’s the objectives for what you’re teaching. You want the students to master X, Y, Z, so you create an assessment to measure their mastery of X, Y, Z, and then you teach X, Y, Z. Finally, you assess the students with the assessment. It’s not “teaching to the test,” it’s called “backwards design” and it’s a best practice.
Advertisements

Are You Testing Me? Part 1

This is part 1 of a 3-part series on assessment.

As a classroom teacher, I was always looking for ways to effectively assess my students’ learning. I came up with some great ways to differentiate through product, but sometimes, I just had to use a traditional assessment. I always thought I was pretty good at creating those assessments, but once I got my current job working for Assessment and Accountability, I realized I’d been doing lots of things that are not best practices when it comes to traditional assessments. I’ve decided to share some biggies with you in the hopes that your classroom assessments can be more valid, effective, and help you inform your instruction.

1. Make sure your questions are just that: questions. While it is acceptable to craft an item so that the answer completes a sentence (this is popular in college entrance exams and AP exams), classroom assessments tend to be more valid and effective (and you learn a bit more about your students’ comprehension) if your items are all worded as questions. Here are some examples.

5-21b

– “a” is the preferred item style.

– “b” is acceptable, but not ideal. Note: If you are going to use this format, be sure the “blank” students have to fill in is at the end of the stem (so, not “____ was the first President of the US.”).

– “c” is unacceptable; it isn’t a question, and students might be confused because they’re not entirely sure what you’re wanting them to do. Plus, it’s not even grammatically correct.

2. Use arbitrary order in your distractors (answer choices). We have human bias when we create answer choices. Statistically, we choose B or C as the correct answer more often than A or D (or E, if using 5 answer choices). This is a problem because if students figure out your bias, they can guess with higher rates of accuracy, which defeats the purpose of assessing what concepts or skills they actually know. If you use arbitrary order, this eliminates bias (or lessens it). Here are the common ways to arbitrarily order answer choices:

a. Alphabetically (when answer choices are one word)

b. By length (when answer choices are more than one word)

c. By the order in which the answers appear in the text (for example, line numbers in poetry or quotations from various paragraphs)

d. Chronologically

e. Smallest to largest (numerical values)

3. Number 2 depends on one thing, though: your answer choices should be roughly the same length. We like to make the right answers either the shortest or longest. It takes some practice and skill to make all your distractors the same length. Students pick up on this bias easily and will often guess an answer choice that is considerably longer or shorter than the others.

4. Punctuate your answer choices correctly. A sentence contains a subject and a verb. Sometimes the subject is what we call “understood.” For example, an imperative sentence (an order), “Clean up this mess!” is a sentence because the subject is understood to be “you” – as in, “You, clean up this mess!” It also has a verb: “clean.” There are also one-word sentences like “No.” and “Yes.” And “Stop.” If an answer choice has a subject and a verb, it should be punctuated as such: capitalize the first word and put an end mark (most likely a period) at the end. If it is NOT a sentence, do NOT capitalize the first word and do NOT put a period at the end. And in this vein, make sure all your answer choices (for a specific item) are the same, grammatically: either all are sentences or all are not. Additionally, make sure they are all consistent in terms of parts of speech, verb tense, point of view, etc. Here are some examples:

5-21a

– “a” is problematic because while “i” and “iv” are correctly punctuated sentences, “ii” is not a sentence, and should not be capitalized and ‘iii” is not a sentence and should not be capitalized or contain a period. Also, “iv” is much longer than the other 3 choices. Furthermore, all 4 answer choices are inconsistent.

– “b” and “c” are both equally acceptable. “b” uses complete sentences for all the answer choices, punctuates them correctly, and they are all roughly the same length. They also use the same verb tense (present). “c” uses a single word or phrase for all 4 answer choices; none of them are capitalized (nor should they be); none of them have end marks (nor should they). Additionally, all 4 are the same (and correct) part of speech (noun).

– “d” is problematic because although they are not sentences and are correctly (un)punctuated, they are not all the same (or appropriate) part of speech. “i”, “ii”, and “iv” are adjectives but “iii” is a noun.

5. Having 5 answer choices isn’t statistically different from having 4 answer choices, in terms of what you learn about your students’ knowledge. Anything less than 4 choices, however, is problematic because it doesn’t allow for differentiation among student mastery levels. Lesson: if you’re doing a multiple-choice assessment, go with 4 answer choices. 5 if you feel you absolutely must, but never less than 4. Side note: if you are going to follow the trend where there are multiple answer choices, you must follow this ratio: for 2 answer choices, you must have 5 or 6 options; for 3 answer choices, you must have at least 6 but not more than 8 options. You should not design a question where there are more than 3 correct answer choices.

6. Make your answer choices reasonable. You won’t learn anything about your students’ mastery if one or two of the answer choices are so outlandish that only a monkey would choose it. And don’t try to trick your students. Think about what you want to learn from their answers. Make your distractors (the incorrect answer choices) things that your students would choose if they had certain misconceptions. For example, if your question is (-2) + 3(8 x 3) / 4(5-7), then make your answer choices options that students could get if they dropped a negative, didn’t follow order of operations, etc. This way, you’ll actually gain useful information from students who get the wrong answer. You’ll know why they got it wrong and it can help you target re-teaching.

Stay tuned for part 2 of this series, which will be posted next week. I would love to hear from you! Please comment or leave a question about this blog entry using the form below!

Be a Convert!

As a middle school math teacher, there never seemed to be an end to the things my students could not do when they came to me. For instance: I thought that a 7th grader would be able to efficiently subtract double and triple-digit numbers. Wrong. I thought that an 8th grader would be able to at least make an educated guess when taking a multiple choice quiz. Wrong.

Side note story:

My first and second years teaching 7th grade pre-Algebra followed a very strict (flexible when necessary, though) schedule where Monday and Tuesday were direct instruction, Wednesday and Thursday were student practice, and Friday was assessment (short, 10-question quiz, and then re-teaching when necessary). So I knew that not all my students would get 100’s on their quizzes, but I did expect them to at least try. Or use common sense. I swear to you, I got responses back like:

Solve: ½ + ¾

  1. 4/6
  2. 5/4
  3. 3/8
  4. 1

Student answer: “No”.

No joke. I think one time I even got the answer “blue.”

I had to find a better way. I had to give them what they needed. So I started making PowerPoint lessons. One that worked really well for them was on converting among fractions, decimals, and percents. In the 3 years I taught math, I found that my students needed, at the least, a refresher on this concept, and at the most, a full-blown lesson with direct instruction on it. My PowerPoint could serve in both capacities. I found that my PowerPoint lessons engaged my struggling students because it was visual and it was structured. It kept them focused, it kept them engaged, it taught note-taking skills, and it presented a document I could easily turn into a grade (through a notebook check or something similar). Yes, it took time on the front end to create, but it solved so many problems that it was well worth it.

Do you have students who need help with converting among fractions, decimals, and percents? Check out my PowerPoint lesson!

converting fractions decimals percents

Making Poetry Meaningful

I am a poetry snob. Perhaps one could call me uncultured, but I am very picky about the poetry I choose to read. To be perfectly honest, I really only enjoy two poets: Edgar Allen Poe, and myself. Well, in general. There are isolated poems from various poets that I enjoy. “The Charge of the Light Brigade” (Tennyson), “Because I Could Not Stop for Death” (Dickenson), and a few others. So suffice it to say that I empathized with my students when it came time for our poetry unit each year. Writing poetry comes easily to me, but that’s not the case for most students. In fact, in my years of teaching English, I found that the predominant reason for my students disliking poetry was that they struggled with writing it. Also, they’d had countless other teachers who had made them memorize and recite poems (I won’t editorialize on that practice…the fact that I NEVER made my students memorize and recite poetry in my class should speak for itself), which mostly served to turn them off of poetry completely.

I found that they also struggled with identifying different poetic devices within poems and analyzing poetry for any sort of meaning. Often, this was due to the fact that the poets they were forced to read were dry and confusing. What if my students were just like me? What if the poet they would enjoy reading the most were themselves? How could I get my students to write meaningful poems when they didn’t know how to write poetry?

First, I started small. I had to disabuse my students of the notion that all poetry had to rhyme or have some set rhythm and meter. That was difficult. Once I got that through their heads, though, I would show them examples of acrostic poems. I always had my students begin with an acrostic poem of their name. Eventually, the majority of my students were able to construct a reasonable attempt at an acrostic poem.

To help them understand rhyme and rhythm (and meter) I would use songs/song lyrics. This is also how I taught the poetic device: refrain. The jump from song to poem was easy with the “Battle Hymn of the Republic.” I found (in my biased opinion, of course) that the best poems for teaching meter/rhythm were Poe’s poems. My two favorites are “Annabel Lee” and “The Raven.” Incidentally, when my own middle school teacher required that we choose a poem to memorize and recite, I chose “The Raven” simply to be obnoxious. It was the longest one I thought I could actually memorize. I got about 3 stanzas in before she forced me to stop and sit down. Yeah, I was “that kid.”

Occasionally, I would give the traditional quiz on the various poetic devices, but I never gave an end-of-unit test. Instead, I wanted what they’d learned to be meaningful to them. From the onset of the unit, I let them know that they would be creating their very own poetry product: a calendar. I would usually have it be the full next calendar year, January-December, although, that might change based on the time of year during which the poetry unit was taught.

I picked 11 of the most important (to me, as the teacher; the most prolific, common, whatever you want to call them) poetic devices we studied during the unit. Then, I told students that they would be responsible for writing their own poems for each month: one poem per month; one device per poem. The 12th month/poem was a free choice for them. I encouraged them to relate the poem to the month, season, or event that occurred during the month. I strongly suggested they think about personal connections to months: birthdays, family trips/vacations, etc. Of course I gave them the major holidays (New Year’s Day, Valentine’s Day, St. Patrick’s Day, Easter, Mother’s Day, Memorial Day, Father’s Day, 4th of July, Labor Day, Halloween, Thanksgiving, and Christmas), but also suggested things like winter break, spring break, exams, summer break, back to school, etc.

I printed out the calendars for them and gave them free reign to decorate them according to their poems (if it focused on a birthday, then they would label that day on the calendar and illustrate it appropriately; if it was Valentine’s Day, same thing). They left ample space for their poems on each month.

I found that the vast majority of my students responded well to this authentic task. I had lots of students who would decide to give the calendar as a gift to a parent or relative (depending on the time of year you can suggest it as a Mother’s Day gift or a holiday gift for a parent/relative). Many of them expressed their excitement that the calendar would be hanging somewhere (on the fridge, in a parent/relative’s office, etc.), and they put in quite a bit of effort. It became meaningful to them. They suddenly were able to write poetry because they had a purpose and context. Over the years I got some very poignant, touching work (Veteran’s Day poems, poems to grandparents or others who had died, etc.). Not all of the poetry was going to win awards, but the students took the assignment seriously and did a reasonably good job. I was easily able to tell who had mastered the various poetic devices and other literary concepts.

The best part about this assignment was that it could be tailored to any level. I started with it in 7th grade and used it all the way up through my high school students. I had poems as simple as limericks and haikus all the way up to full-blown Shakespearean sonnets. And not only did the kids (many of them, anyway) actually enjoy writing the poems, but they (dare I say “all”?) enjoyed coloring and illustrating the months in the calendar. And I was able to photocopy the best ones to keep and display in my room that year and use as examples in subsequent years.

Creating the calendar (in its entirety – writing the poems, illustrating the pages, etc.) usually took about 2 weeks. Sometimes less, depending on the students, but never more. With my honors kids, I would often assign a certain amount as homework so it didn’t stretch too far into class time. I found that the kids needed my help, though, in many cases, so I shied away from assigning the whole (or majority of) thing as homework.

If this sounds like something you’d like to do in your classroom, you are in luck. I’ve already created the templates and rubrics for the project. All you have to do is decide which poetic devices you want to assess.

Click here for the Poetry Calendar Project.