|
|
It is Not the Spanish Inquisition: Designing and Using Effective Multiple Choice QuestionsAne Lintvedt with Laurie Mannino
|
|
It is safe to say that few go into the teaching profession because they like administering tests; and yet assessments are most often how our students' successes—and our own—are judged. Very few teachers come into the profession armed with an understanding of the thinking that should underpin assessment. The lead author of this essay examines the means of creating effective multiple-choice assessments and draws attention to a student-based approach to the subject developed by Laurie Mannino. We both argue that multiple-choice questions can and should go beyond factual recall and encompass assessment of historical thinking. Although most text books come with test banks, the authors of the text book rarely have much to do with the writing of that test bank or any of the other ancillaries. Therefore a very good text book may well be accompanied by a startlingly poor set of ancillaries. Experienced teachers know that test bank questions are often of poor quality, frequently riddled with mistakes, and assess only basic information at the level of recall alone. It is worth learning how to write your own, high-quality multiple-choice assessments so you are not at the mercy of the test banks and so you are assessing your students on a variety of knowledge and skills. During two decades of teaching on-level and AP classes in high school, I became increasingly disillusioned with test bank ancillaries and have come to the conclusion that there is no free lunch. Copying someone else's examination questions—a seemingly easy task—leads to appalling bad assessments and more time spent doing triage, which is harder to do than to have taken the time to write a better test in the first place. I finally learned how to write better multiple choice questions when I worked with the test development professionals at Educational Testing Services (ETS). They taught me the logic as well as the methods of writing superior multiple-choice questions as I worked with them on SAT subject tests, AP Tests, and Praxis test questions. Disclaimer: What I have written below is my distillation and many experiences of writing multiple choice questions, and not anything that ETS has published or endorsed. I am going to start with some basic rules and their rationales, and then discuss the more complex issues of assessing different types of historical thinking skills in theory and in my own practice and also through a student-driven assessment model developed by Laurie Mannio. The Basic Rules
Rationales of the Basic Rules The most important thing to remember about a multiple-choice assessment is that you should be testing content and historical reasoning skills. You should not be designing a test to trick kids or make them waste their time riddling through a maze. That kind of student torture will not help prepare them for any SAT Reasoning or Subject Test, the AP exams, or state standards' exams.
Assessing Historical Thinking Skills The most basic type of multiple-choice question is an information-recall or a content question. This type of question would ask students to recall specific data: events, locations, names, geography, etc. Factual recall certainly favors students who can memorize discrete pieces of information, but it rarely assesses anything more than that. I have listed below a series of historical thinking skills to which we should expose our students and which we all hope our students master. For those who teach Advanced Placement classes, you should know that all the AP history exams are going to start deliberately assessing historical thinking skills in multiple-choice questions within the next few years.
There are more, and also more complex, historical thinking skills that can be assessed with multiple-choice questions. The ones I have listed above will, I hope, get you started along a path of writing thoughtful and varied questions that will truly assess what your students know and what skills they have learned. Once I learned how varied and how well a good multiple-choice question can assess information and historical thinking skills, I stopped using test banks and cringed at the questions I wrote 5 years ago. I don't deny these sorts of questions take a lot of time to think through and compose. My family members and colleagues, sadly, are used to my whining about having to write yet another set of multiple-choice questions. Two things should always be kept in mind when spending that valuable time. First, students hate may hate your well-designed multiple-choice questions, but will be delighted when they perform exceedingly well on AP exams. Second, never, ever hand back your multiple-choice questions to students. Assessing Historical Thinking Skills Teaching students to write multiple-choice assessments can have multiple benefits. Laurie Mannino employs an assignment that directs the student to isolate the most significant information of the unit and distill it into an assessable question. It addresses the learning of historical content; deeper understandings of the significance of themes and historical thinking skills; familiarity with AP-style multiple-choice questions, and last but not least a certain sympathy and appreciation for the work teachers do to create multiple-choice assessments. As all experienced teachers know that to really understand something you should try to teach it to someone else. With the exercises below, Laurie guides her students through both the learning and teaching of history with an assignment of creating multiple-choice questions from their World History reading. Laurie requires that the questions must be based on the AP Themes that are outlined in the College Board AP 2008-9 World History Course Description Book (or Acorn Book) online at http://apcentral.collegeboard.com/apc/public/repository/ap07_worldhist_coursedesc.pdf. This source ensures that the questions will not be merely information-recall and reinforces the importance of the themes of the course. Students must then follow the Basic Rules (outlined above); e.g., 5 answer choices, only one correct answer, etc. A crucial part of the assignment is that in addition to the questions, there must be an accompanying paper explaining why each correct answer choice is correct and why each incorrect choice (or distracter) is incorrect. This exercise will reinforce the information as well as refine the students' ability to relate specific information to more general themes. For AP classes in particular, but for any level World History survey, this assignment could be used either as a straight-forward assessment in the middle of a unit, it could be used as a cumulative assignment at the end of an era. Teachers could then compile a set of student-composed questions and create another assignment that requires students to critique and correct questions, or have students take their peers' assessment, or use student-created questions on unit assessments in one way or another. There are a lot of possibilities here. And certainly, the questions could be compiled and used as review for an end-of-course exam, as Laurie clearly does. An example of the assignment that Laurie would hand out to her students is offered below: Your assignment is to create five AP level multiple-choice questions (MCQ's) using the material found in the required chapter. This assignment will help enormously with the MCQ portion of the AP exam; it will also help with the essay portion in terms of content; and it will help you understand the material in the text. The questions and answers must reflect the World History AP (WHAP) themes as listed below: there must be one question for each theme. WHAP themes:
Please refer to the handouts [the Basic Rules] on how to construct an MCQ and the example sheet in order to better understand the level of question that you need to develop. Any questions? Please ask BEFORE the assignment is due! :-) HAVE FUN! Important details:
Assessing Historical Thinking Skills Whether you use multiple-choice questions sparingly or frequently, it is important to write ones that truly assess students' content knowledge as well as their historical thinking skills. And if you can teach your student to write useful and thoughtful multiple-choice questions, the benefits will be plentiful. Ane Lintvedt teaches history at the McDonogh School in Owings Mills Maryland. She has scored the AP European and World history exams, is a consultant for College Board and is currently a member of the executive council of the World History Association. She can be reached at alintvedt@mcdonogh.org. Laurie Mannino teaches history and religion at Col. Zadok Magruder High School in Rockville, Maryland. She also works as a consultant for the College Board, scores the AP World History exam and has written test questions for national teacher certification programs. She can be reached at Laurie_S_Mannino@mcpsmd.org. |
|
|
|
Home | List Journal Issues | Table of Contents |
© 2009 by the Board of Trustees of the University of Illinois |
|