Creating Effective Multiple Choice Exams

By Meg Gorzycki, Ed.D.

Instructors at all levels use multiple choice exams to test students' knowledge. Multiple choice is the format universally used in standardized testing that often determines whether an applicant will be admitted into the college of one's choice, and whether an elementary or secondary school merits state interventions. As such, multiple choice tests have generated considerable attention. They've also encouraged an industry of education-related entrepreneurs to claim they can teach students how to--beat the test,-- rather than concentrate on how to encourage student learning and make the exams reflect an accurate assessment of what the student has learned.

Multiple choice exams are frequently administered to save time. Other forms of assessment--short answer, essay, etc.--usually take the instructor a longer period of time to grade. Machines can scan a multiple choice answer sheet in seconds while it take hours to read and provide commentary on an exam comprised of several pages of student writing. While multiple choice tests may expedite grading, they also tend to target recall of information: facts, a particular interpretation or perspective, an established insight about cause-effect or significance, or a rehearsed understanding of processes or relationships between items or ideas.

The purpose of this module is to raise instructors' awareness of the strengths and limitations of multiple choice exams, help instructors distinguish between high and low quality multiple tests, and to improve the way instructors assess student learning.

Historically, dialogue represents a universal means of assessing what individuals know and how well they are able to reason.  Ancient teachers, such as Confucius and Socrates, engaged students in conversation that provoked articulate logical, well-reasoned, and insightful responses to perplexing and complex questions. Formal written exams were first administered in China around 210 B.C.E., to hundreds of hopeful candidates competing for civil service jobs. Testing has evolved in response to many variables. The number of students enrolled in schools and universities has steadily increased since the Age of Enlightenment, the scale of test-takers has increased, psychometric methods of measurement have been developed, and technology has provided the means to process data relatively quickly and efficiently; these factors combined to create a demand for instrumentation that could quickly assess what students know (Madus & Dwyer, 1999).

In the Medieval world, in which the modern university took form, those who took any type of an “exam” in order to secure a particular position were likely to be apprentices who were required to prove to guilds and their masters that they had achieved high degree of craftsmanship. The small percentage who studied liberal arts at the university were likely to be given oral exams in which they recited their understanding of complex theological issues relevant knowledge and orthodoxy (Hoskin, 1979). As the production and availability of paper increased during the Renaissance, and as instructors required written proofs of mathematical mastery, the written exam gained popular use.

The need to shift from a qualitative evaluation of student’s knowledge and skill was punctuated by the fact that those who judged the oral exam were often partial and subjective. In 1792, William Farish introduced quantitative evaluation of student work, which allowed for an objective assessment of student knowledge (Hogan, 2007). This development made possible innovations in psychometrics and facilitated the trend towards testing specialized information rather than general education (Hoskin, 1979). Farish’s contribution seemed tailor-made for a society that, like the United States at the close of the 19th century, increasingly orientated its economy, politics and culture to industrialization. The future, it seemed, belong to those who could generate and mange new technologies and knowledge faster than the competition.  Objective testing focused attention on specific facts in evidence rather than on philosophical or theological speculations.

The modern testing movement began with Herbert Spencer (1855)  and Sir Francis Galton, (1869) who, like other naturalists of the Victorian era, wanted to account for the uneven distribution of intelligence in the human population. Spencer believed that meaningful data about human intelligence could only be generated by measurements using widely-accepted criteria. In 1879, the first laboratory to scientifically undertake the study of human behavior opened at the University of Leipzig. Under the leadership of Wilhelm Wundt, the laboratory tested intelligence by studying the subject’s perception, memory, reflex, and sensory acuity (Tichener, 2921). It was not until Alfred Binet introduced his intelligence test in 1905 that scientists reached a consensus: the way subjects responded to readings and images was actually proxy for the reasoning process itself (Zazzo, 1993). With a reliable instrument in hand, educators could potentially test hundreds of students simultaneously and quickly identify achievement norms for each level of instruction; they could also determine who was above or below that norm and sort students according to performance.

Educational psychologist Edward Thorndike developed prototypes of multiple choice tests early in the 20th century (Goodenough, 1950). In 1914, Frederick Kelly, Dean of the college of Education at the University of Kansas,  introduced the multiple choice test, which was then adapted by Lewis Terman, who developed the Stanford-Binet Intelligence Test, the SAT, and a host of other exams claiming to measure knowledge, critical thinking, and reading comprehension (Lemann, 2000).  With the invention of high speed scanners and computers, multiple choice tests quickly became a favored method of student assessment, as it required little reading and content analysis and could be administered to large classes with ease.  At present, the multiple choice exam is perceived by many educators as a prophylactic against what Horace Mann once called the “officious interference of the teacher,” and what others recognize as subjectivity and favoritism (Marbury, 2004)

References

Galton, F. (1869). Hereditary Genius. London: Macmillan and Company.

Goodenough, F. L. (1950). Edward Lee Thorndike, 1874-1949. The American Journal of Psychology, 63, 291-301

Hogan, R. L.  (2007) The historical development of program evaluation: Exploring the past and present. Online Journal of Workforce Education and Development, 2, 1-14.

Hoskin, K. (1979). The examination, disciplinary power, and rational schooling. History of Education, 8, 135-146

Lemann, N. (2000). The big test. New York, NY: Farrar, Strauss, and Giroux

Madaus, G. and Dwyer, L.M. (1999). A short history of performance assessment. Phi Delta Kappan, 688-695.

Mabry, L. (2004). Strange, yet familiar: Assessment-driven education. In Holding accountability accountable: What ought to matter in public education, pp. 116-134. Kenneth A. Sirotnik (Ed.). New York, NY: Teachers College Press

Titchener, E. B. (1921). Wilhelm Wundt. The American Journal of Psychology, 32, 161-178.

Spencer, H. (1855). Principles of Psychology. London: Longman, Brown, Green, and Longmans

Zazzo, R. (1993). Alfred Binet. The Quarterly Review of Comparative Education, 23, 101-112.

What Does the Research Tell Us?

Multiple choice exams have repeatedly drawn criticism for their inability to measure higher level thinking skills (Haladyna, et. al. 2002).  In addition, assessing complex abilities such as reasoning, writing, and mathematical problem solving can be difficult and challenging (Crook, 1988; Stiggins, et. al., 1989).  As a result, higher level thinking skills are not always targeted in student assessments nor proficiently evaluated when targeted. In preparing for multiple choice exams, students often spend less time studying (Kulhavey, et. al. 1975) and tend to take notes on different material than what they focus on when expecting an essay exam (Richards, & Friedman, 1978). Students are more apt to engage in deep learning when preparing for assignment essays that clearly call for complex cognitive activity, and tend not to engage in deep learning when they are preparing for multiple choice exams (Scouller, 1998; Biggs, 1979).  In short, students are highly responsive to expectations, and will frequently perform only at the level of cognition required to pass the course (Fransson, 1977; Marton & Saljo).

Furthermore, when students take multiple choice reading comprehension tests, they routinely try to answer the questions as quickly as possible, do not initially read the essay critically, and return to the reading after reading the questions in order to locate the correct answer; the questions then, determine the focus of the reading (Farr, et. al., 1990).

As repetition rather than reason sometimes influences what individuals believe is true, some researchers have observed that exposing students to batteries of incorrect answers sometimes influences them to hold false assertions as truth (Bacon, 1979; Begg, et. al, 1985; Hasher, et. al., 1977). Even when test-takers are warned against guessing, many subjects select incorrect answers because they sounded familiar (Roedigier & Marsh, 2005).

Research has also identified certain guidelines for writing multiple choices questions and answers in ways that are likely to measure targeted learning outcomes. In one meta-analysis of textbooks and guidelines addressing the composition of multiple choice tests, over 75% of sources (Haladyna, et. al. 2002). agree that multiple choices tests should:

  1. Target important, not trivial, information
  2. Use novel material
  3. Contain clear directions
  4. Embed a central idea in the stem (statement or question to be resolved)
  5. Make sure options in the answer bank are of equal length
  6. Avoid disclosing clues
  7. Make distractor answers plausible

References

Bacon, F. T. (1979). Credibility of repeated statements: memory for trivia. Journal of Experimental Psychology: Human Learning & Memory, 5, 241-252.

Begg, I., Armour, V., Kerr, T. (1985). On believing what we remember. Canadian Journal of Behavioral Science, 17, 199-214.

Biggs, J. B. (1979). Individual differences in study processes and the quality of learning outcomes. Higher Education, 8, 281-394.

Crook, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58, 438-481.

Fransson, A. (1977). On qualitative differences in learning. IV-effects of intrinsic motivation and intrinsic test anxiety on process and outcome. British journal of Educational Psychology, 47, 244-257.

Farr, R., Pritchard, R. and smitten, B. (1990). A description of what happens when an examinee takes a multiple choice reading comprehension test. Journal of Educational Measurement, 27, 209-226.

Haladyna, T M.., Downing, S. M. Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines and classroom assessment. Applied Measurement in Education, 15, 309-334.

Hasher, L., Goldstein, D., & Toppino, T. (1977). Frequency and the conference of referential validity. Journal of Verbal Learning and Verbal Behavior, 16, 107-112.

Kulhavey, R., Dwyer, R., and Silver, L. 91975). The effects of note taking and test expectancy on the learning of text material. Journal of Educational Research, 68, 363-365.

Marton, F. and Saljo, R. (1976). On qualitative differences in learning. II-outcomes as a function of the learning conception of the task. British Journal of Educational Psychology, 46, 115-127.

Richards, J. P., and Friedman, F. (1978). The encoding versus the external storage on hypothesis in note taking. Contemporary Educational Psychology, 3, 136-143.

Roediger, H. L. III and marsh, E. J. (2005). Positive and negative consequences of multiple choice testing. Journal of Experimental Psychology, 31, 1155-1159.

Scouller, K. (1998). The influence of assessment methods on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35, 453-472

Stiggins, R. J., Griswold, M. M., and Wikelund, K. R. (1989). Measuring thinking skills through classroom assessment. Journal of Educational Measurement, 26, 233-246.

Multiple Choice Designs

Conventional multiple choice test consists of one stem and 3-5 optional answers

  1. Which of the following themes was not distinct in the work of Charles Dickens?
    • The indifference of the affluent to the suffering of the poor
    • The crippling effects of criminalizing debt
    • The ruin brought about by self-deceit and ignorance
    • The folly of colonialism and its attendant wars

Alternate-Choice

  1. Which of the following would most likely have been asserted by a Victorian realist?
    • All that one needs to know is tucked away in a cloud somewhere between the heart and heaven
    • The clanking of the mill, the cranking of cogs—the lessons learned young by hands forge the mind

Multiple True-False

  1. You are a novelist of the Lost Generation. You belong to a unique age of writers in western society. Which of the following would describe the protagonists in your work?
    • People of religious faith who find redemption in suffering
    • People confronted with others who steep themselves in materialism
    • Individuals who witness others being devoured by vanity
    • Men and women who are made impotent and indifferent by war
    • Men who are genuinely honored and praised for political zealotry
    • Men and women who readily embrace moral relativism

Matching

  1. Match each term with the description on the right:
    • Hyperbole           A. Figure of speech, non-literal in meaning
    • Allegory               B. Incongruity between intentions and actions  or statements
    • Trope                    C. Exaggerated speech or inflated rhetoric
    • Irony                     D. Communicates moral teachings through symbolic characters

Complex Multiple Choice

  1. Which of the following are medieval secular literary works?
  1. Stabat Mater
  2. Song of Roland
  3. Beowulf
  4. Summa Theologica
  5. Nibelungelied
  1. 1, 2 and 3
  2. 2, 3, and 4
  3. 2, 4 and 5
  4. 1, 3 and 4
  5. 2, 3, and 5

Using Graphics

Use the following graph to answer questions 1-3.

Weekly Income in Hundreds by Gender, Race and Education in the United States

The graph clearly implies that:

    1. As people acquire more education, they automatically earn a decent wage
    2. Those with less than a high school education cannot read well
    3. The average income for college graduates is higher than for those with only some college education
    4. Most men with some college education earn about $1,000 per week
  1. According to this data:
    1. White men with no high school diploma earn twice as much as white women with a high school diploma
    2. Black women with some college education earn  one half of what black women with a bachlor's degree earns
    3. Hispanic women with advanced degrees earn very nearly what black men with a bachelor's degree earns
    4. White men and women with bachelor's degrees earn more than all other groups
  2.  Which statement cannot be verified by the graph?
    1. That in general, college educated men earn more than college educated women
    2. That all races improve their incomes by completing college
    3. That it is not possible to earn over $1,000 per week witout a college degree
    4. That people who earn more money than others have better reading and math skills

Discussion

Using graphs, charts, tables and images challenges students to correctly interpret graphic organizers and to refine their ability to assess the accuracy of assertions that are grounded in data. Using theses graphics in formative assessments of students' understanding can generate discussions about how easy it is to project conclusions into representations of data. for example, in question number three in the above question bank, readers may find it logical that people who have higher incomes also have better reading and math skills; however, the graph provides no data concerning this variable, and so cannot be used to support the assertion.

Pre-Construction Questions

In order for the test to measure targeted student learning effectively, instructors must address the following questions:

  1. What is the purpose of the test? Will this experience be formative or summative in nature? A formative test would not generate grads for the record, but establish the level of student understanding at a given point and allow for the instructor to confidently move forward with new information, or review points students did not fully understand.
  2. What kind of learning is targeted in this test? Is the test supposed to measure student recall of facts and discrete knowledge addressed in course work or should the test target more robust thinking skills, and if so, is the multiple choice modality appropriate?
  3. What is the current level of student understanding and how might the test help students reach the next level?
  4. Would student learning be more effectively assessed and evaluated if the multiple choice test were offered in tandem with short answers, references to graphics, maps, or reading comprehension exercises?
  5. How will the results of the test be used to further and enhance student learning?

Sample Exams: History

Directions

Read the following two multiple choice exams and compare and contrast the questions and answers. Which exam targeted higher level thinking?  Which one made it easier to get a correct response by guessing? Both exams test students’ knowledge and understanding of Cold War origins.

Form A

  1. Which statement is false regarding the Cold War?
    1. It was a conflict that generated no armed conflicts
    2. It is often characterized as a clash of ideologies
    3. It lasted from about 1945 to 1991
    4. It has been credited with fueling the arms and pace race
  2. This event indicated to anti-communists in the west that the USSR was intent on expanding their control over Poland once World War II was over:
    1. The signing of the Atlantic Charter
    2. The occupation of Baltic states
    3. The Katyn Forest Massacre
    4. The Battle of Stalingrad
  3. To help secure democracy in Italy after World War II, the United States:
    1. Negotiated with Stalin to create an Italian constitution
    2. Sent ambassadors to Rome to pray with the Pope
    3. Had the CIA spread propaganda and buy votes
    4. Appointed an Italian-American as Secretary of State
  4. Why was Germany the crucible of the Cold War between 1945 and 1949?
    1. It was located in central Europe which was a good location for trade
    2. It showed strong signs of wanting a return to Nazism after the war
    3. The US wanted it to resume its place in the global economy and the USSR wanted it to be crippled so it would never again attack eastern Europe
    4. It was highly scientifically advanced
  5. Which country’s political fate after 1945 seemed to indicate the collapse of Yalta agreements?
    1. Turkey
    2. Poland
    3. Iran
    4. Czechoslovakia
  6. This term refers to restricting the expansion of communism by way of force and diplomacy
    1. Proliferation
    2. Appeasement
    3. Revisionism
    4. Containment
  7. Which document is mismatched with its author?
    1. Long Telegram/ George Kennan
    2. Two Camp Speech/ Joseph Stalin
    3. Iron Curtain Speech/Vyacheslav Molotov
    4. Speech on the Truman Doctrine/Henry Wallace
  8. The Soviet response to the Marshall Plan was to:
    1. Revive Eastern European nations with the Molotov Pan
    2. Invade North Korea with China’s support
    3. Increase its proliferation of atomic bombs
    4. Cancel plans for popular elections in Poland
  9. This Czechoslovakian leader was able to secure his role as Prime Minister because he capitulated to Moscow’s directives and repression of dissent
    1. Edvard Benes
    2. Jan Masaryk
    3. Andre Zhdanov
    4. Klement Gottwald
  10. Which of these was not a member of the Eastern Bloc?
    1. East Germany
    2. Bulgaria
    3. Hungary
    4. Austria

Form B

  1. Which statement most objectively describes the cause of the Cold War?
    1. The Cold War was a natural result of the competition between capitalism and communism that required little to no  initiatives to perpetuate
    2. The Cold War was essentially caused by the inability of the United States’  military forces to secure Eastern Europe following the defeat of Nazi Germany
    3. The Cold War evolved as the USSR and western allies deployed their own strategies and policies to rebuild the post-war world according to their own needs and values
    4. The Cold War was caused by the failure of Europeans to abandon communism which allowed the USSR to create a world-wide communist threat
  2. What was the significance of the Katyn Forest Massacre from the western Allies perspective?
    1. It paved the way for the Soviets to secure puppet governments in the Balkan peninsula
    2. It suggested that the Soviets wanted to crush Polish military potential to oppose Soviet incursion
    3. It provided evidence that the Nazis had committed genocide in the Soviet Union
    4. It proved that The Soviets had the technology to take Berlin before Britain and the US could
  3. What was the significance of the Italian election of 1948?
    1. It demonstrated the power of the CIA on one hand while on the other demonstrated Italian communist’s willingness to oppose Moscow on the matter of the Marshall Plan
    2. It underscored the fact that the Catholic Church was losing its ability to influence politics and that the communists were improving their ability to influence voters
    3. It revealed the overwhelming mistrust of democracy among the working class while revealing the extreme confidence in democracy held by the upper class
    4. All of the above are true
    5. Only b and c are true
  4. The Berlin Airlift was the result of ______, which was in turn the consequence of ______
    1. The Soviet blockade of West Berlin/the Allied reunification of West Germany
    2. Threats to Polish officials/Soviet sabotage of democratic elections
    3. The Soviet occupation of East Germany/Soviet army advances during World War II
    4. American interest in German science/the US desire to create a super bomb
  5. US reaction to the establishment of a Soviet-friendly government in  Poland in 1946 implied:
    1. The US was prepared to go to war to protect democracy in Eastern Europe
    2. The US did not see the difference between a puppet regime and independent communist leadership
    3. The US had no grand plan to orchestrate and secure democratic governments in Eastern Europe
    4. The US blamed the Poles for their fate because they had failed to repel the Soviets on their own
  1. Containment was predicated on the idea that ___ and pursued by way of ____
    1. Atomic science must not be shared/Legislation and security measures
    2. Western European stability came before that of Eastern Europe/espionage
    3. Communism must not spread/propaganda, economic agreements and alliances
    4. Colonialism should end/manipulating the US and USSR to secure independence
  2. What did the Truman Doctrine and the Two Camp Speech have in common?
    1. Both charted a potential course for foreign relations based on beliefs  about ideological and political adversaries
    2. Both rallied Western European allies around the idea that unless democracy speaks with one voice, it will be overcome by communist propaganda
    3. Both called for massive amounts of money to be invested in economic recovery and weapons production simultaneously
    4. Both a and c are true
  3. The Soviet reaction to the Marshall Plan and creation of NATO suggest that:
    1. The USSR created its policies in reaction to US initiatives rather than a master plan
    2.  The USSR was confident in its capacity to build atomic weapons
    3. The USSR and China were about to form an alliance against the West
    4. The USSR was prepared to negotiate the status of Estonia, Latvia, and Lithuania
  4. Why was Czechoslovakia’s transformation into a Soviet satellite provocative to the US?
    1. It proved that Stalin had intended to absorb Czechoslovakia throughout World War II
    2. It illustrated the ineffectiveness of the CIA operations in Eastern Europe
    3. It demonstrated Stalin’s determination to force all communists to conform to Soviet will
    4. It proved that Moscow could insert communism where it had never existed before
  5. Which statement misrepresents events in the Eastern Bloc?
    1. While communism was popular in Eastern Europe after World War II, it often retained a nationalist element as seen in Italy and Czechoslovakia
    2. Yugoslavia’s brand of communism led to a break with Moscow which the US supported by opening military bases on Yugoslavian territory
    3. The USSR regarded Eastern Europe as a buffer zone and a source of materials with which to improve Soviet technology and industry
    4. The needs of East Germany and the needs of Romania were distinctly different due to the degree to which each of its economies depended on agriculture and trade

Sample Exam: Reading Comprehension and Cold War

Read the following passages and answer the questions that follow.

Throughout history, technology has played a significant role in the outcome of conflict and the ability of nations to accomplish political and economic objectives. Just as the invention of the railroad enabled the industrial nations to penetrate the interiors of Asia, Africa, and Latin America from which it could extract raw materials and to which it could bring finished goods, modern machine guns and submarines gave to those who possessed them the advantage in armed competition for land and markets. During the Cold War, the relationships between government, industry, and the scientific community were radically altered.  Changes in these relationships were pronounced in the United States, which by 1945 had emerged from World War II as a global, political and economic power.

The United States realized even before the war was over that technology and science were essential to national security and its ability to orchestrate and implement its economic and political agendas. In 1944, Dr. Vannevar Bush, an engineer who developed computer analogs (early protocols for what became the World Wide Web), and organizer of the Manhattan Project (which produced the atomic bomb), articulated a compelling rationale for government investment in science. Bush’s publication, Science, The Endless Frontier, asserted that science and technology was the hope of the future as it provided not only for the defense of the country, but for modern communication, improvements in disease control, and improvements in production. Bush argued that government should steadily increase its investment in scientific and technological research.

By 1950, the United States was spending approximately $10 billion dollars annually on scientific research and development. Its commitment to scientific research and development increased dramatically following the Soviet’s launching of Sputnik, an unmanned satellite, in 1957. Sputnik alarmed Americans who perceived the potential military utility of satellites. In 1960, the United States spent about $40 billion dollars on scientific research and development, which did not include funding for facilities and equipment.  In 1957 and 1965, the United States passed legislation that awarded schools and universities millions of dollars to promote and enhance education in math, science, engineering and technology. During the peak of the Cold War, 1958-1968, federal spending on research and development represented almost 3% of the United States’ Gross Domestic Product, with more than half dedicated to national defense.

Questions

  1. How do the first two sentences of the essay function?
    1. They establish a rationale for and justify the use of military interventions
    2. They illuminate the causes of the Cold War
    3. They provide a context for the main points of discussion
    4. They dispel the reader’s assumptions that technology is bad
  2. The passages about Sputnik clearly suggest that:
    1. The Soviet achievement was a catalyst for American investment in research
    2. The Soviet’s achievement was largely expected and so uneventful
    3. The Soviets were at the time the only nation capable of satellite production
    4. The Soviets deliberately launched a satellite to frighten the Americans
  3. An appropriate title for the essay would be
    1. How Vannevar Bush Saved America
    2. The Role of Technology in Imperialism
    3. The Military’s Shopping Spree
    4. Cold War and Scientific Research
  4. The last sentence of the second paragraph implies that:
    1. The United States had billions of dollars reserved for scientific research
    2. That United States up to 1944 had not perceived scientific research with much urgency
    3. That Vannevar Bush was highly influential in political circles and Congress
    4. That the United States owed it to stockholders to invest in research and development
  5. The fourth sentence in the third paragraph evidences that:
    1. Americans understood the relationship between education and scientific advancement
    2. American schools and universities were behind the Soviets in terms of funding
    3. American curriculum was too heavily saturated with liberal arts and humanities
    4. Americans wanted to shift scientific studies from schools to government institutes
  6. What is the purpose of the parenthetical information in the second paragraph?
    1. It offers evidence that justifies Bush’s objectives
    2. It clarifies the significance of Bush’s accomplishments
    3. It proves that Bush’s request is credible and reasonable
    4. It illuminates the relationship between values and science
  7. The use of the world “compelling” in the second paragraph suggests that:
    1. The reader may not understand what Bush is arguing
    2. The author of the essay doubted Bush’s assertions
    3. The author saw merit in what Bush was claiming
    4. The audience Bush addressed was scientifically ignorant

Discussion Points

  1. Form A contains more questions aimed at lower level thinking than Form B. Lower level thinking, according to Blooms taxonomy, targets the student’s ability to recall and explain information.
  2. While Form B asks more questions that are aimed at higher level thinking, such as cause-effect relationships, implications, comparisons and contrasts, and significance of events, it is still possible for students to guess the correct answers and earn a passing grade on the exam.
  3. The multiple choice test does not require students to articulate their understanding with original writing, but when coupled with writing exercises, might provide instructors with some sense of how well students think critically and comprehend the complexity of the subject.
  4. Short essay prompts that require one to five paragraphs and that target a particular cognitive task, such as the student’s ability to evaluate or to synthesize (in the context of this module) might include:
    1. Critique the revisionist perspective of Cold War origins from both a traditional and post-revisionist perspective.
    2. Would there have been a Cold War if Franklin Roosevelt had lived and been re-elected? Provide evidence to support your claims.
    3. What was the most significant event that established the Cold War as the dominant geo-political struggle of the 20th century and why is this so?
  5. Multiple choice questions can be used in conjunction with short readings. By doing so, the instructor may detect the degree to which students read comprehensively and critically. Multiple choice questions stemming from a reading may target the student’s ability to draw inferences, identify implications, perceive bias or perspective, discern main and peripheral ideas, detect lacunae, assess the strength or credibility of assertions and evidence, and recognize themes and patterns.
  6. Following up with students after the test is important to learning. A careful review of the exam will reveal not only how well students performed in general, but allow for students to examine the rational for each test question, why some plausible answers are incorrect, and to discuss the significance of a particular idea or discrete information. Reviewing the test provides an opportunity for instructors to clarify information and deepen students’ understanding. The process also alerts instructors to potential revisions that need to be made in order to proceed with the course in ways that will improve student learning.

Sample Exams: Nursing

Directions

Read the following two multiple choice exams and compare and contrast the questions and answers. Which exam targeted higher level thinking?  Which one made it easier to get a correct response by guessing? Both exams test students’ knowledge and understanding of Public Health.

Form A

  1. Which of these is not a characteristic of alcoholism?
    1. High blood pressure
    2. Swollen and irritated liver
    3. Acute sensitivity in hands and feet
    4. Increase in heart size
  2. The emotional cycling between good moods accompanied by high energy and euphoria and bad moods accompanied by low energy and despair is characteristic of:
    1. Depression
    2. Dissociation
    3. Bi-polar disorder
    4. Borderline personality
  3. An imbalance in electrolytes can be quickly restored by:
    1. Getting adequate sleep for at least one week
    2. Increasing one’s exercise regiment
    3. Drinking fluids such as juices or sport beverages
    4. Increasing the amount of fiber in one’s diet
  4. Migraine headaches are often triggered by:
    1. Lack of regular exercise
    2. Food allergies and stress
    3. Excessive consumption of vitamins
    4. Hyperthyroid activity
  5. According to the National Center for Victims of Crime, which statement is false regarding elder abuse?
    1. Approximately a half a million people over age 65 are abused per year
    2. Women are more likely to be abused than men
    3. Over 80% of the abuse took the form of care-giver neglect
    4. Many elders are financially abused by family members

Form B

  1. The nurse makes a home visit to a 48-year old female recovering from hip surgery. Which finding has the greatest implications for the client’s care?
    1. The only food in the house is that which has been delivered by take-out restaurants
    2. The client has placed a portable toilet in the same room as her bed
    3. The client has a neighbor who routinely mows his lawn at 7:00 AM on Thursdays
    4. The client has some rosacea around her nose and cheeks and an empty bottle of vodka in the trash
  2. A conscious 24-year client is admitted to the emergency room after being involved in a car accident. While tending the client’s injuries, the nurse notices scarring on the clients forearms and that the client has difficulty looking the nurse in the eye, which should alert the nurse to:
    1. Ask the client how he or she is feeling and be prepared to contact a psychiatrist
    2. See that the client has had a tetanus shot recently
    3. Monitor heart rate and blood pressure to detect tachycardia
    4. Make sure the patient’s vision has not been adversely affected by the trauma
  3. Following a client’s colonoscopy, the most important thing for the nurse to attend is:
    1. The client’s comfort level in passing air from the colon
    2. The client’s  level of fluids and electrolytes
    3. The administration of antibiotics
    4. Ensure the client is breathing properly
  4. A 42-year old client enters the emergency room complaining of severe head pain and vomiting. The nurse documents an elevated blood pressure, asks the client to squeeze his or her hand, and escorts the client to a room where the client will wait for the doctor. What additional information is important for the nurse to report to the doctor who will see the client?
    1. Whether the client was recently assaulted or injured by others
    2. Whether the client has any allergies or is sensitive to light
    3. Whether the client has a family or friend to sit with the client
    4. Whether the client exercises on a regular basis
  5. A nurse makes a visit with a friend to the friend’s father, a 78-year old widowed man with Type-Two diabetes who lives alone. While in the man’s home, the nurse detects the smell of urine and notices that the man’s home is littered with unwashed dishes and dirty laundry. The most responsible and professional thing for the nurse to do is:
    1. Ask his or her friend why he or she is not helping the man to care for himself
    2. Make an offer to clean the house and be sure the man’s blood sugar is tested daily
    3. Express care and concern to his or her friend and offer information about in-home care
    4. Contact the county office to report that elderly abuse may be occurring in the home

Reading Comprehension

Read the following passages and answer the questions that follow.

The International Center for the Study of Psychiatry and Psychology (ICSPP) estimates that between five and nine percent of all children in the United States suffer from Attention Deficit Hyperactivity Disorder (ADHD). The disorder is characterized by the inability to sustain one’s attention or focus on an idea or task for a prolonged period. Children who suffer from ADHD are likely to have difficulty learning in traditional, formal school settings, as they are easily bored and distracted. Many also experience behavior disorders including aggressiveness towards others and impulsivity. ADHD has generated considerable concern as its causes are not completely understood and its treatments have unknown implications for those who undertake them.

In a study that examined the prevalence of ADHD world-wide, Dr. Stephen Faraone and his colleagues found that ADHD seems to be more prevalent in the United States and Great Britain than in other nations. Australia, Sweden, Iceland, and Italy reported lower incidents of ADHD than what were found in the United States, causing researchers to question the role of culture as a causal agent. Some claim that differences among national prevalence is due to the frequency and precision of diagnostic procedures.  While the causes of ADHD are not known, scientists believe that contributing factors include genetics, the mother’s use of alcohol and cigarettes while pregnant, diet, and socialization.

Many scientists believe that regardless of the cause, children with ADHD are over-medicated in the United States. Dr. Peter Breggin, Director of the ICSPP, claims that treating children with stimulants, such as Ritalin, Dexedrine, and Adderall creates toxic brains and that the side effects can be deadly. Breggin reports that pharmaceutical companies have profited handsomely from the public’s willingness to rely on psychotropic drugs to solve complex behavior problems. He notes that ADHD might be better addressed through improved parenting and educational reforms. Dr. Gretchen LaFever of Eastern Virginia medical School found in a state-wide study that 84% of the children in Virginia diagnosed with ADHD took medication for their condition and that of those children, 70% were taking two different prescriptions to treat ADHD. She also found that those who were not taking medication for their condition were children without health insurance.

Questions

  1. The purpose of the first four sentences of the essay is:
    1. To create an interest in and compassion for the suffering of children with ADHD
    2. To provide a general understanding of the topic discussed in the essay
    3. To prove that pharmaceutical interventions for ADHD is justified
    4. To raise the reader’s awareness of legal implications regarding ADHD treatment
  2. What is the implication of the second and third sentence of the third paragraph?
    1. That treatment for ADHD should involve more changes in culture and behavior
    2. That treatment for ADHD in America is inconsistent with what other nations do
    3. That pharmaceutical companies have no sincere interest in children’s health
    4. That in the United States many parents do not know how to raise children
  3. An appropriate title for this reading might be:
    1. Bad Schools, Bad Health
    2. Getting the Diagnosis Right
    3. Psychiatric Community Challenges Corporate Medicine
    4. The ADHD Controversy
  4. The last two sentences of the essay offers evidence for which assertion?
    1. That parents can cause illness by not providing health care
    2. That countries with socialized medicine have less children with ADHD
    3. That Virginia has the highest rate of ADHD in the United States
    4. That insurance companies may be profiting from high rates of prescriptions
  5. The third sentence in the second paragraph suggests that:
    1. Many countries do not have the technology to accurately test for ADHD
    2. Some countries have a poor system of filing and tracking health records
    3. The world-wide prevalence of ADHD might increase due to better testing
    4. Repeated testing for ADHD tends to inflate the actual prevalence rates

Discussion Points

  1. Form A tended to prompt recall of facts, while Form B tended to prompt clinical judgment
  2. As with multiple choices tests in other subjects, multiple choice tests in nursing leaves open the possibility that students will guess correctly. Supplementing multiple choice exams with writing components offers instructors the means of understanding how well students understand procedures and how to manage situations that may be exceptions to established rules. Some prompts for one to six paragraph essays might include:
    1. On your hospital rotation, you are to care for a 62-year old female who had just had a bowl resection. Describe the steps you take when you first meet the patient and how you will administer care over the next 72 hours. Identify your chief priorities, how you will meet them, and which variables may cause you to alter your course of action and why.
    2. While at work at the hospital, you notice a fellow nurse taking narcotics from storage and placing it in his or her pocket. Describe what action you would take, step by step, and why.
  3. Critical reading exercises may also accompany multiple choice tests in nursing. These exercises allow the instructor to detect the degree to which students read comprehensively and critically. Multiple choice questions stemming from a reading may target the student’s ability to draw inferences, identify implications, perceive bias or perspective, discern main and peripheral ideas, detect lacunae, assess the strength or credibility of assertions and evidence, recognize proper and improper clinical procedures, and recognize ethical and professional obligations related to nursing.
  4. Following up with students after the test is important to learning. A careful review of the exam will reveal not only how well students performed in general, but allow for students to examine the rational for each test question, why some plausible answers are incorrect, and to discuss the significance of a particular idea or discrete information. Reviewing the test provides an opportunity for instructors to clarify information and deepen students’ understanding. The process also alerts instructors to potential revisions that need to be made in order to proceed with the course in ways that will improve student learning.

Tips and Guidelines for Writing Multiple Choice Exams

General Guidelines with Writing Multiple Choice Questions

  1. Concentrate on testing central, critical content rather than peripheral, trivial content.
  2. Consider the reading level of learners. Question difficulty should come from content not wording.
  3. Make sure all questions are precise, clear, and non-ambiguous. Include all necessary
  4. qualifiers but don’t provide unessential information or irrelevant sources of difficulty.
  5. Avoid words such as always, often, frequently, never, none, rarely, infrequently because
  6. they tend to trip up learners.
  7. Make sure that each item has an unambiguous correct answer or answers (for “select al
  8. that apply” questions).
  9. Make sure that the question is free of clues that might indicate the correct answer.

Tips for Writing Good Multiple Choice Question Stems

  1. The stem should make sense alone. The learner should ideally be able to answer it without seeing the answers/distractors.
  2. Provide adequate and clear instructions. For example, if the learner needs to select the best answer, make this clear. Selecting the best answer is different than selecting a correct answer. If they need to select all of the correct answers, this needs to be specified.
  3. Provide graphics or media as needed but do not provide irrelevant graphics or med

Tips for Writing Good Multiple Choice Question Answers/Distractors

  1. Don’t include any silly or irrelevant distractors.
  2. ALL distracters must be plausible. These are the best types of plausible-but-incorrect distractors:
    • Common errors and commonly held myths or misconceptions (for those with less knowledge or skill)
    • Statements that are true, but do not answer this question
    • Content that is paraphrased incorrectly
  3. You can vary the number of distractors. Three to five distractors is ideal. Remember that a smaller number of answers/distractors increase the probability that a guess will be correct, but don’t add distractors unless they are plausible.
  4. If answers/distractors include best and not-as-good alternatives (“Select the best answer…”), make sure that there is an unambiguously correct answer or answers. Provide enough detail to differentiate best from not-as-good.
  5. Keep answers/distractors about the same length.
  6. Avoid answers/distractors that combine distractors (“b and c”).
  7. Avoid using “all of the above,” and “none of the above.”
  8. Make sure to use different placements of the correct answer. The most common placement of correct answer is c and test-wise learners know this.