الاختيار من متعدد Writing Multiple-Choice Test Items

[Northie]

هذا الموضوع لمعلمي اللغة الانجليزية أومن يضع أسئلة الإختيار من متعدد....... وفي الطريق موضوع مستقل للطلاب عن كيفية الإجابة على هذا النوع من الأسئلة

Exams are over! However,this doesn't mean we delay discussing some issues concerning them to the next round of examinations. Some years ago, the Eng. Dept. sent the following circular on MCT to schools. I found it stored somewhere in my pc since then , and I would like to share it with you here in the forum



Dear colleagues,

With the adoption of Multiple-Choice Test (MCT) as a means of measuring the academic attainment of our students, many training programmes were held by different educational departments concerned with this project,here and around the kingdom. The Educational Supervision ,represented by the English Language Supervision, also conducted many intensive workshops for you to be fully aware of the fundementals of MCTs.
This circular, along with other previous circulars on the same subject, will help you put a few final touches on your examinations to meet the recommended requirements of such a type of testing. I also advise you ,for your professional development ,to visit/contact the English Department in case you have
some questions related to constructing MCTs ,or even to borrow some references on statistical analysis of language tests.



The objective of this digest is to set out some conventional wisdom for the construction of multiple-choice tests, which are one of the most common forms of teacher-constructed tests.

Before proceeding, it will be useful to establish some terms for discussing multiple-choice items. The "stem" is the introductory question or incomplete statement at the beginning of each item and this is followed by the options. The "options" consist of the answer--the correct option--and "distractors"--the incorrect but (we hope) tempting options.

GENERAL OBJECTIVES

As a rule, one is concerned with writing stems that are clear and parsimonious, answers that are unequivocal and chosen by the students who do best on the test, and distractors that are plausible competitors of the answer as evidenced by the frequency with which they are chosen. Lastly, and probably most important, we should adopt the attitude that items need to be developed over time in the light of evidence that can be obtained from the statistical output typically provided by a measurement services office (where tests are machine-scored) and from "expert" editorial review.

PLANNING

The primary objective in planning a test is to outline the actual course content that the test will cover. A convenient way of accomplishing this is to take 10 minutes following each class to list on an index card the important concepts covered in class and in assigned reading for that day. These cards can then be used later as a source of test items. An even more conscientious approach, of course, would be to construct the test items themselves after each class. The advantage of either of these approaches is that the resulting test is likely to be a better representation of course activity than if the test were constructed before the course or after the course, when we usually have only a fond memory or optimistic syllabus to draw from. When we are satisfied that we have an accurate description of the content areas, then all that remains is to construct items that represent specific content areas. In developing good multiple-choice items, three tasks need to be considered: writing stems, writing options, and ongoing item development. The first two are discussed in this Digest.

WRITING STEMS

We will first describe some basic rules for the construction of multiple-choice stems, because they are typically, though not necessarily, written before the options.
1. Before writing the stem, identify the one point to be tested by that item. In general, the stem should not pose more than one problem, although the solution to that problem may require more than one step.
2. Construct the stem to be either an incomplete statement or a direct question, avoiding stereotyped phraseology, as rote responses are usually based on verbal stereotypes.
3. Avoid including nonfunctional words that do not contribute to the basis for choosing among the options. Often an introductory statement is included to enhance the appropriateness or significance of an item but does not affect the meaning of the problem in the item. Generally, such superfluous phrases should be excluded. For example, consider:

"The Saudi flag has two colors. One of them is

(1) red (2) green (3) black (4) yellow"

versus

"One of the colors of the Saudi flag is

(1) red (2) green (3) black (4) yellow"

In particular, irrelevant material should not be used to make the answer less obvious. This tends to place too much importance on reading comprehension as a determiner of the correct option.

4. Include as much information in the stem and as little in the options as possible. For example, if the point of an item is to associate a term with its definition, the preferred format would be to present the definition in the stem and several terms as options rather than to present the term in the stem and several definitions as options.

5. Restrict the use of negatives in the stem. Negatives in the stem usually require that the answer be a false statement. Because students are likely in the habit of searching for true statements, this may introduce an unwanted bias.

6. Avoid irrelevant clues to the correct option. Grammatical construction, for example, may lead students to reject options which are grammatically incorrect as the stem is stated. Perhaps more common and subtle, though, is the problem of common elements in the stem and in the answer.

Other rules that might be listed are generally commonsensical, including recommendations for independent and important items and prohibitions against complex, imprecise wording.

WRITING OPTIONS

Following the construction of the item stem, the likely more difficult task of generating options presents itself. The rules we list below are not likely to simplify this task as much as they are intended to guide our creative efforts.
1. Be satisfied with three or four well constructed options. Generally, the minimal improvement to the item due to that hard-to-come-by fifth option is not worth the effort to construct it. Indeed, all else the same, a test of 10 items each with four options is likely a better test than a test with nine items of five options each.

2. Construct distractors that are comparable in length, complexity and grammatical form to the answer, avoiding the use of such words as "always," "never," and "all." Adherence to this rule avoids some of the more common sources of biased cueing. For example, we sometimes find ourselves increasing the length and specificity of the answer (relative to distractors) in order to insure its truthfulness. This, however, becomes an easy-to-spot clue for the testwise student. Related to this issue is the question of whether or not test writers should take advantage of these types of cues to construct more tempting distractors. Surely not! The number of students choosing a distractor should depend only on deficits in the content area which the item targets and should not depend on cue biases or reading comprehension differences in "favor" of the distractor.

3. Options which read "none of the above," "both a. and e. above," "all of the above," etc., should be avoided when the students have been instructed to choose "the best answer," which implies that the options vary in degree of correctness. On the other hand, "none of the above" is acceptable if the question is factual and is probably desirable if computation yields the answer. "All of the above" is never desirable, as one recognized distractor eliminates it and two recognized answers identify it.

4. After the options are written, vary the location of the answer on as random a basis as possible. A convenient method is to flip two (or three) coins at a time where each possible Head-Tail combination is associated with a particular location for the answer. Furthermore, if the test writer is conscientious enough to randomize the answer locations, students should be informed that the locations are randomized. (Testwise students know that for some instructors the first option is rarely the answer.)

5. If possible, have a colleague with expertise in the content area of the exam review the items for possible ambiguities, redundancies or other structural difficulties.

Source: ERIC Clearinghouse on Assessment and Evaluation .ERIC Identifier: ED398236
Author: Kehoe, Jerard

(Permission is granted to copy and distribute this ERIC/AE Digest)
FURTHER READING
Airasian, P. (1994) "Classroom Assessment," Second Edition, NY" McGraw-Hill.
Cangelosi, J. (1990) "Designing Tests for Evaluating Student Achievement." NY: Addison-Wesley.

Grunlund, N (1993) "How to make achievement tests and assessments," 5th edition, NY: Allyn and Bacon.

Haladyna, T.M. & Downing, S.M. (1989) Validity of a Taxonomy of Multiple-Choice Item-Writing Rules. "Applied Measurement in Education," 2(1), 51-78.

We want a society in which
we are free to make choices,
to make mistakes, and
be generous and compassionate.
Margaret Thatcher

[ACME]

Thanks for sharing it with us

It is helpful

.

[Northie]

Read more about MCTs

Constructing
multiple choice tests

What happens: Learner

* Reads an incomplete statement or a question, also called the "stem"
* Reads three to five alternatives, including
the incorrect options, also called the "distractors"
the correct option, also called the "keyed response"
* Marks his or her choice

How to develop:

* Outline the core content that the test will cover
* Identify and prioritize key points, tasks
* Write out a series of stems
(The question format is generally is less ambiguous than the completion format)
* Write keyed responses in a clear, grammatical sentence
that follows the format of the stems
* Develop alternatives or distractors that follow the grammatical style,
are consistent in length, and avoid quoting the content of the course

When/how to use:

* Appropriate for all levels of cognitive ability
* Objective
* Useful for automated scoring
* Useful for item analysis, internal and over time

Ideal test items and stems:

* Use simple, direct language to present direct, core information for analysis, comparison, evaluation, etc.
(avoid cleverness, trickery, and verbal complexity)
* Include as much of the item as possible in the stem
(avoids repeated information and briefer alternatives)
* Present unique content
Do not build upon other questions
Do not supply answers to other questions
* Avoid negative stems
IF negatives are necessary, they are emphasized with underlined, bolded, CAPITALIZED, italicized, and/or colored indicators
* Use either the "correct answer" or "best answer" format
Correct answer: key response is clearly right and distractors are clearly wrong
Best answer: while distractors can be relatively viable, the key response is clearly demonstrated to fulfill all conditions of the test item. Best answer should avoid "none of the above," "both a. and e. above," "all of the above," options.
* Avoid "All of the following are true, except . . ."
unless testing for exceptions to rules
* Paraphrase, and do not directly quote, course content to avoid burdening students with detailed verbal analyses, to maintain focus on differentiating, as well as to avoid copyright issues
* Qualify significant information at the beginning of the stem:
Background, opinions, etc,: "According to...., ...."
* Do not introduce unfamiliar vocabulary and concepts in the test unless there is a relevant stated purpose in the test directions

Alternatives:

* Avoid generalizations that are open to interpretation
* Use the number of alternatives appropriate to a test item throughout the test, generally three to five (no necessity to use a consistent number throughout the test)
* Sequence alternatives in logical or numerical order;
Should there be no order, randomly assign correct answers in the sequence
* List alternatives on separate lines, indent, separate by blank line, use letters vs. numbers for alternative answers
* Pay attention to grammatical consistency of all alternatives

Keyed (correct) responses

* Vary position in sequence of alternatives

Distractors

* Include common misconceptions as distractors
* Include plausible content or viable cues in each distractor
Consider optional testing formats if distractors are difficult to develop
Avoid meaningless, even humorous distractors
* Re-use key words from the correct alternative to make distractors more viable
* Avoid "All of the above"
One incorrect distractor eliminates it; two correct distractors identify it
* Use "None of the above" as an effective option for factual information (historical dates, math, etc.) to make a question more challenging
* Do not use with a negative stem since it becomes a double-negative
* Do not use "None of the above" in a "best answer" question



Types of Multiple-choice questions:

Base questions upon, and preceded by, a statement, image, map, chart, etc.
Can accommodate alternative learning styles

Use the Roman Type for comparisons and contrasts
Test stem includes two options, each preceded by a (Roman) numeral.
Alternatives present optional combinations:

Example:

Which of the following is (are) accurate about...?

1. First option
2. Second option
1. I only
2. II only
3. Both I and II.
4. Neither I nor II.

See also: Kehoe, Jerard, Writing Multiple-Choice Test Items., 1995, ERIC Clearinghouse on Assessment and Evaluation Washington DC.

[M.o_o.N]

Northie


masha Allah brother

what a great thread

It's really helpful information indeed

a searcher can get use of it if he is studying the Saudi tests , exams and their methods

thank you so much , may Allah reward you

^_^

[Northie]

sirhasan

Moon

Thanks a lot for your kind replies :)

[BloumagrieT]

السلام عليكم ورحمة الله وبركاته

Northie

نفع الله بكـ وغفر لكـ و فتح عليكـ

وأجزل لكـ أجر هذا العطاء ومثوبته

وباركـ في وقتكـ وجهدكـ

كل الشكـــــــــــــــر

[Northie]

Thx sis BloumagrieT