Objective Assessments

Multiple-choice (M/C) items: 

• Four answer options, including one correct answer and three incorrect options
• Stems should typically start with a question word and be in the form of a complete interrogative statement.

Multiple-select (M/S) items:

• 4–7 answer options
• Two or more correct answers
• Include at least the same number of incorrect answers as correct answers
• Stems should typically start with a question word and be in the form of a complete interrogative statement.
• To align with standardized assessments and licensure exams—such as NCLEX and Praxis—instructions are "Choose all that apply."
• To reduce difficulty of M/S items, include the number of correct answer options in the stem when it is clear and not awkward to do so (e.g., "Which two factors affect…"). Then, follow with these instructions:
  • "Choose [#] answers." (Replace "#" with the number of correct answers.)

Example:

What are two functions of fat in the human body?

Choose 2 answers.

a. Fat helps maintain a strong immune system.

b. Fat enables the transport of vitamins A, D, E, and K.

c. Fat helps maintain fluid and electrolyte balance.

d. Fat provides a concentrated source of energy.

• Colleges may not require specifying the number of correct answers in an effort to better prepare students for standardized assessments for certification or licensure. When M/S is used in an OA, the AD should discuss with the assessment strategist whether the instructions should be "Choose all that apply" or "Choose # answers."

Multiple choice exhibit option (MCEO):

• 4–5 answer options
• One correct answer
• Each answer option should be about three sentences.
• Stems should typically start with a question word.

Matching:

• Answer options should be less than 80 characters.
•  Distractors are optional.
• Question stems should be in the form of a complete imperative statement, usually beginning with either "Match" or "Classify."
• The stem wording lists the prompts first and the answer options second (to correspond with the order of the prompts and answer options).
• Longer text should be placed in the prompt section (on the left) so that students can more easily view the prompts in the delivery system.
• The first instruction should state, "Answer options may be used more than once or not at all." 
  • To avoid confusion, this instruction can be omitted when students are required to put things in sequence.
• The second instruction should state, "Select your answer from the pull-down list."

All-inclusive options:  WGU does not use all-inclusive options, such as "all of the above" or "none of the above," nor should there be answer options that combine answer options (e.g., "both A and B").  

Answer option randomization: Answer options should be shuffled in the delivery system; however, option shuffling can break. To ensure the correct option does not always occur in the same order in live assessments, the correct option should not always be in the same position (e.g., option A) in development. This does not apply to CPLT items.

Clarity: The given (if included), stem, and answer options should be clearly, concisely, and simply stated. Whenever possible, minimize the amount of time it takes to read each item. The item should only contain information that is relevant to answering the question, not instructional information. There should not be distracting or confusing information.

Cohesion: All the elements of an item (i.e., given, stem, prompts, and answer options) should clearly relate to each other. All answer options should grammatically, logically, and directly answer the stem.

Correct answer: An item's correct answer(s) should be both 100% correct and the only correct answer(s).

Clues: Neither the question stem nor the answer options should contain clues that reveal the correct answer to students. You can avoid clueing with the following guidelines:

• Answer options should be of similar complexity, detail, and length. Avoid having a single option that stands out from the others, especially if it is the correct answer.
• Distractors should avoid the use of absolutes (e.g., "always," "never," "only") and extremes (e.g., "eliminate," "solely") to indicate an incorrect answer.
• Key terms that appear in the given or stem should not be included in the correct answer unless all answer options include those same terms.

Enemy Items: Enemies are sets of items that differ in wording and construction but should not be included on the same assessment form because of one of the following:

1. They elicit the same cognitive response process and assess the same concept (also called variants).
2. One item provides clues for or gives away the correct answer to another item or items.

Enemy items can be used on different operational forms, but they need to be coded as enemies. Items that are too similar in wording to be considered unique items are called clones, and clones may not be accepted or used.

Focused stem: The stem should contain the central idea of the item, not the answer options. The student should be able to understand what is being asked without reading the answer options.

Negative questions: Avoid negative wording, such as "What is not true…" or "Which of these is NOT correct?" The item should be positively worded unless specified in the evidence statement.  

Necessary information: The item should contain all information necessary to answer the question. This includes tables, images, or other reference materials.

Nonanswers: Avoid nonanswers for distractors, especially when they contradict the question.

Example:

Stem: "What is an effect of Title IX on school systems?"

Distractor: "There is no effect."

The question implies an effect, so this option is easy to eliminate.

There may be some exceptions when a nonanswer is a plausible distractor or a correct answer.

Parallelism: Answer options should be parallel in nature. This means they should all start with the same part of speech and be in the same tense. For example, each option could start with a noun or a present-tense verb.

Plausible distractors: Distractors should be incorrect but still plausible. Distractors should be designed so that a student who does not have competence in the area could consider them to be correct answers. Distractors must be completely believable and should not use humor. Great distractors often come from common mistakes or misconceptions about the material. Distractors should not be incorrect only because they are irrelevant to the scenario or question stem.

Question form: Stems for multiple-choice, multiple-select, and MCEO items should be complete sentences, written in question form (i.e., starting with who, what, which, how, when, why, etc.). However, an introductory phrase may be placed before the question word (e.g., "In which," "For which purpose," "Based on," "According to"), if it helps make the stem clearer. Matching items and order objects items should be phrased as imperative statements.

Readability: An item should test the student's knowledge of the subject, not their reading comprehension (except when comprehension is what is being assessed). A novice should be able to read the item and understand what is being asked. Outside of content-specific language, items should be simple and clear in word choice and sentence structure.

Sensitive content: An item should not cause an emotional reaction to the material and thereby interfere with a student's ability to take the assessment. This includes avoiding the use of humor in the given, stem, or answer options. Whenever possible, avoid using individual names or gender-specific pronouns. If names or pronouns are necessary for clarity or to avoid awkward phrasing, use an equal number for each gender across all items. Brand names should also be avoided unless necessary for the content.

Single idea: Items should represent one idea or concept.

Should: Use "should" rather than "would" or "could." Using "would" and "could" often leads to subjectivity in items; however, they may be appropriate in rare cases.

Example:

A teacher wants to assess how well students have learned the material for a unit on Asian geography.

Which type of assessment should this teacher use?

Reasoning:

If this stem used "would" or "could" instead of "should," students could argue that they would not know what the teacher would use, so any answer could be considered correct.

Exceptions:

A firm begins a strategic business initiative by launching a project that is expected to result in a more efficient and tailored service to its customers.

Which factor would reflect a more efficient and tailored service through the new project?

a. Liaison with contractors
b. Improvement in service
c. Feedback from customers
d. Decrease in customer costs

Specificity: An item's content should not be specifically tied to the learning resource. An industry expert should be able to answer the question without engaging in the learning resource.

Subjectivity: Items should not require students to form an opinion or make assumptions to answer the question. Generally, there should be one objectively correct answer, not a best answer (e.g., "Which of these is best…"). There may be exceptions depending on the content being assessed.

Time sensitivity: Whenever possible, avoid using specific dates and years in order to prevent assessments from becoming outdated. However, in some situations, it may be beneficial to include specific years to create an authentic situation, to reference a specific event, or to make an item clearer and more straightforward (e.g., accounting or history assessments).

What, which, and of the following: What and which suggest different things. Both words are acceptable for use in items, but each word has its own purpose:

• What precedes a verb (e.g., What is the fastest animal in the jungle?).
• Which precedes a noun (e.g., Which dog is wagging its tail?).

Avoid using "Which of the following" or "Which option" when referring to answer options. Use a more descriptive noun instead (e.g., "Which animal," "Which theorist").

Abbreviations: Abbreviations, such as initialisms and acronyms, should be spelled out on first use within the item with the abbreviation included in parentheses after the term. After the first use in each item, the abbreviated form may be used. Some abbreviations may stay in their abbreviated form throughout an assessment if the abbreviation is common knowledge or is part of the required learning for the evidence statement or competency.

Blank spaces: To indicate a blank space within an item, use four underscore marks.

Example:

4 + ____ = 49

Bulleted lists: Bulleted lists within the given statements of items should use automatic bullets. Ensure that all lists are visually clear and readable.

Capitalization: The first letter in answer options should be capitalized to maintain consistency. If the word is a hyphenated compound, only the first word is capitalized (e.g., "Whole-life insurance"). Exceptions may be made for industry-specific standards. Do not capitalize answer options for drop-down lists in the middle or end of a sentence unless the term would otherwise be capitalized (e.g., a proper noun).

Dialogue: Answer options that could be considered verbal responses from an identified person from the given or stem should be put in quotation marks.

Example:

A nurse practitioner (NP) is working with a patient who explains a frustrating situation. The patient states, "I am feeling angry."

Which response should the NP provide?

a. "Try to express your thoughts in a journal."
b. "What causes you to experience the anger?"
c. "Imagine yourself in the situation again."
d. "Have you had similar feelings of anger?"

Given statements: Given statements are separated from the stem by a hard return. Use consistent terminology between the given statement and the question stem. In items that have given statements, stems that refer to information in the given statement should use the word this instead of the.

Example:

Given: John has trouble getting up in the morning.

Stem: What would help John solve this problem?

Introductory phrase in items with passages: Language or literature items that have a given passage of text consisting of four or more sentences or five or more lines should start with "Read the following passage:" or something similar. The wording can be adapted according to the item (e.g., using "paragraph" or "poem" instead of "passage" or providing the title or author name). Short passages, literature, poems, etc. do not need to start with the introductory phrase.

Example:

Read the following passage:

Exercise is beneficial to a person's health and emotional well-being. It is an activity that raises the heart rate and causes exhaustion. It can also lower cholesterol, blood pressure, and weight. Exercise was an integral part of ancient Greek culture. Physical activity produces endorphins that contribute to a feeling of happiness. There are many forms of exercise, such as cleaning the house or climbing stairs.

Which two details support the main idea as expressed in the first sentence of the given paragraph?

Choose 2 answers.

 A. Exercise can lower cholesterol, blood pressure, and weight.
 B. [ . . . ]
 C. [ . . . ]
 D. [ . . . ]

Pronouns: Pronouns should have clear referents and be used appropriately, both in the question stem and the answer options.

Quotation formatting: Whenever possible, OA content should be original and unique. If it is not possible to write an original passage or if a quote is necessary to test evidence statements that need to reference another source, such as literature, a quotation may be used. Quotations should be taken from the public domain or from a learning resource that WGU has already adopted as part of the course. Do not use the quotation in the exact context as used in the learning resource. For example, if a passage appears in a section of the book that teaches the concept of inference, the passage should not be used in an item that asks about inference; however, it may be acceptable to use this passage in an item about differing points of view or some other topic.

All quotations must be accompanied by a full citation. (See the "Quotation Formatting" entry for information on how to format quotations.) Quotations are not exempt from the fairness and bias guidelines.

Put quotation marks around quotes and include reference information below the quote.

Repeated words: If a term or synonymous terms are repeated in every answer option, they should be incorporated into the question stem or omitted when appropriate. That said, the answer options should be clear and straightforward. Therefore, if the item is clearer with repeated terms in the answer options, they should be retained.

Second person: Stems, givens, and answer options should generally be written to avoid second person (i.e., "you").

Terminal punctuation in answer options: Use a period or question mark at the end of answer options only if the option is a complete sentence. This includes imperative sentences (i.e., commands). Sometimes an option seems like a complete imperative sentence but is not. Only an implied "you" as the subject makes a sentence imperative; however, since second person is typically avoided in item stems, these will not frequently occur.

Images: If any images are needed to accompany a stem or given, they should be placed after the stem, as it makes sense to do so. This will enable students using screen readers to know what the item is asking for before delving into the alternate text. (See the "Table Formatting" entry for a similar rule.)

Example:

Which mode of inheritance is possible for this trait, given the following diagram?

[Insert image referenced in the stem.]

Additionally, it is useful for an introduction or reference to the image to be made in the stem. It is advised to put this reference at the end of the stem so that the stem still begins with a question word (e.g., "what").

Color in images: An image should make sense without any use of color. Color should not be used as the only method for conveying information, especially in charts and graphs.

Text in images: WGU's preferred font for assessments is Verdana—with the exception of Courier New for IT code. Use Verdana for any font within images.

Image file type: Provide images in .svg format whenever possible. Otherwise, .jpg, .jpeg, or .png files are acceptable.

Image size: Images will be displayed to users initially at 50% of the image resolution (pixels wide by pixels tall). For example, an image that is 200 pixels by 80 pixels will be initially displayed at 100 pixels by 40 pixels. This makes it possible for users to zoom into the displayed image to 200% without losing any image clarity. Because images will be displayed initially at 50% zoom, all images must be created at twice the resolution. For instance, if normal text content is authored using a font size of 11 pt, then the font size of text in an image should be 22 pt. Any diagram or picture that is part of an image should also be twice the normal size so that it will still look good when it is scaled to its initial 50%.

Table formatting: If one or more tables are needed to accompany a stem or given, they should be placed after the stem, as it makes sense to do so. (See "Images" entry.) Additionally, it is useful for an introduction or reference to the table to be made in the stem. It is advised to put this reference at the end of the stem so that the stem still begins with a question word (e.g., "what").

Example:

What is the profit margin, given the following table?

[Insert table referenced in the stem.]

Ensure that tables in the given, stem, and answer options are formatted uniformly. When multiple tables are used, the column widths should align between the different table to make the tables easier to read. Table and column headings should be in bold. All text should be left justified, unless the industry standard is otherwise. However, monetary amounts given should be aligned by the decimal point. For ADA compliance, tables should be coded in HTML, not included as images.