<< Click to Display Table of Contents >> Navigation:  Sample items > FIMS items

The teachers involved in the FIMS study created 102 test items.

 

They were designed to be used with students aged 13 years.

 

It would not normally be the case that students would be tested with all of the items; 102 items would be a lot for a young student to answer, especially if some of the items are testing at higher performance levels (such as "solving problems").

 

What I will share with you today are results from a 14-item "testlet", by which is meant a short test made by using just a few of the items from the "pool" of 102 items.

 

Here are the first two items:

 

 

The first item is a short-answer "free response" question, also known as a "supply" question. Students had to answer it without using a calculator.

 

The second item is, or course, a multiple-choice item.

 

What are the advantages / disadvantages of using these two item types?

 

Short-answer items are very easy for teachers to write. But a test with these items has to be scored by hand, by a teacher. This takes time, and will involve the need for a scoring key of some sort (which itself takes time to develop).

 

Another problem with short-answer items is that teachers may not agree that a student answer of, for example, 25.3, is wrong -- my wife (for example) always wants to be kind to people and she might say "25.3 is close enough, I will give the student one point".  To prevent this from happening, the scoring key for the item would have to say "only 25.4 is correct; any other answer is to be marked wrong!".  (My wife might ignore this but if she did she should be fired.)

 

Multiple-choice items almost always have just one right answer, often called the "keyed-correct response"; the wrong answers are referred to as "distractors".

 

There are two disadvantages to using multiple-choice items. Students can get such items correct by guessing, and writing plausible distractors is not easy -- if you haven't done it, I can tell you that it's harder than you might think.

 

The advantage of using multiple-choice items is that they can be scored very, very quickly. In fact many times tests with multiple-choice items are scored by a "scanner" fed with what are called "mark-sense" answer sheets:

 

 

It is common for tests to use only multiple-choice items, very common. The main reason is scoring convenience. Quite a number of people also feel that a test may include more items if they are all multiple-choice because multiple-choice items usually take less time for a student to answer.

 

The next page shows all 14 items used on the FIMS testlet.