How can we measure whether a person knows something?

According to Darwin P. Hunt, “To measure something means to assign a number to a characteristic (knowledge) of an object (a person) or event according to a set of rules”.

It should be noted that the meaning of the number is defined by the set of rules assigned.

“The currently used multiple-choice test or any other epistemetric method (measurement of the phenomenon of knowledge) may be considered as a “set of rules” by which the numbers (scores) or measurements are produced – and thus, knowledge may be operationally defined” (Hunt).

In general, tests created today for measuring a person’s knowledge on a particular topic/ subject are “aimed at composing test items that represent the topic; and are fair and unbiased” (Hunt).

To determine whether a person possesses knowledge on simple addition, we can ask questions that are representative of the topic, such as ªWhat is the sum of 12 + 13? ; Or we might pose the question as a response selection or multiple-choice task:

12 +13 = (a) 7

(b) 14

(c) 24

(d) 25

“Current testing practice is to observe which alternative a person selects and infer that he/she knows (if a correct answer is selected) or does not know (if the correct answer is not selected) how to add two digit numbers.

However, a test taker can select the correct answer without knowing how to add, e.g. in the above example, the chance of being correct by guessing alone is 1/4 = 25 percent. The reliance exclusively on the correctness of the answer implies that the person who provides a correct but unsure answer or who made a lucky guess possesses knowledge equivalent to a person who is correct and extremely sure of it”.

I completely agree with Hunt when he discusses how multiple choice questions can measure knowledge. Basically, there is no foolproof and fair way to give a multiple choice test. Students who may know nothing about the material can still receive the same grade as a student who knows all of the material thoroughly, by chance of course. If you strictly follow the guidelines of multiple choice questions, you would have to infer that the student, who guessed on every question and got a one hundred, knows the material as well as the student who thoroughly knows the material and also received a one hundred.

“Similarly, in today’s multiple-choice tests if an incorrect answer is selected, then it is interpreted simply to mean that the person does not know the answer, i.e. is uninformed. This inference is misleading. Specifically, the person may be extremely sure that the incorrect answer which he/she selected is correct and, thus, may be misinformed– which is much worse than being uninformed. A sure-but-wrong belief, used confidently as a basis for making decisions and

 Taking actions, may lead to surprising errors in performance – sometimes with tragic results”.

To further elaborate on my previous comment, I also strongly agree with Hunt on his opinion about the misleading natures of multiple choice tests. If a student gets a multiple choice question wrong, the teacher/test giver would then have to assume that the student does know that portion of the material. But how does a multiple choice test compensate for a student who believes that his answer is correct (possibly by misreading or received invalid information), but is actually incorrect?

 http://andrewvs.blogs.com/usu/files/p100.pdf

Advertisements
Leave a comment

1 Comment

  1. This is very clever…and made my head spin a bit. There is a science to writing MC questions…I fear many teachers do not really do a good job at writing them, and as a result, there are many poorly worded questions (which is why I always put extra points on quizzes!!)
    some new information on the topic informational post: attempts to synthesize information and form new meaning; well organized
    written in an interesting style and voice; words chosen mostly reflect author’s personality and brings content to life for the most part; sentence fluency is mostly achieved
    few spelling errors; few grammar errors; some formatting to help make the post easier to read; multimedia adds new information or perspective to post; several links included that add to the reader’s understanding post; may be categorized or tagged; citations for most images

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: