assessment: why is "demonstrate your understanding of ..." not measurable? (best, master)
Please register to participate in our discussions with 2 million other members - it's free and quick! Some forums can only be seen by registered members. After you create your account, you'll be able to customize options and access all our 15,000 new posts/day with fewer ads.
EXAMPLE: if the goal/objective is for the student to "demonstrate his understanding of an author's arguments," and there's a 10-question quiz on said arguments, and the students gets, say, a 20 (out of 100) on the quiz, then clearly the student has NOT "demonstrated his understanding" of the article.
If, on the other hand, said student gets a 90 or 100 on the quiz, then as the professor, I would likely believe that the student HAS "demonstrated his understanding" of the article.
So why is the wording "demonstrate your understanding of ..." considered to be "not measurable? Note, I've googled this, but all I get is a bunch of sites saying "you can't measure someone's understanding" (they all seem to say this) -- but they give no explanation whatsoever as to WHY you can't. So I am confused.
From the student's perspective, it may well be the student does have a good understanding but the student's understanding differs from the person that wrote the test. (One of my complaints about standardized testing too.)
Understanding is in many cases a subjective thing. Often requiring linkages to other background knowledge to achieve.
This is very true. An understanding of trigonometry is a very different thing from an understanding of Kant's philosophy or of a painting by Hieronymus Bosch.
Learning objectives should be specific. The learning objective should give the learner a clear idea of what they should be able to do by the end of a unit.
You have the measurement part down, it's the clear and specific part that needs some work when you say "demonstrate understanding of an article." If they are just identifying the main idea, then you would use the words "identify main idea".
Here are two good websites explaining how to write an effective learning objective.
"Well-written learning objectives should include observable behavior which can be measured. “Learning” and “understanding” are admirable instructional goals, but they are ambiguous and not observable or measurable." List of Measurable Verbs Used to Assess Learning Outcomes
Learning objectives should be specific. The learning objective should give the learner a clear idea of what they should be able to do by the end of a unit.
If you ask me, that's training rather than learning.
I am being told that I should be using words like "explain ..." instead of "demonstrate your understanding of ..." The problem is, the assessment I am using (it's a first-year course) is a simple quiz that, well, TESTS THEIR UNDERSTANDING OF the main arguments in the reading. The students are NOT identifying or explaining or interpreting or any of those other things -- they're just showing me that they read and understood the arguments by answering questions about the article.
I DO appreciate the responses here -- but the irony is that what the student is being asked to DO is to show that they understand an argument by simply answering questions about the article that has the argument. So how do you word the "objective" when the assessment item is a simple multiple-choice quiz?
The irony is that I DO understand (ha!) the Understanding by Design framework, and I try to design my courses that way -- i.e., by asking what students need to know/be able to do by the end of the semester, then working backward. But that is apparently not working because I am not using the correct academic lingo. Aaaggggh.
I'm guessing that you're developing a course syllabus!
Have you tried wording it like "the student will demonstrate that they understand the main arguments" instead of "has an understanding of the main arguments"? Although those sound alike, the committee might demonstrate an understanding of a difference between them.
...
I DO appreciate the responses here -- but the irony is that what the student is being asked to DO is to show that they understand an argument by simply answering questions about the article that has the argument. So how do you word the "objective" when the assessment item is a simple multiple-choice quiz?...:
Well, if you don't mind an input from the student's perspective, don't use a multiple choice quiz. I hate those things. Why? Well in terms of this discussion I have to put my understanding in terms of your words. It may seem a minor point, but often in a multiple choice test, more than once answer can be "right" depending on how different individuals interpret key words in both the reading and in the test question. I know that open ended questions are harder to grade, but the give the student a chance to put their reasoning on the paper. And that reasoning will tell more about their understanding than multiple choice.
I'm guessing that you're developing a course syllabus!
Have you tried wording it like "the student will demonstrate that they understand the main arguments" instead of "has an understanding of the main arguments"? Although those sound alike, the committee might demonstrate an understanding of a difference between them.
Yes, developing a syllabus for an outside organization's review (it's just for me and doesn't affect my job in any way -- I get a stipend for doing it). I've developed literally dozens of syllabi on classes from first-year survey courses to final-year capstone courses and they've gotten high marks for clarity, choice of readings, variety of assessment tools, depth, breadth, etc. from faculty colleagues and administrators.
Your suggestion made me chuckle, because I know the reviewers would hate it because again you're using that bad word "understand."
I think what it comes down to is that this outside organization asks me to have goals/objectives and assessment items (like quizzes, a paper assignment, or whatever) for everything I want the the students to do. But for testing CONTENT KNOWLEDGE AND UNDERSTANDING (i.e., not testing deeper understanding), a multiple-choice quiz should be fine.
The outside reviewer keeps saying that I need to say what the students will be able to DO in each module. But the next sentence tells me that saying the student will answer questions on a quiz, incorporate terms into a discussion post, or whatever -- in other words, saying what students will DO -- is NOT telling the reviewer what the student will do, it's simply giving an "activity" that the student will do.
So again, I clearly don't have the jargon down right, as "do" apparently means something particular in assessment-speak -- but she didn't tell me what that WAS.
Quote:
Originally Posted by tnff
Well, if you don't mind an input from the student's perspective, don't use a multiple choice quiz. I hate those things. Why? Well in terms of this discussion I have to put my understanding in terms of your words. It may seem a minor point, but often in a multiple choice test, more than once answer can be "right" depending on how different individuals interpret key words in both the reading and in the test question. I know that open ended questions are harder to grade, but the give the student a chance to put their reasoning on the paper. And that reasoning will tell more about their understanding than multiple choice.
I DO understand what you're saying, but when I use multiple-choice items -- which, incidentally, I write myself, I don't get from a test bank -- they really are mostly quick checks of whether the student did the reading and understood -- at a basic level -- what the author was saying.
I validate quiz (and test) items after students take them, and I throw out any question that appear to not be valid. (EG no one gets it right, or the students who do BEST on the quiz as as whole get that question WRONG -- which tells me there's something wrong with the question.)
Tnff, I would bet that if you took my quizzes after reading an article, you'd do fine. And you would also be able to do OTHER things like write papers, participate in discussions, etc. in order to show me your mastery of the material. (And incidentally, even for quizzes and exams, I often allow what I call an "earned curve" assignment, in which the students go back to the original source material, find the correct answers, and explain why they're correct. I offer this assignment because the whole point is for the students to learn the material, and this is one way of giving them a second chance to do just that.)
The irony is, some departments at my college use multiple-choice exams to assess their entire programs and that is acceptable to our assessment office. THAT seems very bizarre to me.
Please register to post and access all features of our very popular forum. It is free and quick. Over $68,000 in prizes has already been given out to active posters on our forum. Additional giveaways are planned.
Detailed information about all U.S. cities, counties, and zip codes on our site: City-data.com.
Please register to participate in our discussions with 2 million other members - it's free and quick! Some forums can only be seen by registered members. After you create your account, you'll be able to customize options and access all our 15,000 new posts/day with fewer ads.
And you would also be able to do OTHER things like write papers, participate in discussions, etc. in order to show me your mastery of the material. (And incidentally, even for quizzes and exams, I often allow what I call an "earned curve" assignment, in which the students go back to the original source material, find the correct answers, and explain why they're correct. I offer this assignment because the whole point is for the students to learn the material, and this is one way of giving them a second chance to do just that.)
