Please register to participate in our discussions with 2 million other members - it's free and quick! Some forums can only be seen by registered members. After you create your account, you'll be able to customize options and access all our 15,000 new posts/day with fewer ads.
Yes, developing a syllabus for an outside organization's review (it's just for me and doesn't affect my job in any way -- I get a stipend for doing it). I've developed literally dozens of syllabi on classes from first-year survey courses to final-year capstone courses and they've gotten high marks for clarity, choice of readings, variety of assessment tools, depth, breadth, etc. from faculty colleagues and administrators.
Your suggestion made me chuckle, because I know the reviewers would hate it because again you're using that bad word "understand."
I think what it comes down to is that this outside organization asks me to have goals/objectives and assessment items (like quizzes, a paper assignment, or whatever) for everything I want the the students to do. But for testing CONTENT KNOWLEDGE AND UNDERSTANDING (i.e., not testing deeper understanding), a multiple-choice quiz should be fine.
The outside reviewer keeps saying that I need to say what the students will be able to DO in each module. But the next sentence tells me that saying the student will answer questions on a quiz, incorporate terms into a discussion post, or whatever -- in other words, saying what students will DO -- is NOT telling the reviewer what the student will do, it's simply giving an "activity" that the student will do.
So again, I clearly don't have the jargon down right, as "do" apparently means something particular in assessment-speak -- but she didn't tell me what that WAS.
...
My next question is totally a side bar but are you developing a training program for a government agency or DoD? Only reason I ask is I was one of the SMEs on such a project a few years ago and the bizarreness you describe fits what we went through to a T. There was constant conflict between the SMEs, the developers, and the assessment folks over the same issues you describe. We never really resolved it.
My next question is totally a side bar but are you developing a training program for a government agency or DoD? Only reason I ask is I was one of the SMEs on such a project a few years ago and the bizarreness you describe fits what we went through to a T. There was constant conflict between the SMEs, the developers, and the assessment folks over the same issues you describe. We never really resolved it.
Interesting parallels, but no, mine is strictly for my own professional development, it's just reviewed by an outside organization and I get a stipend for doing it (which of course I want!). I actually like THINKING about these things because I really care about my teaching, but when the reviewers are this confusing and simply cannot give me a straight answer as to why "understand" is confusing or what "do" means, it's frustrating!
There's nothing wrong with a multiple choice quiz, that's not the actual issue. A good lesson has a clear goal. The students come from a position of not being able to do something, such as calculating the circumference of a circle, and by the end of the lesson the student is able to calculate the circumference of a circle.
Part of the problem with your learning objective is your lesson seems somewhat muddled. Reading a book and answering some basic factual questions about the book. Where does the teaching part come in? What did the students not know how to do before you taught them the lesson? If they aren't learning anything, then it's not really a lesson. For example, many schools have independent reading time and students may answer some questions about what they read. I think it's great to give students reading time, but this is not an actual lesson, it doesn't require a teacher, the students aren't learning anything new.
Going back to my circumference example, a poor learning objective would be "the students will understand circumference." I think anyone would have the same question, what about circumference will they understand? What will they be able to do? A specific, measurable learning objective would be "When given the diameter or radius of a circle, students will be able to calculate its circumference."
Which statement is measurable? A student "understands circumference" or a student can "correctly calculate the circumference of a circle when given its diameter."
A separate question would be, how do you know when a student is successful or has mastered the learning objective? That would be where your 8 out of 10 questions answered correctly would come in.
There's nothing wrong with a multiple choice quiz, that's not the actual issue. A good lesson has a clear goal. The students come from a position of not being able to do something, such as calculating the circumference of a circle, and by the end of the lesson the student is able to calculate the circumference of a circle.
Part of the problem with your learning objective is your lesson seems somewhat muddled. Reading a book and answering some basic factual questions about the book. Where does the teaching part come in? What did the students not know how to do before you taught them the lesson? If they aren't learning anything, then it's not really a lesson. For example, many schools have independent reading time and students may answer some questions about what they read. I think it's great to give students reading time, but this is not an actual lesson, it doesn't require a teacher, the students aren't learning anything new.
The BASIC questions are simply to test WHETHER THEY READ (and whether they "got," i.e. UNDERSTOOD, the author's argument at the most basic level). I don't know if you teach at a college (or have), but it is a MAJOR undertaking to simply get students to do the most basic amount of reading. So yes, some quiz questions DO test ONLY whether they read or not. It's definitely NOT higher-level learning and it's also, incidentally, NOT the type of teaching I'd like to be doing -- but it's the reality.
In other words, if they have not even READ the foundational article that I asked them to read, then there is no way they could do anything further -- i.e., they can't "explain" or "compare/contrast" or any of those other things I'd like them to be able to do. So yes, some assessments HAVE TO BE at that basic a level. And I wish they didn't have to be, but alas, I live in the real world.
Quote:
Originally Posted by TXRunner
Going back to my circumference example, a poor learning objective would be "the students will understand circumference." I think anyone would have the same question, what about circumference will they understand? What will they be able to do? A specific, measurable learning objective would be "When given the diameter or radius of a circle, students will be able to calculate its circumference."
Unfortunately, that kind of language differentiating doesn't really work for my field (although I can see that it may very well work for YOURS). "Understand an argument in an article" seems to me to be very different from "understand circumference." Does that make sense?
Incidentally, I heard from my college's assessment person today and we've set up a Zoom meeting for Thursday. I think he totally "got" my confusion from the outside reviewers, because he basically said, yes, this language doesn't make sense at first. So I think it's just something I have to learn. We'll be talking more about it in a couple of days, and I am actually looking forward to that conversation.
No college teaching here I hear what you are saying, I do teach math, so I have a better understanding of strong learning objectives with something I've taught. I've taught kids in grades 2-9, from struggling learners with significant challenges to advanced students.
I don't mean to sound hard on your lesson, but reading a book is not a lesson by itself. For it to be a lesson, you have to teach something. If they read a book about Sally with blonde hair, and the question is just what's the name of the main character or what color is her hair, that's just memorizing facts. There's really not a teaching aspect to it, unless you are giving them specific strategies to help remember those facts.
You could build a lesson about using effective reading strategies by teaching them a strategy and then having them apply the strategy to a selected text. If you didn't teach them anything though, then it's not a lesson. I'm not a reading teacher, but I have worked with elementary school reading teachers and the focus we had was on summarizing a text, predicting what might happen next or what characters were feeling, and decoding the meaning of unknown words.
We also had independent reading where students would answer basic recall facts about a book in order to get points through the Accelerated Reader program. However, that's wasn't a lesson by itself. Again, I'm not trying to sound harsh, but you have to teach them something for it to be a lesson. Nobody would call me to come observe a lesson and rate them as a teacher where students just silently read an article and then independently took a quiz over the facts of the article. There wouldn't be a lesson or any teaching to observe. The students could just as easily do those tasks at home.
The BASIC questions are simply to test WHETHER THEY READ (and whether they "got," i.e. UNDERSTOOD, the author's argument at the most basic level). I don't know if you teach at a college (or have), but it is a MAJOR undertaking to simply get students to do the most basic amount of reading. So yes, some quiz questions DO test ONLY whether they read or not. It's definitely NOT higher-level learning and it's also, incidentally, NOT the type of teaching I'd like to be doing -- but it's the reality.
In other words, if they have not even READ the foundational article that I asked them to read, then there is no way they could do anything further -- i.e., they can't "explain" or "compare/contrast" or any of those other things I'd like them to be able to do. So yes, some assessments HAVE TO BE at that basic a level. And I wish they didn't have to be, but alas, I live in the real world.
Unfortunately, that kind of language differentiating doesn't really work for my field (although I can see that it may very well work for YOURS). "Understand an argument in an article" seems to me to be very different from "understand circumference." Does that make sense?
Incidentally, I heard from my college's assessment person today and we've set up a Zoom meeting for Thursday. I think he totally "got" my confusion from the outside reviewers, because he basically said, yes, this language doesn't make sense at first. So I think it's just something I have to learn. We'll be talking more about it in a couple of days, and I am actually looking forward to that conversation.
Could you still use that wording by saying “Demonstrate understanding by doing X, Y, and Z?”
For example, when I taught ESL abroad, the kids had to memorize 500 English sentences. It was the same 500 for all three years of HS. They just blurted out the week’s series of 10 or 12 sentences, without having to show they understood a lick of it. I think they had other activities to show the read and memorized them.
I know in the past I’ve had professors require discussion questions be submitted by students to show they read the material. Perhaps they did not fully understand it, but at least then the discussion questions would help the professor know where people were struggling or what aspects of the reading students wanted to discuss. In the one class where it was used, the topic was super political, so it was not a situation where people could just take others’ questions. In that class we were expected to submit our discussions and contribute to the discussion in each class. That was how we demonstrated our understanding, as the final project was a paper on a topic of our choice.
Unfortunately, that kind of language differentiating doesn't really work for my field (although I can see that it may very well work for YOURS). "Understand an argument in an article" seems to me to be very different from "understand circumference." Does that make sense?
...
Coming from an outside perspective, I see no difference between "understand an argument in an article" vs "understand circumference." Standing alone neither tells me, the student what I have to do to understand. One of the things I have to do to understand circumference is know how to calculate it from the radius. In that perspective what do I, as a student, have to do to demonstrate understanding of a argument in an article?
It could be that "What is that action a student has to do to demonstrate understanding?" that they are looking for.
No college teaching here I hear what you are saying, I do teach math, so I have a better understanding of strong learning objectives with something I've taught. I've taught kids in grades 2-9, from struggling learners with significant challenges to advanced students.
I don't mean to sound hard on your lesson, but reading a book is not a lesson by itself. For it to be a lesson, you have to teach something. If they read a book about Sally with blonde hair, and the question is just what's the name of the main character or what color is her hair, that's just memorizing facts. There's really not a teaching aspect to it, unless you are giving them specific strategies to help remember those facts.
You could build a lesson about using effective reading strategies by teaching them a strategy and then having them apply the strategy to a selected text. If you didn't teach them anything though, then it's not a lesson. I'm not a reading teacher, but I have worked with elementary school reading teachers and the focus we had was on summarizing a text, predicting what might happen next or what characters were feeling, and decoding the meaning of unknown words.
We also had independent reading where students would answer basic recall facts about a book in order to get points through the Accelerated Reader program. However, that's wasn't a lesson by itself. Again, I'm not trying to sound harsh, but you have to teach them something for it to be a lesson. Nobody would call me to come observe a lesson and rate them as a teacher where students just silently read an article and then independently took a quiz over the facts of the article. There wouldn't be a lesson or any teaching to observe. The students could just as easily do those tasks at home.
You ARE sounding harsh but I think you just misunderstood (which likely means I wasn't clear enough!). I teach at the college level. And of course reading a book isn't a "lesson" -- it's HOMEWORK, and the quiz is simply meant to test them on whether they read it or not. The TEACHING comes in the actual class during which I might give a very short lecture (10-15 minutes) that's related to the readings (and includes terms/concepts/theories/ideas from our discipline, with lots of examples), then we discuss the readings, typically watch a related video or do a related activity, then make connections between/among everything. My classes tend to be extremely interactive.
For the outside review, we were asked to state what students would be able to "do"* after each unit. Hence, my "demonstrate your understanding of the reading you did by answering these questions." But that doesn't work for what the reviewers meant by "do."*
(*See my next answer!)
Quote:
Originally Posted by tnff
It could be that "What is that action a student has to do to demonstrate understanding?" that they are looking for.
No, they actually DON'T want that! Hence my confusion.
Quote:
Originally Posted by RamenAddict
Could you still use that wording by saying “Demonstrate understanding by doing X, Y, and Z?”
Ironically, that is actually how I wrote a lot of the goals/objectives -- but the answer is no if the "doing" is something like posting to a discussion board, taking a quiz, or whatever. Those are considered ACTIVITIES, they are not considered "doing" something by the outside reviewers. (Now can you understand why I find the assessment language they're using very confusing? They say "tell us what students will be able to do" but then they say "don't tell us about activities they will do." Hopefully I'll have a better understanding of the various meanings of "do" after my meeting with my college's person on Thursday! )
Hi Karen- any updates on your meeting to enlighten us as to what the college person wanted from you?
Yes! I met with our assessment person today via Zoom, and I'd given him the email from the outside reviewers in advance, and he thought that some of their critiques were a bit odd too, especially when it came to being very very picky about wording. (In other words, he would have "passed" a lot of the modules' objectives, but alas, it's not up to him.)
EXAMPLE: "Students will be able to explain the 3 kinds of blah blah blah from the handout." WRONG!! Literally, the fact that I included "from the handout" is apparently very bad. If I take off the words "from the handout," then it's fine. The issue seems to be that including "on the handout" is being too specific about what they're doing (in the usual sense of the word "doing") for that unit; the objective is supposed to be broader, which basically means more vague, even though it has to ALIGN (that's a big assessment word, and I totally get that) with the assessment instrument, i.e. the quiz or paper or whatever. (But this is one that made our assessment guy kind of shake his head since he didn't really see a big problem with including the words "from the handout," since that IS where the students would be getting the info.)
ANOTHER EXAMPLE: "Students will describe x, y, and z in a discussion posting." Again, WRONG!!! This time, it's the fact that I included "in a discussion posting" -- even though that's where they'll be doing the describing. Nope, not supposed to SAY that, apparently, at least according to the outside reviewers, who kept saying "written as an activity." Again, our own assessment guy thought that was being way too nit-picky too.
Now, SOME of the critiques he agreed with simply because "assessment lingo" DOES NOT ALLOW THEM -- like saying that the student will "understand" something. He actually said that the example I gave (of multiple-choice questions simply testing a student's understanding of an argument from a reading) shows that "understanding" (in the usual sense of the word) IS measurable, but the problem is that it's not directly OBSERVABLE (he thinks that's a better word to conceptualize it, and it DOES make more sense to me), meaning we can't observe the student's brain doing the understanding. So, OK, at least I can understand that (ha! there's that word "understand" again!) although I think it's kind of ridiculous.
So our assessment guy basically said "change 'demonstrate their understanding' to 'explain' or 'list' or 'define' or whatever" -- even though the quiz questions don't really ask students to "explain" or "list" or whatever. Basically, he said that if the reviewers can see that OVERALL, in, say, a 10-question quiz, students ARE in a sense "explaining" or "defining" or whatever when you look at the WHOLE of the quiz, then it should be OK.
Again, I do somewhat understand what he's saying (and at least he went beyond saying "just do it, Karen, don't ask questions") -- but I was thinking, "You've got to be kidding" through some of our meeting, and I expressed that to him, and he totally understood. (I am laughing again ... could we MEASURE his understanding? Apparently not. But I felt it. )
In any case, I got the $2,000 stipend for developing the course and submitting it, which is very nice, and I can get another $2,000 stipend if I make the "corrections" that the outside reviewers asked for, but frankly, I don't know if I will have time to do them, which will be a shame. (The rules were kind of changed in the middle of the game, which of course added to my frustration -- my college really wanted SYNCHRONOUS online classes, which are mostly taught via Zoom and are, at least for me, about 95% the same as my in-person, on-campus classes. But the outside reviewers wanted ASYNCHRONOUS classes -- i.e., no class meetings, just instructions for the students to do certain things over the course of a week or two weeks or whatever. But the outside reviewers didn't TELL us that the course had to be ASYNCHRONOUS, so most of us developed synchronous classes and then had to re-do them for the outside reviewers. So I basically had to create my class 2 ways, which was as much of a pain as it sounds like -- and I say that as someone who LOVES doing new preps even after >20 years of teaching and has probably done more new preps than anyone else at my college. But this was all a bit much.)
So there's the update, and thanks for asking. Does that make sense? Would you like to take a quiz?
Last edited by karen_in_nh_2012; 02-18-2021 at 06:09 PM..
Please register to post and access all features of our very popular forum. It is free and quick. Over $68,000 in prizes has already been given out to active posters on our forum. Additional giveaways are planned.
Detailed information about all U.S. cities, counties, and zip codes on our site: City-data.com.
Please register to participate in our discussions with 2 million other members - it's free and quick! Some forums can only be seen by registered members. After you create your account, you'll be able to customize options and access all our 15,000 new posts/day with fewer ads.
So, OK, at least I can understand that (ha! there's that word "understand" again!) although I think it's kind of ridiculous.
