The problem with standardized testing (SAT, university, elementary school, quantitative)

[nana053]

This is specific to the FCAT in science in Florida, but it seems to me that it is endemic in terms of the way these tests are made. Robert Krampf who wrote this piece is a science teacher who travels and gives demonstrations in many museums and other venues.

Problems with Florida's Science FCAT Test? | The Happy Scientist

according to the Test Development Center, it appears that it is acceptable to use scientifically correct answers for wrong responses on the Science FCAT as long as FLDOE does not expect a fifth grader to be educated enough to realize that the wrong answers are scientifically correct.

[jtab4994]

It is frightening that the designers of the test assume no 5th grader would possibly know the correct answers, and therefore they created a sloppy test. Unbelievable. I wonder if they intentionally designed it so below-average students would score high?

[marigolds6]

These types of questions arise out of two standard test requirements:
That the correct answer be the "best" answer and that wrong answers are "attractive distracters".
In other words, the wrong answers must be answers that a student would select if they are using information inconsistent with the standards. You cannot just have blatantly wrong answers, but rather must have a justification as to why the student would select that as an answer.
As a result, using a scientifically correct answer that is not the "best" answer according to the standards is the best way to design a test question to fit the design standards.

But, there are other design standards that the test ignores, like question completeness. A question prompt should be complete and answerable without reading the answer stems. This is standard is part of the reason the SAT eliminated the infamous quantitative comparison questions.
Questions should also mimic classroom learning methods (the other nail in the coffin for quantitative comparisons), which clearly these questions do not do. There are other issues too. Try reading the observation question without the actual question. You can easily pick out the answer without knowing the question because the correct answer stem is structured so differently.

[jtab4994]

Apparently then, part of the reason American students lag behind in science is tests like this, which ignore "right" and "wrong" answers, forcing 5th grade students to decide which of several correct answers is "best". How is a 5th grade student who recognizes a number of correct answers to a question, supposed to make that judgement?

In one example given in the article, a student would be "wrong" is they said the softness of rose petals can be determined scientifically. That's actually true, but the test designers thought 5th graders would not know that. While most probably don;t know that, what about the bright youngster who does? Why penalize intelligence? Especially on a science test!

[marigolds6]

It is not that they did not 'think' 5th graders would know that.

The curriculum standards specify that 5th graders would not know that yet. People mistakenly assume that tests find if a student is at or above grade level. They only determine if the student is at grade level. A student above grade level will run into significant problems on a curriculum standards based test.

[HappyTexan]

That's like asking about the square root of a negative number. If you haven't taken higher level math then the answer is no, you cannot take the root of a negative number.

Then you learn about imaginary numbers and that answer becomes yes, yes you can and the answer is "i".

Once you learn that you can never, in your logical thinking, say no again.

That one got me when I went back to school to take refresher courses. When everyone else said "no", I said "yes".
Then the teacher told me "But we didn't learn it yet here so the answer is no."

[Fun Value]

I think it is a poorly made test. Ambiguous questions can and should be fixed.
I suspect there's enough politics involved so many want a faulty test.
A multiple choice test is easy to grade and cheap, but should be accompanied with essays.

[jtab4994]

Originally Posted by marigolds6

It is not that they did not 'think' 5th graders would know that.

The curriculum standards specify that 5th graders would not know that yet.

(emphasis added)

Sounds Orwellian to me. I take it personally because this kind of thing used to happen to me in my early school days, and I am afraid it will never change.

[jtab4994]

Originally Posted by HappyTexan

That's like asking about the square root of a negative number. If you haven't taken higher level math then the answer is no, you cannot take the root of a negative number.

Then you learn about imaginary numbers and that answer becomes yes, yes you can and the answer is "i".

Once you learn that you can never, in your logical thinking, say no again.

That one got me when I went back to school to take refresher courses. When everyone else said "no", I said "yes".
Then the teacher told me "But we didn't learn it yet here so the answer is no."

The teacher could have said that the course only deals with Real numbers. Then you wouldn't have to "go along to get along". It's like teaching a Spanish class and specifying that Castilian spelling, pronunciation, and grammar would be used, so a person with knowledge of a South American dialect would know what's expected of them.

[compSciGuy]

This is an issue with the test designers not the specific format of the test. I've had plenty of crappy graders and wrong answers from elementary school through university (including science related coursework). I learned before 5th grade you need to give the teacher/grader what they what and not the best or correct answer. I've also heard and been told plenty of times, do it the way it's done in this class via the instructor, not the book, not some other related class (you have or will take), not some way that's from more advanced material, etc. Granted there are a few teachers wanting the best answers, but those are in the minority and cannot (and should not) be assumed for some test where you don't know who's grading it.