The new Physics Released Test Question set was posted yesterday on CDE's STAR resources page.
The typographical issues (mentioned in the previous post) have been repaired.
Each year's RTQ set includes the previous year's set with 15 new items blended in. The new items come from the previous year's test (administered the previous Spring). The 15 new items are located as follows:
Investigation and Experimentation: 8
Motion and Forces: 14, 20, 25
Conservation of Energy and Momentum: 29, 35, 41, 44
Heat and Thermodynamics: 47, 53, 56
Waves: 63, 71
Electric and Magnetic Phenomena: 81, 87
The total item count in the Physics RTQs now stands at 89. It should be 90, but one RTQ was removed from the set in 2008. The removed item was authored in the very early days of the CST and was not up to today's standard of item quality. So it was discarded.
I wouldn't be a proper curmudgeon if I didn't have at least a minor quibble. While RTQs have been released each year for several years now, it seems each new year brings a new file organization structure at cde.ca.gov. That is, one cannot hope that this year's RTQs will have the same URL as last year's. The new document is intended to replace the old document, yet a new file path is created each year. It could very well be that I am the solitary Californian whose style is cramped by this annual reorganization.
As always, let me know what you think.
well, immediately I see that question #2 has more than one correct answer since either a motion sensor or accelerometer will work, question #4 also has more than one potentially correct answer since with no scale it is almost impossible to tell if the objects are accelerating or near terminal velocity. It's also depressing that many of the questions are the "Do you want the Physics answer, or the real answer" since magically skateboards and rail cars have no friction, rotational inertia, and in some cases mass. Hmm, 62 also has more than one correct answer (since a wave packet is a superposition), 68 is wrong since vibrations from a landslide would be transmitted through the ground.
ReplyDeleteAh well, at least Pennsylvania is not alone in having poor science assessment test questions.
#2. I'll admit I've never seen a low-g accelerometer used to generate x vs. t and v vs. t graphs in real time. If you've seen this done, please send me a link. Remember, you *could* measure the height of a tall building using a barometer, but the barometer might not be the "most appropriate" tool for the task. The standard in question demands students be able to identify the best tool for the job.
ReplyDelete#4. The item asks for the *best* explanation, and A is clearly the best explanation given the stem/prompt.
#62. A couple of things. First, I'll give you a dollar form every HS physics student who knows what a wave packet is if you'll give me a penny for each one who doesn't. Second, I really don't see/hear the term "superpositioned wave" in my travels through physics terminology. I see terms like "superposition." "superposed," "interference," "beats," and so on. I've seen mention of reflected waves, refracted waves, diffracted waves, and polarized waves. But not superposed waves. Me not seeing it/hearing of it doesn't mean it's not out there. But I doubt any student got this one wrong because they thought the stem better described a superposed wave packet than it did a longitudinal wave.
#68. Seismic vibrations are all good fun, but the item asks about *hearing* the landslide (moonslide?). The astronauts might feel the vibrations through their boots, but they won't hear it.
But on to more important matters: where are Pennsylvania's physics RTQs? Or any other state's for that matter. Questions intended for every student enrolled in high school physics statewide.
Seems like California's the only state putting items out there. Am I not looking hard enough?
As always, keep in mind these items are not written in a vacuum. Each one must specifically address one and only one content standard.
Actually New York has been releasing entire exams for years: http://www.nysedregents.org/testing/scire/regentphys.html I've been known to "borrow" questions for quizzes and exams.
ReplyDeleteFor #2, you need to have a circuit to integrate using an accelerometer to find velocity, and integrate that to find displacement. This is most commonly done when the sensor has to be mounted on the moving object (e.g. missiles, aircraft) a quick google search brings up http://cgm.engineering.ucdavis.edu/Publications/signal.htm if you want more gory details.
I know #4 asks for 'best' but including a reference object and time scale would not detract from the question in any way and would certainly make a "correct" answer more clear.
As for #62, you dodge the issue. It's like the "When a wave passes from a material of one speed of propagation to another at an angle it is:" and the "correct" answer is supposed to be Refracted, never mind that is also guaranteed to be (at least partially) reflected. It is easy to write questions where the incorrect answers are actually incorrect, that's why standardized test questions are supposed to be vetted. PA's science assessment test (we don't have one for Physics or any science subject area, just "Science" which is in fact as ridiculous as it sounds) was so awful last year that the contract with the company writing it was terminated in the first year, so only finding a couple to quibble with on California's is not bad.
I'd still argue with you about #68. If the frequencies were in audible range, vibrations through the feet (and body) might be heard, and the suit would be a gas filled cavity. Do you think that they'd hear something if they kicked something, or with the famous golf swing? I've never read anything one way or the other, but I'm curious.
Fair enough.
ReplyDeleteRegarding the NY Regents Exam: I was under the impression that that course and exam was a cut above the common physics course. Like an "honors" course.
Was I mistaken? Is the Regents taken by all physics students statewide? I'll toss in a caveat for California's "statewide" administration of its Content Standards Tests: we only test high school students through grade 11. Seniors are not assessed for anything.
And make no mistake, I have issues with a lot of what California's test contractor offers up for the Physics CST.
But you and I both know that there are precious few perfect physics multiple choice items, especially those constrained to assess a single issue and intended for a statewide population.
Round up any 100 questions and distribute them to physics teachers at any regional AAPT meeting. I doubt there would be a dozen that the attending teachers would agree were great items.
We're a tough group to please. And there's nothing wrong with that.
For better or worse, all the questions used on California CSTs are vetted to a certain extent. An Assessment Review Panel advises the CDE on which questions to take and which ones to reject. The ARP includes content-area experts from primary, secondary, and post-secondary education and from industry. The process is thorough, but it is not perfect.
One thing I will say about the items that make it through to the operational form (and then to the RTQ sets) is that they pass through content vetting and field test statistical vetting. Selected items were answered correctly by a sufficient number of students. And were more likely to be answered correctly by students who did well on the overall test while being missed by students who did poorly overall.
The hope is that we get better items as the years go by.
Honestly I'm not sure what the situation is with the Regents, but in most places Physics is at least an elective, usually higher level course (despite most states having some sort of Physics standards allegedly for all students).
ReplyDeleteI'd agree that you'd be lucky to find a dozen multiple choice questions that would pass, and I'd bet that at least half of those would be vector diagrams or free body drawings.
I understand the difficulty of testing a single topic or standard by a question, I just wonder how much of any assessment that should be. Personally I assess using a mixture of multiple choice, short answer: explain what will happen in a scenario, why things are done the way that they are (e.g. bearings go on the inside of a wheel, brakes further out), or how to solve a real-world problem, and "Show your work" numeric problems. Assessment of that type would be prohibitively expensive as a standardized test, so instead we only test the way that's easy to grade? Seems to be a poor solution to me. We (in PA) are willing to use classroom teachers to grade writing assessments by a rubric, but I've never heard of a similar scheme even being proposed for science assessments.
I'm sorry if I come off as negative, but I've just been dealing with the fallout from PA's miserable science assessment (the state's spun it as "Poor results" from the students, never mind that they canceled DRC's contract for the tests, for the very sound reason that the test was awful), and that the Physics standards are beyond a mess (the Forces section specifically mentions curved mirrors, but not Gravity as just one example). By comparison CA looks like heaven...I've just been getting a lot of practice at shooting at bad questions, assessments, and standards.
I can't recall which Blog first directed me to yours (Swans on Tea, perhaps?) but I'm enjoying reading it. Thanks for sharing with the community.
No worries. Neither you nor I will let statewide standards or assessments wreck otherwise good classroom instruction.
ReplyDeleteAnd welcome to the RTQ-bash bash. It was much more vociferous in the early days of RTQ releases. But it's always good sport.
Be careful, though. If you're not, they'll stick you on some kind of Assessment Review Panel so that you're the one getting shot down. That's what they did to me. Not as much fun as throwing stones from the outside, but important nonetheless.
Feel free to criticize any item you like (or dislike, I suppose). I'm happy to get the feedback, and I won't always disagree.
Some stinkers make it through. But you should see the stuff we turned away!
I have looked at the released materials. The questions are pretty basic. They are also pretty poorly written.
ReplyDeleteI cannot really see the point of these silly tests continuing in the 21st Century.
Anonymous above,
ReplyDeletePlease feel free to send me your specific criticisms. And, as always, if there's another state doing a better job than California, please point me in that direction.
I have yet to find--anywhere--a better set of physics RTQs. Remember, each question must assess one (and only one) of California's 45 physics content standards. And these questions must be answered by every non-senior in a California public school enrolled in a physics class. The questions pass through many filters before they're included in an operational form.
And what is the basis for your belief that these tests are going away? Obama's new Secretary of Education stands in firm support of testing.