A Goldilocks analysis of the Physics RTQs

Each year since 2003, the California Department of Education has released 15 questions from the physics CST (California Standards Test). The growing collection is known as the Physics RTQs (Released Test Questions). One RTQ was removed from the set for bad behavior. It deserved it. So as of 2009, we have 89 physics RTQs.

There are 45 content standards in California's 9-12 Physics. How well do the RTQs cover the standards?

One might think that once the number of RTQs reached 45 (in 2006), all the content standards would have been covered. But the reality of selecting RTQs is not so simple. The process is necessarily complex. As a result, some standards were represented with multiple RTQs while others went uncovered. (Remember, too, that the Investigation and Experimentation Standard Set must also be represented in the RTQs.)

As of this year's RTQ set--including items from the 2008 test--all 45 physics content standards are covered.

One hopes that future RTQ releases will fortify weakly-represented standards and increase the variety of item types in the set.

There are many things one can do when not burdened with spouse or children. I will not burden this post with an exhaustive list. But one task that falls under the aegis is my thorough

Physics RTQ Distribution Analysis PDF XLS.

This gem is a spreadsheet listing all the RTQ items by year of release and (more importantly) by standard. The actual items are in the RTQ PDF from the CDE; the spreadsheet merely shows their distribution. Go ahead, click and open it. I dare ya! (PDF or Excel.)

This representation makes it easy to perform a Goldilocks analysis of the RTQs. That is, you can see which standards are over-represented, which ones are under-represented, and which ones are represented just right.

You really should open it and marvel at the OCD that was required to prepare it. And, of course, thank your lucky stars that you do not suffer such a level. But for the faint-of-anal-retention, I offer this:

2009 Executive Summary of the Physics RTQ Distribution
1. All Standard Sets are represented in proportion to the test blueprint.

2. Most individual standards are represented in balanced proportion.

3. The following individual standards are over-represented in the RTQs. Future RTQs sets should be selected in a manner that will exclude new items from Standards
1.A, 1.B, 1.G, 2.A, 3.C, 4.C, and 5.B.

4. The following individual standards are under-represented in the RTQs. Future RTQs sets should be selected in a manner that will include new items from these standards.
a. Primary needs lie in Standards 2.F, 4.F, 5.D, 5.E, 5.F.
b. Secondary needs lie in Standards 2.E, 3.B, 3.E, 4.A, 4.D, 4.E, 5.H.

For those who haven't memorized the standards by their alphanumerical designations, a truck will be around later to bring you to re-education camp. In the meantime, you can check the Blueprint. Spoiler alert: Set 1 is Motion and Forces, 2 is Conservation of Energy and Momentum, 3 is Heat and Thermodynamics, 4 is Waves, and 5 is Electric and Magnetic Phenomena. See the blueprint for the letters.