Test Correction Mastermind

I clearly wanted to turn assessments (summative assessments, mind you) into learning tools, as evidenced by Test Correction Journals (see previous post). But I wanted to do it differently in my AP Physics classes. Some students in AP Physics 2 had been in Physics, so TCJs were old hat to them.

So I developed Test Correction Mastermind for my AP students. The broad goals are the same: get students talking to each other about test items with the possibility of some sugar (points) at the other end if they got the questions sorted.

As was the case for TCJs, this activity is done well after the test has been administered in class and the make-up window has closed. This time, I give each lab group (3 or 4 students) a blank BairdTron and a copy of the test. Again there are 4 forms and I give different groups different forms. The questions are all the same, but scrambled differently. Different forms in adjacent groups minimizes the value of conversations overheard between neighboring groups. 

The objective is to complete the test as if it were a group activity. But each group is in competition with all the other groups to get the top score (20 multiple choice items on the AP unit tests, and one free response—the free response isn't part of the TCM).

The reward structure is as follows: each member of the group gets a point for every correct answer above 15 (P=score–15). The top-scoring team members get 4 more points on top of that; second place folks each get 2. At most, a student could add 9 points to their test score. Among AP students in whom competitiveness is sometimes strongly expressed, 9 points is plenty.

When about 20 minutes of class time remain (57-minute periods), I rattle my thunder drum and announce the "The oracle is now open for 5 minutes!" Group members submit their BairdTron for scoring and return to their group. I tally up the number of incorrect answers, write that number on their form, and call them back to retrieve the scored document. 

I do not indicate which items are incorrect, only the total number of items they answered incorrectly. 

Test Correction Journal

Assessments: quizzes, tests, exams. There is a big part of the teacher's soul that hates these things. We love our subject matter and love sharing it with our students. We love it when our students are engaged with our subject matter and learning.

Tests? What a chore. You need to construct them. You need to administer them, You need to grade them. Yuck. Yuck. And yuck.

Construction: what should be assessed and how? Is that question too hard or too easy? Administration: to what extent can cheating be prevented and how much energy is that going to cost? Grading: do you hate multiple choice or do you hate hours upon hours of grading?

To students, assessments may seem like nothing other than punishment.

Of late, there has been much examination of grading. Anything smacking of "traditional" is guaranteed to be on blast in Teacher Twitter. Some teachers have gone all in on Standards Based Grading and are eager to evangelize that movement. Others are all for ungrading, and are keen to share The Good News about that with anyone who will listen.

Having given my first assessments in 1986, I grant myself a modicum of "old curmudgeon" license. I don't recall much talk about SBG or ungrading on Twitter back then, mostly because there was no Twitter. Web 1.0 was still in the future (it exploded in 1995). If it was talked about at the conferences (national or local AAPT), those conversations eluded me. Should I have jumped on the SBG or ungrading bandwagons as soon as I heard from proponents? Maybe. My own district was pushing Myron Dueck's Learning Targets hard.

My aversion to going with anything that I deem to be trending may well be a character flaw, and I own it. I was very late to listening to anything by Norah Jones because her album was plastered all over the record stores. (Remember record stores? Yes, I'm that old.) If she was that popular, she wasn't for me. I figured it out eventually.

In any case, here's what I did with Physics unit tests in the vacuum of my own classroom kingdom.

My district bans a Jewish surname...

...from emails sent to district-managed student accounts. Here's the story:

In COVID-era Distance Learning, I have found Socrative to be a useful platform for administering student assessments. Among its useful features is the ability to send students their own individual scored test, showing which questions they answered correctly and which ones they answered incorrectly. The results can be sent to student email accounts with the push of a button. [You are correct to assume the instructor had to enter those email addresses into Socrative. But it's a one-time investment.]

This is a great feature for when we do Test Correction Journals.

But in my very first attempt to leverage this feature, it failed. I had, of course, tested the feature before using it with students. And it worked. But when I sent my students their test results, students insisted they did not receive them. I tried it using Chrome, I tried it using Firefox. Nothing worked. I had to create a privacy-respecting, individualized way to tell students which specific test items they missed. I created one breakout room per student, joined each one—one at a time, and announced which items they missed. While class time was burning.

I quickly came up with a functional workaround for the next period, but it fell short of what I needed it to do. And why didn't the student email solution work? 

Students suggested that the district blocks third party emails from student accounts. The vendor, Socrative, meets all legal privacy standards. District tech services said they do not block third parties. Further investigation indicated the emails were blocked because they contained objectionable content. That offensive content appeared in the following question. TW: "Objectionable Content."

[Secrets of the Psychics] Psychologist and former palm-reader Ray Hyman found that he had the greatest success with clients when he  

A. gave a straight-up reading in accordance with palm-reading guidelines 

B. told clients the opposite of what he saw in their palms 

C. imagined that he lived in an earlier era 

D. assumed his clients were skeptical of palm-reading and his abilities

There it is: the objectionable content, clear as day. What's that? You didn't see it? Look again. "hymen" is right in there. Prurient anatomical terminology titillation that would offend any community's standards of propriety. High school students must be protected from such filth.

Okay, not "hymen" exactly, but "hyman," a simple misspelling of a highly salacious, practically pornographic word. Okay, not "hymen" or "hyman" exactly, but "Hyman," a not uncommon Jewish surname. I presume my college hall mate's name, Steve Hayman would trigger that comprehensive filter, too, lest bad actors use simple misspellings to skirt content filters. 

Some might judge the content filtering of Jewish surnames (that are not anatomical terms) as anti-semitic. I'm confident the San Juan Unified School District doesn't intend it that way, but here we are. "Ray Hyman" triggered a full and immediate IP ban due to the district's sweeping content blocking protocols. 

I was given no reason to hope that this filter would be removed by my district's tech services, now that they are aware of the embarrassing error. It seemed as if the onus was on me not to include terms that would trigger bans, and I should just know what all those terms are.

The filtering is clearly far too aggressive and highly impractical. Blocking my instructional program (without notification) for the "crime" of including a Jewish scholar's name? That's beyond a bad look.

It's indefensibly paranoid and ignorant, in my assessment.

UPDATE: This is apparently just another instance of The Scunthorpe Problem, or as Tom Scott calls it, The Peniston Problem. 

The CAASPP Practice items ... were online a year ago

The CAASPP / CAST practice / training / released test questions were apparently posted on Monday, February 24, 2020.

The hope I had nourished for them was in vain.

I had hoped for new items. Improved items. Robust items. But that's not what we got.

What we got last year's released items with newer publication dates and new graphics on the cover. If anything of substance has changed between last year's released items and this year's released items, someone will need to let me know in the comments.

CAASPP Practice Items (51—the same 51 released last year, so read about them here.)

CAASPP Training Items (7—All Life Science items. And all of them previously released.)

I've clearly been taking this new round of assessments far too seriously. When I'm wrong, I'm wrong.

CAASPP link added to sidebar...

UPDATE 2/20/20: With testing about one month away, the CAASPP/CAST practice and training items remain in "Coming Soon" status. Will we run through the administration of the tests before practice and training items are posted? Time will tell.

California Assessment of Student Performance and Progress is the part of the California Department of Education responsible for the statewide tests administered to students, such as the CAST (California Science Test).

They ostensibly publish training and practice exams ahead of the operational administration in March. As we close out January and head toward February, the training and practice exam status is as follows:

Scoring Guides for CAASPP: CAST, Smarter Balanced, CAA Practice Tests, and CSA

California Science Test (CAST)

Practice Tests

• CAST Practice Items Scoring Guide—Grade Five (Coming Soon)
• CAST Practice Items Scoring Guide—Grade Five, Braille (Coming Soon)
• CAST Practice Items Scoring Guide—Grade Eight (Coming Soon)
• CAST Practice Items Scoring Guide—Grade Eight, Braille (Coming Soon)
• CAST Practice Items Scoring Guide—High School (Coming Soon)
• CAST Practice Items Scoring Guide—High School, Braille (Coming Soon)

Training Tests

• CAST Training Items Scoring Guide—Grade Five (Coming Soon)
• CAST Training Items Scoring Guide—Grade Five, Braille (Coming Soon)
• CAST Training Items Scoring Guide—Grade Eight (Coming Soon)
• CAST Training Items Scoring Guide—Grade Eight, Braille (Coming Soon)
• CAST Training Items Scoring Guide—High School (Coming Soon)
• CAST Training Items Scoring Guide—High School, Braille (Coming Soon)

I added a permanent link in the sidebar to the right to make it easy to check in and determine CAASPP's definition of "Coming Soon").

The administration is, once again, scheduled for March. Fingers crossed that these items will post prior to that administration.

CAST Reference Sheet: The Good, the Meh, the Bad, and the Wha?

With the NGSS-based California Science Test (CAST) set for its maiden administration this spring, a reference sheet has been prepared. Students have access to the reference sheet while they take the exam. Here's the reference sheet.

CAST High School Reference Sheet

Reference sheet development can awaken passions that physics instructors didn't know they harbored. So let's go through it. (It might be helpful to print the sheet or have it open on another screen. Blood pressure medication? A preemptive dose might save a life here.)

FORCES AND ENERGY
Good: Newton's Second Law, the weight equation, universal gravitation, Coulomb's law, gravitational potential energy, kinetic energy, work-energy theorem. All good.

Meh: Using w for weight. A little bit non-standard, but the judges will allow it.

MOTION
Good: F∆t = m∆v, the use of ∆v in the equation for acceleration: the ∆ is critical here.

Meh: The use of ∆t rather than t. We never create problems involving isolated clock readings (t); intervals (∆t) are always given. The ∆ in front of the x in the equation for speed? I can take it or leave it.

Meh: The use of J for momentum. That's a rare one. I've seen it, but it is not common. I prefer ∆p, and I don't think I'm alone.

Bad: s for speed. I am reminded of a line from Star Trek: "A Vulcan would not cry out so." In this case, I will turn that phrase to "A physics teacher would never, ever use s to denote speed. Ever." I hope I'm not being ambiguous here. The very use of s for speed makes me wonder if anyone associated with physics instruction was involved in developing this reference sheet. A math teacher might make such a miscue, but not a physics teacher.

Wha?: Using the Greek letter, rho (ρ), for momentum. Why? Lowercase ρ is a symbol used to represent density in high school or college courses. It is also used to represent resistivity. Nobody ever uses ρ to represent momentum. Anywhere, ever! Momentum is represented with the letter p. Extra effort is required to insert the letter ρ. I am dumbfounded.

KEPLER'S LAWS
These are here for the benefit of items addressing Earth Science PE, HS-ESS1-4. I'll let them "Meh" or "Wha?" the notion that e = f/d is among Kepler's Laws.

WAVES
Good: The wave equation and the Planck-Einstein relation. I would have put them in that order, but okay.

Wha?: The reference sheet is set in a sans serif font. In high school, we use lowercase f for frequency. Someone at CAASPP seems to have found the special character, ƒ, which bears a striking resemblance to an italicized, lowercase f. If a pianist sees two of those in a row (ƒƒ), they'll hit the keys hard. The symbol, itself, is called "F with hook," or "florin symbol" and is used to denote the Dutch guilder, for example. This has been Sheldon Cooper's "Fun with Fonts" with your host, Sheldon Cooper.

EXPERIMENTAL DESIGN
Mostly good. It it's me, I'd use a cross (×) rather than a bullet (•) to indicate multiplication. The bullet really isn't a multiplication symbol. The dot (·) is, but the cross is more appropriate here.

At the risk of belaboring a point made in the post critiquing the practice items, I will point out to our gentle readers that no reference to index of refraction appears on the reference sheet. No n = c/v and (thankfully) no Snell's law. Recall that one of the practice items expected students to be familiar with the meaning of index of refraction.

CONSTANTS
If someone at CAASPP sees this, please explore the use of the dot symbol ·. On my Mac, I hold down  the shift and option keys while typing a 9. It's more professional than the big bullet • that you get with option+8. And drop the spaces in unit configurations.

I see N·m as better than N • m for newton-meters.

In all, the reference sheet is more good than meh, bad, or wha? But I wouldn't be keen to print it up and have students use it throughout the year. Replace the speed s with a v and the momentum rho with a p, and I can live with the remaining quirks.

CAST Practice—An Analysis of the Physics Questions

The California Department of Education's California Assessment of Student Performance and Progress program has released a set of High School Practice Items in connection to the state-mandated exams to be administered this spring.

As detailed in a previous post, six of the 50 released items relate to high school physics topics. Sixteen are from Life Science, 15 from Earth Science, 9 from chemistry, and 4 from Engineering Design.

Remember: chemistry and physics topics have been combined into the more omnibus "Physical Science" realm of the high school science Performance Expectations (PEs). While NGSS's HS-PS1: Matter and Its Interactions is predominantly chemistry, HS-PS2 Motion and Stability: Forces and Interactions, HS-PS3 Energy, and HS-PS4 Waves and Their Applications in Technologies for Information Transfer are predominantly physics PEs.

Nevertheless, 9 of the 15 released HS-PS items assessed HS-PS1 while 6 were shared among PS2, PS3, and PS4. If there is a blueprint available for the composition of the operational exam, I am not aware of it. So the Practice Items may or may not reflect the mix of the operational exams.

UPDATE: I found the blueprint: for the high school test, see page 8 of the CAST Blueprint. Not surprisingly, it appears the operational exam will include physics items in greater proportion than in the released practice items.

Let's take a look at those six. I will state each item's Item-Level Claim Statement (ILCS). Click the ILCS to see the actual item. My analysis will follow each ILCS.

---

Identify the relationship between mass and acceleration. [Click to see item.]
In general, the item is perfectly reasonable. It speaks to the interpretation of graphed data obtained through a laboratory activity that might be done in a high school setting. The item links solidly with the corresponding PE (HS-PS2-1: Analyze data to support the claim that Newton’s second law of motion describes the mathematical relationship among the net force on a macroscopic object, its mass, and its acceleration.).

But there is a problem. And I say this as someone who had the privilege of sifting through many, many potential exam items offered by ETS for use in state-mandated testing from 2003 to 2013. The problem is the title of the graph.

Classroom teachers come across as dictatorial joy-crushers when they make any attempt to direct their students to conform to established conventions. Titling a graph is but one such challenge. The title of a scientific graph is Dependent Variable vs. Independent Variable.

The graph on this item is correctly constructed as Acceleration vs. Mass: experimenters would have measured the acceleration of a cart while varying the cart's mass. But the graph is titled Mass vs. Acceleration. I have to assume this was an oversight on the part of ETS and anyone (if there was anyone) tasked with content review.

---

Mathematically determine the properties of the system using the conservation of momentum of objects in the system.
A fair enough item to assess the corresponding PE (HS-PS2.2: Use mathematical representations to support the claim that the total momentum of a system of objects is conserved when there is no net force on the system.).

Some might object that the mathematics is simplified too much in this item. It does leave open the question of how deep the arithmetic might go in operational items. If you make this question about a neutron and a deuteron combining to form a triton, the principle is same, but the numbers are more formidable.

In the event that anyone tries to champion the new assessments as ground-breakingly novel, consider this released test question from the old Academic Content Standards era: Conservation of Momentum RTQ. In fairness, the new one does have color. And kilograms, rather than tons!

But this brings up an ongoing problem suffered by standards exam item-writers. Standards (now PEs) are often important-sounding principles that turn out to be difficult to write a variety of items for.

---

Select the design solution that best meets the provided criteria about momentum and force during a collision.
This item reveals the challenges associated with writing questions for PEs like HS-PS2-3: Apply scientific and engineering ideas to design, evaluate, and refine a device that minimizes the force on a macroscopic object during a collision. The item has a very long stem leading to the answer choices. And the choices are quite word-heavy, too.

Students with a firm grasp of the fact that increasing impact time reduces impact force will find their way to the correct answer. The item will make them work for it, though. I'm not faulting anyone here: awkwardness is in the nature of writing items aligned to standards (PEs) like this.

---

Create a correct mathematical representation to determine the components of gravitational potential energy in the Earth-ball system and kinetic energy.
This one requires a written response and is graded with a rubric.

It is linked to HS-PS3-1: Create a computational model to calculate the change in the energy of one component in a system when the change in energy of the other component(s) and energy flows in and out of the system are known.

Might it connect more directly to HS-PS3-2: Develop and use models to illustrate that energy at the macroscopic scale can be accounted for as a combination of energy associated with the motions of particles (objects) and energy associated with the relative position of particles (objects)? The clarifications on PS3-2 say "Examples of phenomena ... could include the conversion of kinetic energy to ... energy stored due to position of an object above the earth. Examples of models could include diagrams, drawings, descriptions, ...]

Aligning items to PEs is sometimes a dark art; I'm inclined to grant CAASPP latitude on this matter.

I have reason to suspect Rhett Allain will not approve of dropping a tennis ball from the top of a building and stating that air resistance is negligible. Neglecting air resistance is a simplifying step. Dropping a tennis ball might be a requirement of item-writing guidelines that prohibit scenarios that might cause significant injury. Dropping a cannon ball would inherently render air resistance more negligible, but would also present a greater potential (!) for damage if carried out in real life.

There's some contrivance at work here: Determining the kinetic energy of a dropped ball in reality would involve measuring its mass and its speed. The speed, alone, allows one to determine the height from which the ball was dropped. But the standard demands items, so here we are.

---

Describe how wavelength is related to the change in the medium.
The PE being addressed here is HS-PS4-1: Use mathematical representations to support a claim regarding relationships among the frequency, wavelength, and speed of waves traveling in various media. The Disciplinary Core Idea expands this to: The wavelength and frequency of a wave are related to one another by the speed of travel of the wave, which depends on the type of wave and the medium through which it is passing.

One word that doesn't appear anywhere in the HS-PS PEs is "refraction". So we have a judgment call here. Do we assess attainment of this PE by asking about what does or doesn't change at an optical boundary during the process of refraction?

Students must also know what index of refraction refers to and that greater values indicate slower transmission speed in transparent materials. I'm delighted to teach the material; no one gets through my course not knowing how rainbows work. But I might have omitted index of refraction thinking no harm would come to my students based on a strict reading of the PEs and DCIs. I would have been wrong.

There are certainly topics I am skipping based on my reading of NGSS. Which ones will show up in the assessments? Time will tell. I'm not a huge fan of surprises like this.

Minor point: Numerical values were used for the indices of refraction and for the wavelength of the laser. I'm not sure why an infrared wavelength was chosen. When I use lasers in class, I prefer to stick to visible light varieties.

---

Quantify the change in energy associated with the appropriate change in the relative orientation of the two objects.
The item is a pretty spot-on assessment of the PE here, HS-PS3-5: Develop and use a model of two objects interacting through electric or magnetic fields to illustrate the forces between objects and the changes in energy of the objects due to the interaction.

I like this one because it shows a weakness in my own instruction that will require a bit of patching up.

During classroom instruction, students should be able to examine a number of systems and understand how potential energy changes within them. A rock and the Earth, two opposite charges, two like charges, magnets, springs... Students should know where potential energy is zero and where it is maximized in a given system.

I'll need to work on more explicit instruction of that beyond gravitational systems.

---

If you made it all the way to the end, pat yourself on the back. This was a long one. I will nourish a hope that we get more released items each year. I was unambiguous about this priority when I served on California's Assessment Review Panel. But panelist's wishes were not always accommodated. I really am trying to be subtle here! In any case, I will try to maintain cautious optimism.

Having said that, I will add that the most fun I had practicing and honing the craft of teaching physics occurred in the years when my students were not assessed with end-of-course exams intended to enforce a measure of accountability. My students in that era did not leave my course with woeful gaps in their physics knowledge. But that's just me shaking my fist at the sky.

Practice items for California's upcoming NGSS test

The time has come for students of science in California. That's right, CDE's CAASPP's NGSS CAST from ETS goes operational this spring.

That wasn't a cat walking across my keyboard; that was the initialisms and acronyms that spell out the new state-mandated assessment regime in California.

NGSS is, of course, the Next Generation Science Standards. CAASPP is the CAlifornia Assessment of Student Performance and Progress. CAST is CAlifornia Science Test. (I'm suddenly envious of the science tests given in Missouri and Virginia, less so for those in Colorado, Delaware, Georgia, Hawaii, Maine, Nebraska, Rhode Island, Washington, Wisconsin, and lastly, Louisiana. ETA: Looks like I missed Michigan and passed Pennsylvania. I'll be here all weak.)

NGSS has been around for some time now, with its multi-colored documentation of Performance Expectations, Science and Engineering Practices, Disciplinary Core Ideas, and Crosscutting Concepts. California has digested NGSS into its own Science Framework. The Framework and NGSS articulate broad, sweeping visions of science instruction.

CAASPP is the program tasked with testing. The previous iteration of this (in California) was State Testing And Reporting (STAR). This is where the lofty visions presented in the vision documents (NGSS/Framework this time) must be broken down into test questions ("assessment items"). As it was during the STAR era, the California Department of Education (CDE) has contracted the services of Educational Testing Services (ETS) to develop the CAST. (The proliferation of initialisms and acronyms indicate the importance of the enterprise.)

Assessments are where the rubber of the grand visions meet the road of perceived accountability. I say this as someone who served on the Golden State Exam development committee and was appointed by the State Board of Education to serve on CDE's CRP (Content Review Panel, later rebranded as the ARP: Assessment Review Panel).

I am a relentless advocate for released test questions (RTQs). The vision documents are necessary, I suppose. But they never really specify measurable outcomes. The visions boil down to "all students should be knowledgeable in science and capable of performing scientific analysis and related tasks." But vision documents quickly blow up into phonebook sized documents with webs of interrelated objectives that can make tri-level chess seem simple by comparison. But with a sufficient bank of RTQs, I will know exactly what your vision was. Inductive reasoning is really the only way to connect standards and assessments, here.

Until recently, there were just a handful of CAST RTQs available. You can find them on the California Science Test Training Items Scoring Guide: High School (PDF).  The mix of topics (PE/DCIs) left much to be desired.

Life Science: 6 - Earth Science: 1 - Physical Science: 0 - Engineering Design: 0

Remember that in NGSS, high school science opposes three domains: Life Science, Earth Science, and Physical Science. The old subjects known as "chemistry" and "physics" have been combined into Physical Science. But not even one RTQ covered anything in Physical Science.

More recently, a raft of 50 RTQs has been launched. You can find them on the California Science Test Practice Items Scoring Guide: High School (PDF). The mix seems a bit better:

Life Science: 16 - Earth Science: 15 - Physical Science: 15 - Engineering Design: 4

Of the 14 items tagged as Physical Science, 6 related to topics covered in physics. The other 9 were chemistry questions.

Comments and criticism of specific items will be offered in a later post.

#AAPTSM18: Alternatives to AP Physics 1 and AP Physics 2

At AAPT's Summer Meeting 2018, I attended session AD: High School, with considerable interest. After a series of College Board-friendly talks by AP Physics Redesign proponents, Mt. Olive High School's Brian Holton presented "AP Physics 1: A Seasoned Perspective".

Holton's talk was clearly not sanctioned by the good people of The College Board. But his expression of frustration and exasperation with AP1 resonated with me. Apparently my expression of frustration and exasperation resonated with him, too. (He cited my lament in his talk.) His critique was much more robust than mine was.

A small group of us had a combination perambulation, ventilation, brainstorm as we migrated to our next sessions.

We concurred that dropping AP Physics 1/2 from a school's curriculum constituted a marketing challenge for any school that would dare to try. We now advertise and market our schools on the basis of the breadth an scope of Advanced Placement offerings and performance.

AP courses are to be added to a school's course catalog; not removed. That other high school being visited by shopping 8th-graders and their parents is offering AP Physics, so your school must match.

One idea we tossed around was running a course that would prepare students for the SAT II Physics exam. Does anyone, anywhere run such a course? I'd love to hear from anyone teaching such a course. For now, it's just a thought. And The College Board still wins.

I know AP Physics C fans are happy with their exams. Abandoning AP1 and AP2 for an SAT II-based course is a different set of conversation. One might argue that outstanding performance on the SAT II Physics wouldn't get students out of any physics course at a college or university. I would hasten to add that outstanding performance on an AP Physics 1 or 2 exam doesn't necessarily exempt a student from intro physics courses at college, either.

Here's what the SAT II Physics exam covers. (A physics content-based assessment: how tantalizing!)

Mechanics 36%-42%
Kinematics, such as velocity, acceleration, motion in one dimension, and motion of projectiles
Dynamics, such as force, Newton’s laws, statics, and friction
Energy and momentum, such as potential and kinetic energy, work, power, impulse, and conservation laws
Circular motion, such as uniform circular motion and centripetal force
Simple harmonic motion, such as mass on a spring and the pendulum
Gravity, such as the law of gravitation, orbits, and Kepler’s laws

Electricity and magnetism 18%–24%
Electric fields, forces, and potentials, such as Coulomb’s law, induced charge, field and potential of groups of point charges, and charged particles in electric fields
Capacitance, such as parallel-plate capacitors and time-varying behavior in charging/ discharging
Circuit elements and DC circuits, such as resistors, light bulbs, series and parallel networks, Ohm’s law, and Joule’s law
Magnetism, such as permanent magnets, fields caused by currents, particles in magnetic fields, Faraday’s law, and Lenz’s law

Waves and optics 15%–19%
General wave properties, such as wave speed, frequency, wavelength, superposition, standing wave diffraction, and Doppler effect
Reflection and refraction, such as Snell’s law and changes in wavelength and speed
Ray optics, such as image formation using pinholes, mirrors, and lenses
Physical optics, such as single-slit diffraction, double-slit interference, polarization, and color

Heat and thermodynamics 6%–11%
Thermal properties, such as temperature, heat transfer, specific and latent heats, and thermal expansions
Laws of thermodynamics, such as first and second laws, internal energy, entropy, and heat engine efficiency

Modern physics 6%–11%

Quantum phenomena, such as photons and photoelectric effect

Atomic, such as the Rutherford and Bohr models, atomic energy levels, and atomic spectra
Nuclear and particle physics, such as radioactivity, nuclear reactions, and fundamental particles
Relativity, such as time dilation, length contraction, and mass-energy equivalence

Miscellaneous 4%–9%
General, such as history of physics and general questions that overlap several major topics
Analytical skills, such as graphical analysis, measurement, and math skills
Contemporary physics, such as astrophysics, superconductivity, and chaos theory

Where we are with California NGSS state tests

This post is really just a link share. I don't know about you, but I often get so wrapped up in teaching physics that I lose track of where the state is in terms of mandated statewide assessments.

Gone is STAR and its CSTs. In development is the California Science Test (CAST), part of the California Assessment of Student Performance and Progress (CAASPP) [No program is legit unless it has a handy acronym or initialism.] So CAST is the product of CAASPP Science.

The skinny: The CAST is for eligible students in grades five, eight, and once in high school. It's currently in development with pilot tests being run. The test will become operational in the 2018-19 academic year.

Here's the California Department of Education's CAST web page.

Here's a link to released practice test items. When I looked, I was disappointed. Perhaps material for a post to follow. Take a look for yourself at Online Practice and Training Tests Portal.

And here's the promotional video offered on that page.

I have stopped recommending the AP1 and AP2 exams—when will you?

I expressed my misgivings about the AP Physics 1 and AP Physics 2 exams in a previous post. Once again, this is a rant. If you're not into rants, kindly scroll to the next post.

The purpose of this post is simply to show the global exam score distributions, three years into the redesign. I used the numbers posted by Total Registration.

First up: the final three years of the AP Physics B exam scores. Fives were earned by 15% of students, fours were earned by 19%, threes by 27%, twos by 17%, and ones by 22%. (Approximately one third succeeded with 4s and 5s.)

Next up: the first years of the AP Physics 1 exam. Fives were earned by 4% of students, fours were earned by 14%, threes by 21%, twos by 30%, and ones by 31%. (Approximately one fifth of candidates succeeded.)

Try not to judge too harshly those who have lost enthusiasm for algebra-based AP Physics.

To complete the Algebra-based exams, consider the AP Physics 2 exam. Fives were earned by 9% of students, fours were earned by 15%, threes by 34%, twos by 32%, and ones by 9%.

The College Board hastens to offer explanations for the precipitous plummet, especially in regard to AP1. The AP2 scores actually match the legacy APB results reasonably well. There's reason to speculate that many (if not most) AP2 examinees are successful veterans of the AP1 exam.

1. Teachers aren't teaching the new course correctly. They haven't undergone enough in-service retraining. (This is part of the admission that the AP1 and AP2 exams aren't really about physics anymore, but about Big Ideas. Physics topics are merely the canvas upon which the Big Ideas are painted.)

2. Too many students are taking the AP1 exam. This is where I'm trying to help. And you can, too. I assure my students that I will do my best to prepare them for the exams as they are. But I encourage them to not submit to the exam. This reverses a position I held for 25 years. If we work together, we can help to reduce the glut of AP1 examinees.

The Physics team at the College Board has replaced the muscular Camaro that was AP Physics B with shiny Lamborghinis called AP Physics 1 and AP Physics 2. Emulating the college course that examinees could test out of is now a long-abandoned goal. The new courses hope to be the richest, most immersive physics courses that could be imagined.

I don't find the course syllabi to be particularly practical for high school students. The courses go well beyond virtually all introductory college and university courses. The AP Physics B exam was more demanding than those administered at the college level. And the AP1 and AP2 exams amplify that disparity to 11.

Surely this escalation of difficulty of the AP1 and AP2 exams must have resulted in a global advancement of the way colleges and universities accept and interpret passing scores. If it has, I am unaware of it. In the old days of AP Physics B, college acceptance of passing scores was a crazy patchwork. To the best of my knowledge, the crazy patchwork remains unchanged even with the more stringent AP1 and AP2 exams.

You might wonder why AP Physics C instructors remain enthusiastic about their course and exam. They were unaffected by the AP Physics B redesign. Let's bring them into the mix.

While the AP1 and AP2 scores plummeted, the un-redesigned AP C Mechanics exam scores continued to flourish. Over the past three years, fives were earned by 31% of students, fours were earned by 28%, threes by 18%, twos by 13%, and ones by 10%. (Nearly 60% success rate.)

The score of 5 (highly qualified) now represents the mode.

The un-redesigned AP C Electricity and Magnetism exam scores did well, too. Over the past three years, fives were earned by 30% of students, fours were earned by 24%, threes by 14%, twos by 19%, and ones by 13%. (More than half succeed.)

The score of 5 (highly qualified) continues to represent the mode.

AP Physics C continues to be a physics content-based exam. That is, its blueprint refers to topics in physics instead of the more philosophical Big Ideas that form the core objectives of AP1 and AP2. This may well be a key to the continuing success enjoyed AP C examinees.

Of course there's more to this discussion. I'm obviously ventilating a bit here because this continues to be a source of frustration. AP Physics C is not a viable option at my school due primarily to its prerequisites. Intra- and inter- district competition for school enrollment prevents me from simply abandoning AP1 and AP2. If not for that, I would.

With haste.

Everything about AP Physics changed except...

My joyless, long-winded assessment of the new AP Physics 1 and AP Physics 2 exams below was long enough as it was. I couldn't suitably fit in all my misgivings and supporting tangents.

As I said, the redesigns is a shining utopia that may well be worthy of pursuing. The old AP Physics B has been torn apart and used for parts in the new, sweepingly grander vision. In very real ways, classroom teachers have been compelled to abandon the old college replacement course and deliver a course that matches what many would describe as the best college courses offered in the nation (if not the world). All in the confines of a high school setting.

The new course represents a rebuilding from the ground up. Everything is different—for the better. To pass AP Audit muster, teachers must attest to conforming to the many layers of the New Vision, and that they've rebuilt their courses from the ground up.

What good fortune it is for the College Board that in spite of the upheaval visited upon classroom teachers far and wide, the exam, itself, required no real structural change!

The old exam consisted of dozens of multiple choice questions and a handful of free-response questions. After the exhaustive overhaul required of AP-approved physics teachers, the corresponding new exam consists of dozens of multiple choice questions and a handful of free-response questions. At the College Board's end, the multiple choice will require machine scoring as it already exists, and the free-response will require readers and table leaders as they have been using since the exam's inception.

So it's not the same old wine in a new bottle. Rather, a completely new wine in an old bottle. As demanded by the bottlers, themselves. Good fortune for the bottlers; tough rows to hoe for the workers in the vineyards.

Why I no longer recommend the AP Physics exam

[This post carries the "rant" label, so it is presumed that your mileage may vary. I never presume my own experience is universal. None of this post refers to AP Physics C.] 

Since I began teaching Advanced Placement Physics B in 1986, I have strongly encouraged all my AP students to purchase and take the AP Physics exam each May.

For now, I am recommending that they do not.

As we know, AP Physics B has been re-imagined as AP Physics 1 and AP Physics 2. The old vision of AP Physics acting as a college equivalent was dispensed with in the redesign. AP Physics B (as well as content standards prevalent in the 2000s) were seen as too content-focused, and lacked emphasis on process. Not an unfair criticism, to be sure.

The newly redesigned AP Physics 1 and AP Physics 2 are ambitiously constructed, multi-layered visions of physics course utopia. They are Lamborghinis sent in to replace the Camaros that existed until now. College courses, for the most part, remain largely content-driven.

The vision of the AP course being a college equivalent has been abandoned. AP Physics 1 and AP Physics 2 are the best courses time, talent, and energy can buy. Meticulously crafted courses of physics perfection.

But as of now, it's beyond my budget. And beyond the budget of my highly capable students. It is very much in an experimental phase. Version 1.0 of a complex new machine that we have been asked to be the early adopters of.

There is evidence to suggest the nation's AP Physics 1 and 2 teachers and students failed to abruptly snap to The College Board's new vision. Scores between the 2014 AP Physics B administration and the 2015 AP Physics 1 and 2 administrations show a massive shift in the wrong direction.

In 2014, more than 60% of AP Physics B candidates passed the exam with a score of 3 or better. Of those, 34% passed with a 4 or 5 (over 15% were 5s).

In 2015, more than 60% of AP Physics 1 candidates failed to earn a score of 3 or better. Fewer than 20% of the candidates scored a 4 or 5 (only 4% were 5s).

It could be that America's physics students suddenly became less capable. But that's not likely. It's more likely that America's best and brightest physics students weren't prepared for the radically different new exam.

The College Board would argue that they provided extensive information on the newly designed courses. And indeed they have. Exhaustive, really.

But part of The New Vision is that there is no longer a blueprint for exactly what content is to be covered and in which amounts. Physics content mastery is no longer tested, it is now presumed. Instead of distributing questions across a well defined subject-matter topic list, questions are not cast across broadly-defined Big Ideas and Science Practices. Mastery of these can be assessed via questions that merely use physics content as the canvas upon which the Big Ideas and Science Practices are painted.

True mastery of the Big Ideas and Science Practices, via Learning Objectives and bits of Essential Knowledge, painted on the canvas of physics, demands a level of engagement I find to be unreasonable. Don't get me wrong, I'd love for my students to be single-mindedly focused on everything we do in physics: in class and beyond the classroom. But my students have other things going on in their lives that demand attention. Many, many other things, as is typical of highly capable, multi-talented college-bound students.

It is certainly The College Board's prerogative to offer a product for purchase as they see fit. But in my opinion, they got it wrong. My misgivings may well be mine, alone. But that's not the sense that I get when talking candidly to other AP Physics teachers. I assume some folks love the redesign. I trust they'll light me up in the comments because I have not talked to such folks in person.

Speaking only for myself, I would say The College Board did what most august bodies do when undergoing a revisioning process: they spent their time, talent, and energy creating a grand vision. With a multi-dimensional outline and hierarchy. A complex and ornate structure. They then committed that vision to a tome of well over 200 pages.

What would have been more useful to me is for them to have spent more time, talent, and energy developing valid practice items. Practice multiple choice, practice multi-correct, practice free response (all types). I am very much "an assessment guy," having served on AAPT's Editorial Exam board, California's Golden State Exam development team, and California's Content/Assessment Review Panel.

So my process is inductive rather than deductive. If you show me a large sample of assessment items, I can determine what your grand vision was. But if you show me your grand vision, I cannot figure out exactly what your assessment items will look like. Maybe others can deduce better than I can.

But I have yet to see a grand, sweeping process (including you, NGSS) that respects the value of assessment and released items. The spotlight is always shown upon The Vision. Pesky questions about assessment and released items are marginalized with responses such as, "We'll get to that later," "We don't have the budget for that," "The task of item development will be given to other people who are not part of the Blue Ribbon Visioning Panel".

AP Physics redesign advocates encourage teachers to participate in week-long AP training sessions, and I do not doubt that such trainings offer great value (assuming the time and money can be managed). I also know of someone who did the AP training and was left with the impression that rotation was a huge focal point for AP Physics 1. Workshops will always carry the biases of their instructors.

What's more, suppose a student does make a passing score on the redesigned exam. They've shown they can drive that Lamborghini with aplomb. As far as I know, colleges are no more likely to grant credit for the post-redesign exam than they were for the pre-redesign exam.

I'm already past TL;DR territory, so I'll wrap it up. For now, I cannot recommend my students pony up $100 to (most likely) end up on the wrong side of the cut points on an experimental exam. I'll do my very best to prepare them for the exam. But it's fair that they know the extent to which the odds are not in their favor.

With luck, there will be a time in the future by which a sufficient catalog of released items exists so that I feel comfortable that my students will be able to fare well on the AP1 or AP2 exam. But I have no reason to think released item development is a priority for The College Board. So I'll have to wait them out. I do not know how long it will take.

I do know of schools that are switching from AP1 and 2 to AP Physics C, and I know of colleagues who are looking for college credit alternatives to AP1 and AP2. I understand the motivations for these moves completely. To those who are delighted with the redesign, I wish only happiness and good cheer. And the hope that they realize their own experience is not universal.

Did the sands of AP Physics 1 and AP Physics 2 just shift again?

UPDATE: I've been told that the my interpretation is mistaken (which happens only on days that end in a "y"). The equation sheets posted on the page linked to below are the real and true 2015 equation sheets of record. The course description documents that include a previous version of these sheets need to be updated with the revised equation sheets, and it's that that will not occur until April.

AP Physics 1 and 2 teachers, I trust you all knew the newly-released equation tables were undergoing revision.

The College Board's statement regarding the equation tables

Revisions to the AP Physics 1 and 2 Equation Tables
The AP Physics 1 and 2 equation tables – previously published in the course and exam description, curriculum modules, and practice exams – are being revised to provide complete definitions of symbols. Updated versions of those documents will be available by April 2015. [Emphasis mine] In the meantime, please disregard the equation tables in the current versions of these publications. Instead, refer to these new, revised equation tables and share them with your students…"

I recently turned 50, so I realize that my recollection that the release of the revised tables was set for February 2015 could be in error.

But the College Board exhausted my patience when it repeatedly set and then pushed back the publication of the redesigned AP1 and AP2 course descriptions, starting in 2008.

Like I said, it could be my own memory at fault here.

If they set a February date for the new equation tables and then moved it back to April, who among us would be surprised?

If they did retreat from their own deadline on the equation tables, I'm guessing they didn't push back the deadline for classroom teachers to submit course syllabi for the AP course audit. You know the one: where you spell out in great detail the minutia of these courses that you've never taught based on a 230-page course description only recently made public, so as to show that you've turned on a dime from the old vision to the new vision. You had the summer to retool, redesign, and re-imagine.

OK—break's over… kind of

I know it's been a while. And my output was off for 2014 compared to previous years.

The reduced output relates directly to berthing AP Physics 1 and AP Physics 2 at Rio while also retooling Physics from the California Content Standards version that I had polished for the past decade to the Next Generation Science Standards version that is expected to blossom in the next year or so.

So yeah, making this all happen at school to the best of my abilities keeps off the streets (and blogs) at night.

I have no idea when this will be completely sorted. It would be an error to presume that once the year is done, I'll have The Perfect Physics, AP 1, and AP 2 curricula all worked out and ready to repeat until retirement. But I also don't expect things to be as bad for me next year as they are this year.

I do have some groovy posts in the works. They'll be up soon! Neil deGrass Tyson's Cosmos and the Tavurvur volcano are topics.

But I must wave the "My Dear Machine" flag once again. I still love the song, and Leigh Nash's voice is dreamy.

Sixpence None The Richer - My Dear Machine

Planning the year: NGSS, AP1, and AP2 - Part 1: The broad strokes

My assignment for the 2014-15 academic year includes Physics, AP Physics 1, and AP Physics 2. The Physics course should be aligned to Next Generation Science Standards. Our students are years from facing NGSS assessments; right now school officials of nearly every stripe are focused on Common Core State Standards nearly to the exclusion of any other academic concern.

Last year marked my first attempt to let go of the past decade+ focus on California's now-abandoned academic content standards in physics.

Advanced Placement Physics 1 and Advanced Placement Physics 2 debut this year, and Rio has enrollments in both. Now that AP Physics B is dead and gone, The College Board is laying the AP1 and AP2 cards on the table. My friend, Chicagoland physics teacher extraordinaire, Martha Lietz, pointed me to her AP Physics resource page. And I have been poking around in the links!

This post will give a general direction of how I plan to implement the three courses at my school.

Physics remains a first-year course with an Algebra 1 prerequisite. It needs to fulfill the needs of NGSS as well as provide a suitable foundation for students who might elect to move onto AP Physics 2. It is not feasible to allow Rio's pipeline to AP2 be restricted exclusively to AP1 "alumni".

AP Physics 1 is a first-year course for highly-motivated students who have passed Algebra 2. It must cover the AP Physics 1 syllabus provided by the College Board. But it must also cover the expectations of NGSS.

AP Physics 2 is a second-year course for highly-motivated students who have passed Algebra 2 and a first-year physics course (Physics or AP Physics 1). It must cover the AP Physics 2 syllabus provided by The College Board.

In any case, here's my plan so far. Click to embiggen. Subject to change!

And yes, I'm willing to try the current fashion of energy before momentum. Wide ties/narrow ties. It seems that among the cognizanti, teaching momentum before energy is on par with wearing a wristwatch or typing two spaces after a period as far as age indicators go. I know there are strongly-held beliefs, arguments, and preferences here. Sometimes I think we get too exercised about such things.

NGSS is a Renaissance, not an Upheaval for Physics Teachers

Your use of the word upheaval is overly sensational. According to Webster's, upheaval means: "a major change or period of change that causes a lot of conflict, confusion, anger, etc.". This characterization would only apply to AP Physics B teachers, a small subset of high school physics teachers (and now a null set!). As for NGSS, I would use something like "freedom", "autonomy", or even "Renaissance". Unlike previous top-down efforts to shackle professionals to a checklist of factoids, this set of standards is more about the process of teaching students how to think and use information to understand the world they live in. There is a large degree of freedom given to teachers to determine how they want to approach achieving the NGSS. NGSS is very similar to the approach outlined in decades-old documents like the Project 2061 "Benchmarks for Scientific Literacy" and the 1991 California Science Framework. It is the pendulum swinging back to a better time in physics teaching. If you rely on your professional judgement as to what constitutes good physics teaching practices, you will not have to worry very much about adapting to NGSS.

Even if you are a teacher that is experiencing "a lot of conflict, confusion, anger, etc." regarding NGSS, I say relax and enjoy the next 2 school years without worrying about preparing your students for state-mandated tests. The earliest these could return would be the 2015/16 school year and the people I work with that are more involved in this process expect them later than that. This is from the FAQ page on NGSS for California:

"When will there be new assessments for the NGSS?
The earliest new science assessments might be available is the 2014–2015 school year. However, due to the short timeline, new science assessments will most likely not be available until the following school year.

Will Smarter Balanced Assessment Consortium (SBAC) have science assessments for NGSS?
At this time, new science assessments will likely be developed much like the assessments of SBAC . However, it is still too early to know exactly how and when new science assessments will be administered.

In SSPI Torlakson’s report Recommendations for Transitioning California to a Future Assessment System, Recommendation 4 encourages the development of new state science assessments consistent with the newly adopted NGSS for California, that include item types consistent with the SBAC assessments (e.g., short and extended constructed-response items and performance tasks)."

The full list or FAQs can be found here:

http://www.cde.ca.gov/pd/ca/sc/ngssfaq.asp

I suggest they dig up the old Golden State Exams for Physics and complete this retro cycle!

Dan Burns
Los Gatos High School

California Physics: Gem of the state-mandated science tests

The data is in for what appears to be the last round of end-of-course content standards tests.

And in science, Physics finishes on top!

Here are the 2013 Advanced and Proficient percentages:
Biology: 49%
Chemistry: 40%
Earth Science: 37%
Physics: 53%

While CSTs were first administered in 2001, the California Department of Education uses 2003 for its baseline data.

Here are the 2003 numbers for comparison:
Biology: 37%
Chemistry: 31%
Earth Science: 21%
Physics: 29%

Here are the year-by-year numbers with trendlines.

While the other science CSTs were going down, Physics managed to go up!

The award for highest proficiency rate goes to... Physics with 53% in 2013.

The award for most-improved goes to... Physics for posting a 24-point improvement from 2003 to 2013.

Are there other ways to analyze the data? Yes. Even to the point that Physics isn't the science winner? Yes. Am I going to elaborate on those? No.

Congratulations California public school physics teachers: you are—objectively—the best!

To access your school's 2013 STAR results, start at the 2013 STAR Test Results page. From there, select your county, district, and school. Then click the "View Report" button.

To see district, county, or state results, simply leave deeper fields unspecified.

Rio's 2013 Phyz students are best ever

Rio's 2006 Phyz students held the top spot for six years. They scored as 72% Advanced or Proficient in Physics. With the STAR program's End Of Course (EOC) California Standards Tests (CSTs) winding down, this year's classes rose to the challenge. They will go down as Rio's best physics students ever, as measured by the state of California.

The physics students of 2006 had edged out those from 2001 by  single percentage point.

The physics students of 2013 eclipsed 2006 with a stunning 77%!

Forty-one percent performed at the Advanced level; 36% came in as Proficient. Seventeen percent were rated Basic, 2% Below Basic, and 3% Far Below Basic. (Due to rounding, this does not add to 100%.)

I'll update this post with graphics and more analysis eventually. For now, I'm going to enjoy the buzz.

And congratulate Rio Phyz 2013 as our best physics students ever!

Click here to see if this takes you to schoolwide results.

UPDATE: As promised, here is a breakdown and some longitudinal context for the 2013 data.

We begin with the most detailed breakdown: the number of students at each performance level. Performance levels are Advanced, Proficient, Basic, Below Basic, and Far Below Basic.

Rio Americano/Baird Student Performance, 2008-2013

This analysis leaves something to be desired. There's too much data to see larger, more important trends. A more telling chart compares "good" to "bad". That is, the numbers of proficient (or better) students to the number of basic (or below) students. This chart captures the relative proportions and the overall sample sizes from year to year, and is therefor the most useful chart for analysis.

Rio Americano ADV+PRO vs. BAS+BB+FBB, 2008-2013

Boiling it down too far yields a simple "horse race" result: what percent of the school's test-takers were either advanced or proficient. You lose sample size data here, so things can be somewhat misleading.

Rio Americano %ADV+PRO, 2008-2013

If this chart makes it appear as if 2012 in an anomaly, that's because it was. Rio did not have AP Physics in 2012. We will not have it in 2014, either. But EOC CSTs appear to have run their course, so no worries.

Rio's Physics CST proficiency rate of 77% stands as the highest mark among all the EOC tests. No other CST administered at the school had better results.

Districtwide, our 77% rate places us third among our nine comprehensive high schools. My hat is (again) doffed to Bella Vista and Mira Loma for their physics awesomeness.

Can too much emphasis be placed on the proficiency rate? Absolutely. An arguably better metric is the proficiency number: how many students scored as proficient or better. Rio's biology proficiency rate is always at or near the top for the school. While the physics rate of 77% is better the biology rate of 63%,  biology's proficiency number (63% of 521, which is 328) far outshadows physics' proficiency number (77% of 87, which is 67).

Still though, as a school, we end the Physics CST era on the highest note we've ever played.

California STAR Physics Reference disappointment

Despite high-level assurances that the California Department of Education (CDE) would update its published physics reference sheet. It has yet to happen, and much of California is deep into the 2013 STAR-testing administration season. We will quietly hope California's physics CST-takers won't find the changes too surprising, confusing, or off-putting.

The problem was detailed in a previous Blog of Phyz post.

The Physics Reference Sheet as used on the operational form of the STAR Physics CST is different from the Physics Reference Sheet included on the most recently published Released Test Questions document. And it's different from the Physics Reference Sheet offered as a stand-alone PDF on the CDE's STAR Program Resources: CST Science Reference Sheets page.

As of this posting, the reference sheet available at this link carries a 2005 copyright date. The sheet used on last year's operational form seems to have been changed in 2012. I must presume it's being used in 2013 as well. But to see it, you'll have to refer to the previous Blog of Phyz post. The state of California has not made this document public.

Disappointing. But not entirely surprising. I hope, at some point, to learn why CDE and ETS moved (with neither review by nor approval of its own Assessment Review Panel) to alter the document that had been used from 2004 through 2011.

They chose to then compound this error by updating neither the RTQ document nor the stand-alone reference sheet. It makes them appear completely out of touch and unconcerned. Or as if they're trying to keep the new reference sheet a secret. I can only hope this is not how they wish to be seen.