I currently teach a class called CIS (College in the Schools) Physics. This course is identical to the University of Minnesota course Introduction to College Physics: Phys 1101W, including the college credit/transcript that students receive for competing the course. Recently the high school teachers involved in the course got in a small discussion about grading. We had received little to no training on how to correctly grade Exams and Labs, the two main sources of the course grade (85% between the two categories). I proposed that we spend some time at the summer workshop working on grading consistently. The ‘U’ professor in charge of the CIS piece replied that the Physics Education Research group at the ‘U’ had done some research on the exams in particular. They gave a variety of TA’s (that’s who grade the exams there) exams with a rubric and got a wide range of scores. They repeated the process with a more refined rubric with little to no improvement in the range of scores. It seems that even physics problems, which are basically math problems (there is one ‘right’ answer), grading is basically subjective because of the differing levels of partial credit for a response. The saving grace at the ‘U’ is that they have one TA grade one problem for all students, so that there is consistency among that one problem. However, this obviously doesn’t help year to year variation. I found it interesting that even at the University, in a relatively popular course with a seemingly rock-solid answer scheme, subjectivity is still a significant problem.
Here is example of a question that is phrased poorly. The context of this problem is that a bullet is smashing into a block of wood, which then slides along a surface, eventually coming to a stop due to friction.
I (and the rubric-Creator) had pictured that students would apply conservation of momentum for the time while the bullet and block were colliding to find the desired velocity, and was grading as such. However, it is just as valid to apply conservation of energy, working backward from the time the block comes to a stop. In fact, given that we know the distance the block slides and we do not know the initial velocity of the bullet, it actually makes more sense to start the problem this way and solve in reverse.
So I ended up giving my students who solve the problem in reverse full credit as it was completely valid based on the wording of the problem. For next year the wording will change to Find the velocity of the block after the bullet smashes into it in terms of the masses of each and the initial velocity of the bullet. To do so they will have to apply conservation of momentum. I would just leave it ambiguous and solve it either way, but the next question is about the energy part of the problem.
Now a note on completion type problems. I (and the ‘U’) have long used multiple choice to assess some of the concepts involved in physics. After studying the benefits of completion type problems (Ooserhof, Conrad, & Ely, 2008), I am thinking of switching some of my MC questions to completion. The nice thing about completion, when done correctly, is that learners have to actually construct the answer themselves. Take, for example, a question relating to the previous sitiuation.
A bullet smashes into a block of wood. If one knows the final velocity of the wood and the masses of both the wood and the bullet, the speed of the bullet can be found by applying the physics principle of _________________. (Law of Conservation of Momentum)
This is useful in determining whether students understand when to use a particular physics ‘tool,’ which is a common problem for students. One must be careful not to phrase the question poorly;
A bullet smashes into a block of wood. One should use __________ to analyze this situation.
As we have spent time studying the efficiency of such collisions, it is plausible (and somewhat correct) for a student to reply with efficiency or energy disipation or even energy conservation, if the energy dissipated is included in the energy conservation equations (note: it would be impossible to tell if a student was in fact including energy dissipation this from their answer, another reason it is a bad question).
As a STEM educator I believe it is important that I spend the time to create quality questions that assess students on the content that matters and to minimize subjectivity as much as possible. It is definitely a journey that often can only be followed through the trial and error process.
Source
Ooserhof, A., Conrad, R.-M., & Ely, D. P. (2008). Assessing Learners Online. Upper Saddle River, NJ: Pearson Education, Inc.
Here is example of a question that is phrased poorly. The context of this problem is that a bullet is smashing into a block of wood, which then slides along a surface, eventually coming to a stop due to friction.
Find the velocity of the block after the bullet smashes into it.
I (and the rubric-Creator) had pictured that students would apply conservation of momentum for the time while the bullet and block were colliding to find the desired velocity, and was grading as such. However, it is just as valid to apply conservation of energy, working backward from the time the block comes to a stop. In fact, given that we know the distance the block slides and we do not know the initial velocity of the bullet, it actually makes more sense to start the problem this way and solve in reverse.
So I ended up giving my students who solve the problem in reverse full credit as it was completely valid based on the wording of the problem. For next year the wording will change to Find the velocity of the block after the bullet smashes into it in terms of the masses of each and the initial velocity of the bullet. To do so they will have to apply conservation of momentum. I would just leave it ambiguous and solve it either way, but the next question is about the energy part of the problem.
Now a note on completion type problems. I (and the ‘U’) have long used multiple choice to assess some of the concepts involved in physics. After studying the benefits of completion type problems (Ooserhof, Conrad, & Ely, 2008), I am thinking of switching some of my MC questions to completion. The nice thing about completion, when done correctly, is that learners have to actually construct the answer themselves. Take, for example, a question relating to the previous sitiuation.
A bullet smashes into a block of wood. If one knows the final velocity of the wood and the masses of both the wood and the bullet, the speed of the bullet can be found by applying the physics principle of _________________. (Law of Conservation of Momentum)
This is useful in determining whether students understand when to use a particular physics ‘tool,’ which is a common problem for students. One must be careful not to phrase the question poorly;
A bullet smashes into a block of wood. One should use __________ to analyze this situation.
As we have spent time studying the efficiency of such collisions, it is plausible (and somewhat correct) for a student to reply with efficiency or energy disipation or even energy conservation, if the energy dissipated is included in the energy conservation equations (note: it would be impossible to tell if a student was in fact including energy dissipation this from their answer, another reason it is a bad question).
As a STEM educator I believe it is important that I spend the time to create quality questions that assess students on the content that matters and to minimize subjectivity as much as possible. It is definitely a journey that often can only be followed through the trial and error process.
Source
Ooserhof, A., Conrad, R.-M., & Ely, D. P. (2008). Assessing Learners Online. Upper Saddle River, NJ: Pearson Education, Inc.
