General issues

 

As noted in the Overview, we have designed homework problems in the Six Ideas course (particularly the Modeling, Rich-Context, Derivation, and Advanced problems) to give students challenging practice in applying physical ideas to realistic situations and/or to deal with tough but important theoretical issues. While the examples in the book provide some guidance, they cannot and do not serve as templates for all the problems, and students cannot succeed by hunting for formulas. This means that a homework grading system for a Six Ideas course should not emphasize answers (as traditional homework grading might), but rather should reward students for making a serious and thoughtful effort and give them meaningful feedback.

This is a serious issue, because we discovered (in an early class taught by someone who was grading homework in a traditional way) that not designing an appropriate grading system can cause some serious student discontent and rebellion.

The sections below describe the system we have currently use at Pomona College. It is the culmination of years of experimentation and evaluation to find a system that rewarded the right student behaviors, gave useful feedback, and required the least amount of grading effort. Versions of this system has also been successful at Washington University and other locations. Because the grading system is much simpler than traditional grading, it is practical even for very institutions.

THE INITIAL EFFORT

 

At Pomona, we generally assign roughly five to six M-level problems (two per assigned chapter) and one or two R-level problems per week. (One can adjust the level by including a few B-problems in place of harder M problems.) On the homework due date each week, each student uses his or her cell phone (or other scanning facilities) to electronically submit their homework to a special email address before class. (We have an assistant on hand in class to scan the homework of the handful of students who do not have a smartphone, and we also accept xeroxed copies.) We require PDFs if at all possible (not pictures, which are sometimes quite hard to read) and recommend several scanning apps that seem to work well. The scan, photo, or xerox represents a record of the students' initial effort on each problem, and its main purpose is to deter cheating during the correction phase.

DISPLAYING SOLUTIONS

 

A professor who has registered on this website and been verified and given instructor status has the ability to create a password-protected list of problem solutions and define a time window during which students can view the solution. Students with the instructor-defined password can use the ProbViewer webapp to display the viewable solutions.

After class on the due date, the solutions to the assigned problems become viewable. Students use the displayed solutions to correct their own work, using a green or purple pen to distinguish the correction from their original work. They hand in the physical paper with their initial work and corrections when they next come to class.

GRADING

 

Each assigned problem is then graded (by a student TA or the professor) on a 10-point scale. We currently use a rubric that looks like this:

  I   C

□ □  Description (Sufficient & Clear, with diagram if needed)

□ □  Model (Principles & Approximation Stated and Correct)

□ □  Valid Math (Sufficient and Correct)

□ □  Good Notation (Symbolic Algebra, Units & Vectors)

□     Plausible (Right Units / Magnitude / Sign)

   □ Thoughtful Correction (not too much or too little)

____   Total – Deduction for Missing Parts ____ =  Final _____

Each box is worth one point, so a student starts with 10 points and retains those points if the boxes remain unmarked. The grader's marks indicate deductions: a single slash for 0.5 points off and a full X for 1 point off. The grader can underline some of the subcategories (the words or phrases listed in parentheses) to clarify why points were deducted. The left column refers to initial efforts, and the right column to corrections.

If an initial effort is missing parts, the grader marks the boxes based on what is provides and then deducts points based on what fraction of the initial effort was missing. If the student does not submit any initial effort, the grader draws a vertical strike through the "I" column, equivalent to a deduction of 5 points.

We create a sheet with as many copies of the rubric as there are assigned problems in the homework set. The grader makes as many copies of the sheet as students in the class, and after grading each person's homework, staples the grade sheet to the set before returning it.

Since the grader is not correcting or commenting on the student's work, and grading does not require a close analysis of what the student did wrong, the grading process takes literally seconds per problem. The grading process is also fairly robust and does not require a very sophisticated assessment of the solution. We have successfully used undergraduate graders (and carefully developed and tested this scheme with the help of undergraduates), though in sufficiently small sections, the process is easy enough that we professors sometimes just do it ourselves.

The main value of this grading scheme is most of the points are awarded for effort. A student who made an honest initial effort and then corrected that effort carefully will usually earn at 7.5 points or more, even if the initial effort was completely wrong. At the same time, it does reward getting the problem right the first time, and also provides better feedback about what a student is doing wrong than previous schemes (including some previously described on this site). The correction step requires that students closely study an expert solution and develop self-criticism skills.

We typically drop the five or so lowest individual problem scores in a given semester so that we are not constantly pestered by students offering excuses.

One does have to develop a few rules for handling errors that are not easily embraced by the rubric above. What if the student does the wrong problem? Doesn't do all parts of the problem? Just copies the correct solution without evaluating what they did wrong? Doesn't use the green or purple pen, making it hard to distinguish correction from initial effort? (This is a general problem with grading rubrics, though.) Late initial efforts are handled automatically by the system (anything submitted after the solutions are accessible is automatically a "correction"), but one still needs to decide what to do about late corrections.

A diligent grader might actively compare final solutions with initial solutions to ensure that the student has not attempted to pass off corrected work as initial work. But in practice, we find that the deterrence provided by students knowing that their initial work is on record and the extremely high cost of getting caught (an academic honesty citation) is sufficient. Spot-checks have never yet turned up evidence of student dishonesty.

(We have also used a variation where only one problem per week is graded this way and we give a 10-minute quiz at the beginning of class on the day the set is due on one of the other problems (selected at random). This has the benefit of requiring even less grading, and we have found that students work hard on reviewing the problem set to prepare for the quiz.)

BENEFITS AND COSTS

 

This system has many, many advantages over traditional grading. Just some of these are:

• The system makes it safe for students to try difficult problems and learn through failure.
• It supports the concept of homework as practice (like a sports practice).
• It explicitly rewards students for checking their initial work.
• Students are given strong incentives to carefully study expert solutions.
• Students practice and learn valuable skills of self-assessment.
• The system requires much less grading effort than traditional homework grading.
• Even undergraduates can adequately do a decent grading job.
• The system provides an automatic rule for handling late initial efforts.

The system does have a few disadvantages that one should thoughtfully address:

• Students can and do have various technical problems submitting initial efforts.
• The system is a bit complicated and requires explanation.
• One must decide what to do with solution inadequacies not imagined in the grading rubric.
• Seriously checking the initial efforts requires extra work.
• The online homework display system sometimes (but rarely) goes down.
• The system requires human graders and some actual paper handling.

A TWO-PASS VARIATION

 

Counting on virtually every student to have a smartphone has only recently (2015) really become reasonable. Before that, we used a two-pass system where students physically handed in their initial efforts. The initial efforts were "pre-graded" (usually by a student TA) and students could then correct and resubmit any solution that did not earn a 10 initially.

This system had a few modest advantages: it provided an expert check on the student grader's work, and students did not have to spend time assessing a solution they did correctly (though there was a downside to that also). But the disadvantages were huge: in addition to the obvious problem of requiring twice as much grading effort, we found the paper flow difficult to manage, and the time lapse between students doing the problem initially and getting feedback is much longer.

Even so, the system was still workable (and still required less effort than traditional grading). If we understand correctly, Washington University used this two-pass system successfully for some years. We therefore recommend even this variation for a Six Ideas class above any kind of traditional homework grading.

COMPUTER-BASED GRADING

 

Most available web-based grading systems focus on more traditional (B-like) homework problems and base their grades on answers rather than on reasoning. Such systems do not work well with Six Ideas-type problems (and indeed, few have any Six Ideas problems in their databases).

However, we do have people at McGraw-Hill working on creating a web-based package specifically for Six Ideas courses. The main disadvantage is that not all of the problems in the text are are easy to implement in any package that focuses on numerical answers.