MECHANICS

 

Background

An article by Richard Hake in the January 1998 issue of American Journal of Physics ("Interactive engagement versus traditional methods: a six-thousand student survey of mechanics test data for introductory physics courses," Am. J. of Phys, 66, 1, pp. 64-74) discusses the difference between traditional physics classes and what Hake calls "interactive engagement" classes in terms of students' pre-instruction and post-instruction performance on a standardized test of Newtonian mechanics called the Force Concept Inventory test (see D. Hestenes, M. Wells, and G. Swackhammer, "Force Concept Inventory", Phys.Teach. 30, 3, pp. 141-158 (1992). The FCI test is a multiple-choice test that examines students' abilities to apply the concepts of newtonian physics to everyday situations. The test uses non-technical language to describe the situations, but employs distractors that specifically address common-sense misconceptions about physics.

Hake defines the normalized gain g on the FCI to be the average increase in students' scores on the FCI divided by the average increase that would have resulted if all students had perfect scores on the post-instruction test. After analyzing results from more than 6000 students, Hake argues that the normalized gain is a meaningful measure of how well a course teaches Newtonian ideas to students. In particular, Hake finds that introductory physics course that can be characterized as "traditional" in teaching style get normalized gains of 0.23±0.04, a quite narrow range in spite of the wide range of student initial scores. By contrast, the normalized gains earned by the courses Hake calls "interactive engagement" courses are in the range 0.48 ± 0.14, a statistically significant difference. Hake examines and discards a number of possible explanations for these results, and concludes that the teaching methods employed in the courses make the difference.

The results described below are for courses using the 2nd edition of the book, which came out in the spring of 2002. Because the 3rd edition is so new, we have no results yet for classes using this edition, but we do not expect the results to be worse.

Pomona College

At Pomona College, we offered gave students the FCI as a pretest and a post-test in 2004 and 2006. Pomona students pre-test scores were high to begin with (with an average score of about 75%.) We recorded the following normalized gains for those years.

• Fall 2004:  0.59
• Fall 2006:  0.68

As one can see, these normalized gains are at the high end of Hake's range for interactive-engagement courses.

In 2009, we changed the structure of our introductory course so that Newtonian mechanics is now spread over two semesters, making it much harder to test student progress on the FCI. This is why we do not have more recent results.

De Pauw University

Dr. Vic DeCarlo has graciously given permission to post unpublished results for the Six Ideas course at DePauw University. The normalized gains that they found for the FCI are also at the high end of Hake's "interactive-engagement" range:

• Fall 2002:  0.58
• Fall 2003:  0.60

Students achieved these gains in spite of having significantly lower pre-test scores (in the 40% to 50% range), showing that Six Ideas courses can work well for students having a broad range of initial preparations.

Washington University

Researchers at Washington University recently published a paper (Cahill, et al., Multiyear, multi-instructor evaluation of a large-class interactive-engagement curriculum, Phys. Rev. ST Phys. Educ. Res. 10, 020101, 2014) that analyzes in great detail the differences between a Six Ideas class and a traditional introductory physics class at that institution.

Classes at Washington University were significantly larger (about 120 students per section) than at Pomona and at DePauw. Pre-test FCI scores were in about 65%. Technically, the researchers calculated a gain c instead of Hake's normalized gain g that is different in how it handles the scores of students who actually got worse between the pretest and post-test, but since most improved, the results should be reasonably comparable. The FCI gains c were:

• Fall 2009-2011:  0.29 ± 0.01  (compared to 0.17 ± 0.01 for the traditional course)

If we interpret these c-values as being roughly equivalent to Hake's normalized gains, then each gains were somewhat more than one standard deviation below Hake's average gain for the corresponding types of course. However, we do see that the Six Ideas course delivered a statistically significant improvement over the traditional course.

ELECTRICITY AND MAGNETISM

 

Background

The Brief Electricity and Magnetism Assessment (BEMA) is a multiple-choice conceptual test similar to the FCI except that it covers concepts in electricity and magnetism. According to published results, pretest scores differ little from chance in any setting, so one can simply focus on post-instruction scores. In one calibration test, students experiencing traditional instruction earned scores of 46% ± 3%, those in interactive-engagement courses earned scores of 56% ± 2%, and students taking a course based on Sherwood and Chabay's Matter and Interactions textbook earned final scores of 58% ± 3%. (Sherwood and Chabay are also the creators of the BEMA.) This is a tough test: even senior physics majors at Carnegie Mellon University only scored about 80%.

Pomona College

We gave the BEMA at Pomona at the end of E&M instruction using the so-called "draft 3rd edition" of unit E (which is actually quite different than the actual 3rd edition). Our average final score was

• Spring 2007:  58%

This is basically the same as the interactive-engagement and Matter and Interactions results reported above even though a Six Ideas course spends less than half the time on E&M that other introductory courses (including those based on Matter and Interactions) typically do.

Washington University

The article by Cahill, et al. described in the mechanics section above, also analyzes results for the BEMA. In this case, even though reported pretest scores were a bit higher than chance, the final scores were 39% for the Six Ideas course and 38% for the traditional course during 2009-2011. These results were not significantly different. We don't know why the larger gain experienced at Pomona were not realized at WU: possible issues include the differences in class size, student population, and/or instructors.

STUDENT ATTITUDES

 

Background

The Colorado Learning Attitudes about Science Survey (CLASS) is a 42-question tool for determining what students believe about physics and the process of learning physics. Students describe on a 5-point scale the extent to which they agree or disagree with the statements on the survey. Scoring for each statement is based on whether the student's response agrees or disagrees with how an expert would respond to that statement. The survey questions probe different categories of attitudes, such as what the student thinks about how closely physics is connected with the real world, his or her confidence in solving problems, and so on, with 4 to 8 questions pertaining to each category scattered throughout the test. This tool is designed to be given as a pretest and post-test, to examine how instruction has changed attitudes.

The striking result is that analysis of CLASS results for traditional lecture-based classes, lead to significant declines in scores, meaning that students leave the class with less expert-like attitudes than when they entered. Certain kinds of instruction (for example, courses based on Physics by Inquiry) can lead to net gains, typically a few percent of the total score.

Washington University

The article by Cahill, et al. described in previous sections also measured changes in CLASS results for the traditional class and the Six Ideas class. When the CLASS was administered at the end of the fall semester (but before E&M instruction), the results were quite striking: the average CLASS score decreased by 3.8% in the traditional class, but increased by 2.3% in the Six Ideas classes. The difference was statistically significant, and it is quite rare in any large-class setting to see an increase in CLASS scores. Sadly, instruction in E&M hurt things in both classes, leading to net decreases in CLASS scores, but the shift in the traditional class (–12.5%) was much larger than in the Six Ideas class (–6.0%). Again, the difference was statistically significant.

THE BOTTOM LINE

 

Analysis using a wide variety of tools in a number of different contexts show that students taking Six Ideas courses do better (or at worst the same) as in traditional courses. Large classroom sizes seem to reduce the gains somewhat, but do not erase the benefits.