Do Freshmen Who Take Modern Biology Learn Any Biology?

Do Freshmen Who Take

Modern Biology (03-121) Learn Any Biology?

by Peter B. Berget

To answer this question I decided to participate in a freshman testing program that is sponsored and conducted by the Center for Innovation In Learning (CIL). To participate in this "controlled" study, I had to design an assessment instrument that could be administered both by the CIL during freshman orientation week and by the Modern Biology instructors during the 98-99 academic year. Early in the summer of 1998, Alan Koretsky and I designed a 16 question biology "test". That test was given during freshman orientation week (Auguest 1998) to every freshman entering Carnegie Mellon in the Fall. The exact same test was given as part of the final exam in 03-121 at the end of the Fall 1998 semester (December 1998). It was also administered on the very first day of class to the Spring 1999 03-121 class (January 1999). Alan and I thought that students who completed 03-121 ought to have been able to get most of the correct answers to the questions on the "test".

The results of these testing opportunities and the statistical analyses (all compliled by Ann Chenoweth in the CIL) are given below. The "executive summary" is that there was a statistically significant, 50% improvement in test scores (from an average score of 8 to 12 correct answers) for those freshman students who took the Modern Biology course. The control group of freshman who completed one semester of classes not including Modern Biology failed to show any improvement. Thus unlike several freshman level, introductory courses that have been analyzed by the CIL study, Modern Biology actually seems to have a positive effect on knowledge in biology.

A question that needs to be answered is whether we wish to continue this study by repeating it in the coming years. If we do, should we refine our assessment instrument?

Here is the "test" with a summary of how all 980+ freshmen performed.

The Results

Of the 980+ freshmen who went through the freshman orientation testing process, 58 enrolled in and completed Modern Biology in the Fall of 1998 and 43 started Modern Biology in the Spring of 1999. These groups comprise the test and control groups. The test group: 58 freshmen (29 males and 29 females) took the "test" during freshman week and at the end of Modern Biology Fall 1998. The control group: 43 freshmen (28 males and 15 females) took the "test" during freshman week and at the beginning of Modern Biology Spring 1999. The control group scored 7 (out of 16) at the beginning of the Fall semester and scored 7.3 at the beginning of the Spring semester. The test group scored 7.9 at the beginning of the Fall semester and scored 12.4 at the end of the Fall semester. The statistics are shown below. Not shown is that the average score for all 980+ freshmen in August was 7.018.

A preliminary question analysis indicates that there is at least one question which is too easy, two questions that appeared to be uninfluenced by the course and two other questions on which there was some improvement but maybe not as much as one would like.

The error bars below are ± 1 standard error.

Here is some interesting information about what colleges these students were enrolled in.