Research by Feldstein et al. (2012) took place at Virginia State University, where OER were implemented across nine different courses in the business department. 1,393 students took courses utilizing OER. Researchers found that students in courses that used OER more frequently had better grades and lower failure and withdrawal rates than their counterparts in courses that did not use OER. While their results had statistical significance, because of a new core curriculum employed at Virginia State University’s Business school, the two sets of courses were not identical. Thus while these data are highly interesting, we should not generalize them too far. 315 students completed a survey regarding their perspective on the shift to the OER, and almost 95% of responding students strongly agreed or agreed that the OER were “easy to use” and 78% of respondents felt that the OER “provided access to more up-to-date material that is available in my print textbooks.” Approximately two-thirds of students strongly agreed or agreed that the digital OER were more useful than traditional textbooks and that they preferred the OER digital content to traditional textbooks.
Hilton et al. (2013) chronicled a study that took place at Scottsdale Community College (SCC) in Arizona. In the fall of 2012, OER were employed throughout five different math courses at SCC, affecting 1,400 students. Issues with the initial placement tests made it so that only four of the courses could be compared; nevertheless, the results of Fall 2012 (when OER was used) compared to Fall 2011 and 2010 showed that student results on department exams were approximately the same before and after the OER implementation. Surveys completed by 910 students showed that 78% said they would recommend the OER to their classmates. Similarly, 83% of students agreed with the statement that “Overall, the materials adequately supported the work I did outside of class” (only 5% of students disagreed with this statement). Faculty members were likewise positive about the open materials. Of the 18 faculty members who reported on their view of the OER, 50% said that it was of the same quality as traditional textbooks, 33% said it was better, and 17% said it was worse.
Gil et al. (2013) (OA preprint, final published version), surveyed students over a five year time, asking them to compare blogs that featured OER resources versus other blogs (all blogs were directly related to student coursework). Approximately 500 students were surveyed; on average, 40% of students said that the blogs featuring OER were of equal quality to the blogs that did not feature OER, 45% of students said that the blogs with OER were superior and 15% said they were inferior. They also found a correlation between increasing access to OER and improving student grades; however, these results were tentative as they were not able to compare the results from individual students; rather this was a trend that was observed over time and may possibly have been due to other factors.
Hendricks, Reinsberg, and Rieger (2017) examine the use of OER in a physics course at the University of British Columbia. 143 students completed surveys about the OER; students reported using the open textbook with the same frequency as their traditional textbooks. Moreover, 93% of respondents said it was the same or better than textbooks in other courses. The researchers also found that in one year of OER adoption students saved approximately $85,000 (Canadian dollars) and that student final exam scores and grade distributions remained the same after OER adoption as they had been previously.
Bowen et al. (2012) completed a rigorous study comparing the use of a traditional statistics textbook with Carnegie Mellon’s OLI at six different institutions. Participating students were randomly assigned to either the face-to-face class with a traditional textbook or a “hybrid” class that used the OER resource. Both groups took the same standardized test at the beginning and end of the semester, as well as a final examination. 605 students took the OER version of the course, while 2,439 took the traditional version. Students who utilized OER performed slightly better on the standardized exam than those who did not. The difference in outcomes was not statistically significant.
Hilton and Laman (2012), focused on introductory Psychology courses taught at Houston Community College (HCC). In the fall of 2011, twenty-three sections composed of 690 students used an open psychology textbook. The introduction of an open textbook was correlated with an increase in class grade point average, an increase of the average score on the departmental final examination, and a lower course withdrawal rate. No causation was claimed.
Pawlyshyn et al. (2013) reported on the adoption of OER at Mercy College in New York. In the fall of 2012, 695 students utilized OER in Mercy’s basic math course, their pass rates were compared with those of the fall of 2011, in which no OER were utilized. Researchers found that the pass rates increased from 63.6% in fall 2011 to 68.9% in fall 2012 when all courses were taught with OER. Similarly, students who were enrolled in OER versions of a reading course performed better than their peers who enrolled in the same course using non-OER materials.
Fischer et al. (2015) performed follow-up research on the second year of implementation at the schools studied by Robinson (2014). Their original sample consisted of 16,727 students (11,818 control and 4,909 treatment). From this sample, there were fifteen courses for which some students enrolled in both treatment (n=1,087) and control (n=9,264) sections (the remaining students enrolled in a course which had either all treatment or all control sections and were therefore excluded). While this represents a large sample size, students in treatment conditions were only compared with students in control conditions who were taking the same class in which they were enrolled. For example, students enrolled in a section of Biology 111 that used OER were only compared with students in Biology 111 sections that used commercial textbooks (not students enrolled in a different course). Thus when diffused across fifteen classes, there was an insufficient number of treatment students to do propensity score matching for the grade and completion analyses. The researchers found that in two of the fifteen classes, students in the treatment group were significantly more likely to complete the course (there were no differences in the remaining thirteen). In five of the treatment classes, students were significantly more likely to receive a C- or better. In nine of the classes there were no significant differences and in one study control, students were more likely to receive a C- or better. Similarly, in terms of the overall course grade, students in four of the treatment classes received higher grades, ten of the classes had no significant differences, and students in one control class received higher grades than the corresponding treatment class. Researchers utilized propensity score matching before examining the number of credits students took in each of the semesters as this matching could be done across the different courses. Drawing on their original sample of 16,727 students, the researchers matched 4,147 treatment subjects with 4,147 controls. There was a statistically significant difference in enrollment intensity between the groups. Students in fall 2013 who enrolled in courses that utilized OER took on average two credit hours more than those in the control group, even after controlling for demographic covariates. ANCOVA was then used to control for differences in fall enrollment and to estimate differences in winter enrollment. Again, there was a significant difference between the groups, with treatment subjects enrolling in approximately 1.5 credits more than controls.
Allen et. al. (2015) designed an experiment in which an experimental class of 478 students used the OER ChemWiki as its primary textbook, while the control class of 448 utilized a commercial textbook. The two sections were taught the same semester at back-to-back times using the same faculty member and teaching assistants. Students in both classes were given the same midterm and final exams. Researchers found no significant differences between the two groups both with overall exam results and item-specific questions. Beginning of the semester pre-tests, combined with final exams showed no significant differences in individual learning gains between the two groups. Student surveys regarding time spent on the class found that students in both groups spent approximately the same amount of time preparing for class.
Robinson (2015) devised a quasi-experimental study involving seven courses at seven colleges involved in Project Kaleidoscope. Outcomes of students using open textbooks (n=3,254) were compared with outcomes of students using traditional textbooks (n=10,819) in comparable courses. Controlling for covariates, Robinson found that students using open textbooks in the business and psychology courses earned lower average grades and were less likely than students using traditional textbooks to pass the course with a C- or better. Students in other five courses showed no significant difference in average grades or successful pass rates. In the biology courses, students using OER were more likely to complete the course than students using traditional textbooks. Across all courses, students using open textbooks took slightly more credits than students using traditional textbooks. This study is important in that it contains “the first finding of a negative effect associated with OER adoption” (p.59).
Wiley et al. (2016) focus on open textbook adoption at Tidewater Community College. Across two pilot semesters (fall 2013 and spring 2014) 23,232 students enrolled in traditional versions of selected courses. An additional 753 students enrolled in “Z” versions of these same courses (“Z” standing for “Zero textbook costs”). 3.57% of students dropped the course in the traditional courses versus 2.79% in the Z courses – a statistically significant difference. The authors hypothesize that money kept by universities from lower drop rates could support open textbook initiatives.
Hilton et al. (2016) focus on an additional two semesters at Tidewater (their results include fall 2013, spring 2014, fall 2014, and spring 2015). They studied the combined the drop, withdrawal, and C or better grade analyses to estimate the throughput rates of students taking traditional verses Z courses. In the face-to-face courses (Control n = 36,223 Treatment n = 1,151) 59.8% of students in non-Z courses made it through the successive hurdles of drop, withdrawal and passing the class, compared with 66.4% of students in the Z courses, for a difference of 6.6%. Combined the drop, withdrawal, and C or better grade analyses to estimate the differences between the groups in the overall success rate from students’ registration to the final grade. In the hybrid/online courses: (Control n = 7,000, Treatment n = 863) 54.2% of students who started in non-z courses successfully made it through the course with a C or better, compared with 59.8% of students in the Z courses, for a difference of 5.6%.
Croteau (2017) synthesizes reports written by faculty members as part of the Affordable Learning Georgia OER adoption program. Across twenty-four separate data sets involving 3,847 students using OER, Croteau found no significant differences in student pass-rate before and after implementing OER. Additional analyses of relevant data sets likewise showed no significant differences in completion rates or final exam scores.
Winitzky-Stephens and Pickavance (2017) examine a large-scale OER adoption across 37 different courses across several different general education subjects. Over a four-year period of time, 26,538 students enrolled in non-OER versions of these courses while 7,588 students enrolled in the OER versions. The multilevel models used by the authors find no significant difference between courses using OER and traditional textbooks for continuing students, and a small benefit for new students.
Chiorescu (2017) explores OER adoption in a college algebra class and found that by adopting an open textbook, each enrolled student could save $86.00. Across four semesters, 447 students used commercial resources and 159 students used OER. Depending on which semesters were compared, students were either as likely or more likely to pass the class when OER was used; moreover, significantly fewer students withdrew when OER were implemented.
Wiley, Tonks, Weston, and Webb (2017) study the use of student-created OER as a means to improving student performance. Students created tutorial videos, chapter summaries, and review games for a particular topic, some of which were placed into later versions of the course. The average grade on student assignments across several semesters (181 students) rose significantly as more student-created OER were added to the course.
Grewe and Davis (2017) find a positive relationship between the use of OER and student academic achievement in an online history course. She studies 146 students who enrolled in the fall of 2013 and spring of 2014; these courses had an identical instructional design (but different content and different instructors). In examining the final course grades of 146 students, Grewe demonstrates that OER adoption can be associated with higher student performance.
Petrides et al. (2011) administered surveys to instructors and students who utilized an open statistics textbook called Collaborative Statistics. >In total, 31 instructors and 45 students participated in oral interviews or focus groups that explored their perceptions of the OER which they had utilized. The authors found that “Cost reduction for students was the most significant factor influencing faculty adoption of open textbooks” (p. 43), partly because it increased student access. 65% of students surveyed reported a preference for using open textbooks in the future because they are generally easier to use.
Bliss et al. (2013a) reported on surveys completed by 11 instructors and 132 students at seven different colleges. 60 percent of instructors stated their students were equally prepared when OER replaced traditional texts and 30 percent said their students were more prepared. One of the eleven instructors felt students were less prepared. All 11 instructors surveyed stated they would be very likely to use open texts in future courses. The students were also very positive regarding the OER materials. When asked to compare the OER texts to traditional texts, only 3% felt that PK texts were worse than their typical textbooks, 56% said they were the same quality, and 41% said they were better than typical textbooks.
In an extension of the previous study, Bliss et al. (2013b), studied OER adoption at eight different institutions of higher education. They surveyed an additional fifty-eight teachers and 490 students across the eight colleges regarding their experiences in utilizing OER. Approximately 50% of students said that the OER materials had the same quality as traditional textbooks and nearly 40% said that they were better. Students focused on several benefits of the open textbooks. Many cited technical advantages of the digital texts. In addition, the free cost of their open texts was important to many students. 55% of teachers reported that the open materials were of the same quality as the materials that had previously been used, and 35% felt that they were better.
Lindshield and Adhikari (2013) studied the perceptions of students who utilized a digital OER textbook in a Human Nutrition class. One hundred and ninety-eight students completed a survey in which they shared their perceptions of the OER text. “Students favorably rated their level of satisfaction, liking the idea of the [digital OER], ease of [digital OER] use, not having to buy a textbook, and preferring the [digital OER] versus buying a textbook for the course.” Moreover, they found that students disagreed or somewhat disagreed with statements to the effect that they would like to have a traditional textbook in addition to OER.
Allen and Seaman in their Babson Survey (2014) surveyed 2,144 college professors regarding their opinions on OER. Of the 34% (729) who expressed awareness of OER, 61.5% of respondents said that OER materials had about the same “trusted quality” as traditional resources, 26.3% said that traditional resources were superior, 12.1% said that OER were superior. 68.2% said that the “proven efficacy” were about the same 16.5% said that OER had superior efficacy and 15.3% said that traditional resources had superior efficacy. It is important to note that the professors surveyed in this study expressed awareness of OER, but had not necessarily utilized OER. In contrast, in the above-mentioned four studies the professors had actually utilized OER, perhaps having a greater basis on which to judge the quality of OER resources.
Pitt (2015) reported results of two surveys of 126 educators, conducted 2013 and 2014-2015, as part of a collaboration between OER Research Hub (OERRH) and OpenStax College (OSC), a provider of open textbooks. Around 65% of respondents reported that using OSC helped them meet diverse learners’ needs, while a minority of respondents said that the OSC materials made teaching easier, enabled innovation or changed their pedagogical approach. More than 65% of respondents perceived greater learner satisfaction for their students using OER. Nearly all respondents said that having using OSC materials increased the likelihood that they would recommend OSC materials to peers.
Jhangiani, Pitt, Hendricks, Key, and Lalonde (2016) examined awareness, usage, outcomes, and perceptions of OER among British Columbia post-secondary faculty. As part of a collaborative project between the BCcampus-led Open Textbook Project and OER Hub, researchers surveyed post-secondary educators through an online survey disseminated via email and social media. Of the 78 respondents, 77% had used OER. Most respondents rated OER quality as comparable or superior to that of traditional materials; educators who had adopted OER rated the quality significantly higher than educators who had not. Respondents reported that the top two barriers to using OER were finding relevant and high-quality OER. Faculty at research-intensive universities reported significantly lower barriers to finding high-quality OER than did faculty at teaching-intensive universities or colleges/institutes and were the most likely group to create or adapt OER.
The California OER Council (2016) released a white paper focused on OER adoption in CA higher education. 351 students completed a survey about their use of OER. When students were asked if the OER textbook chapter(s) were better than the traditional, 42% said the OER textbook as better, 39% said they were about the same, 11% rated the textbook as worse than the traditional textbook and 8% declined to answer. Of the 351 students in the survey, 71 printed the textbook and 209 used a PDF. 16% of students wanted to have the option to purchase a printed copy of the textbook from the bookstore for a small fee (10% of students wanted to print the textbook themselves). The predominant platform for reading e-textbooks is a laptop computer (only 89 of 351 students reported reading from their cell phones). Perhaps most importantly, 100% of the students in the study wanted to use OER textbooks in the future and would recommend the use of OER to friends. Sixteen faculty shared their perceptions about their use of OER. Seven faculty of sixteen felt that the OER textbook was superior to the traditional textbook for the course. Five faculty rated the OER as equivalent to the traditional textbook. Faculty were not as positive about the support materials (PowerPoints, Test banks) available with the OER textbooks. Half of the faculty felt that the support materials lacked quality. 25% of faculty felt that implementing the support materials took a significant amount of time. In their comments, the biggest comment made by faculty was about the need for support materials or the amount of time they spent in developing them for this adoption.
Delimont et al. (2016) surveyed 524 students in thirteen different courses at Kansas State University regarding their use of OER. They state, “Students indicated that they were somewhat satisfied taking courses using [OER] and used them somewhat more to more than a normal textbook. Students rated the [OER] as good quality and indicated that they were somewhat easy to use. Students agreed that they preferred using [OER] instead of buying textbooks for their courses.” Thirteen instructors were interviewed as well; all but one said they “preferred teaching their course with [OER] instead of a traditional textbook. Several (11/13) indicated that customization was a reason for this preference.” Similarly, all but one faculty member planned to keep using OER.
Illowsky, Hilton, Whiting, & Ackerman (2016) surveyed 325 students who used various versions of an open statistics textbook. In the first survey (n=126) students were surveyed about an earlier version of the textbook. “When students were asked to imagine a future course in which there were two sections, one offering traditional printed texts and the other offering texts such as the one they used in this course, 50% of students said they would choose the class with texts like those offered in this course. Only 19% said they would enroll in the course with the traditional printed text, and the remaining 32% said they would have no preference.” In this first survey, “When specifically asked how they would rate the quality of this text as compared to other textbooks they have used, 143 (62%) said that it was the same as books in their other courses, 57 (25%) rated it as better than other texts and 31 (13%) rated it as worse than other texts they have used.” The second survey (n=94) surveyed students who used the later OpenStax version. “In answer to the question, “How would you rate the quality of the texts used for this course?” 70% said it was about the same as the quality of the texts in their other courses, 23% said it had better quality and 7% said that it was worse.” There is a clear drop in the number of students who say that the OER is worse than traditional texts between the two studies. While the study does not point this out, it is noteworthy that the OpenStax textbooks have been adopted at a tremendously faster rate than previous iterations of the textbook.
Jung, Bauer, & Heaps (2017) report on the perceptions of faculty members who used OpenStax textbooks. This study adds significant value by focusing not simply on “OER in general,” but on a specific set of OER. They survey 137 faculty and find that only 16% report taking more time preparing to teach a course using OpenStax textbooks, and that of these, 78% felt that the benefits of OpenStax were worth the additional time. In terms of quality; 81% thought that OpenStax textbooks have same or higher quality as commercial textbooks. Thematic analysis revealed faculty perceptions of what constitutes high-quality open textbooks.>
Fischer, Ernst, & Mason (2017) examine the rich corpus of the reviews hosted by the Open Textbook Network (OTN). The OTN’s Open Textbook Library has been an extremely beneficial way to identify open textbooks and includes a very useful feature of allowing faculty members at participating universities to review open textbooks. This study examines 416 reviews of 121 textbooks and finds that reviewers generally gave open textbooks high ratings (a median of 4.5/5 overall rating).
Cooney (2017) studies three sections of a Health Psychology course that replaced a traditional textbook with OER. Six students were interviewed and 67 surveyed regarding their perceptions of the OER; findings indicate that students were generally positive about the OER. Just over 80% of 67 students surveyed rated the OER as being better than a traditional textbook, with an additional 16% saying it was of the same quality.
Jhangiani and Jhangiani (2017) survey 320 post-secondary students in British Columbia who were enrolled in courses that used an open textbook. They find that a majority of students report not purchasing at least one of their required textbooks and that a substantial portion of students report taking fewer courses and/or dropping courses based on textbook costs. Students favorably rated open textbooks, with 96% of survey participants stating that they were at or above average.
Watson, Domizi, & Clouser (2017) surveyed 1,299 students at the University of Georgia who used the OpenStax biology textbook. These students were directly asked to “rate the quality of the OpenStax textbook as compared to other textbooks they had used. Sixty-four percent said the quality was about the same, and 22% said it was higher or much higher than other textbooks. With regards to the readability of the OpenStax textbook, 64% of the students said it was “good” or “very good,” 31% said it was “fair,” and 5% said it was “poor” or “very poor.”
