Evaluation Methods
As educational uses of technology have become more common, faculty members, administrators, and staff members have begun to investigate the impact that technology is having on students' learning environments. Are students able to integrate the use of technology into their study habits? Does it permit them to work effectively with their peers? Have their learning outcomes improved? A well-conceived evaluation project can produce very illuminating answers to these questions. For instance, a recent technology survey of students at the University of Minnesota revealed that:
- in general, students' feel strongly positive about the effects of technology on their educational experiences;
- students find WebCT sites to be the most useful for their coursework of any technology listed on the survey; and
- students find that procrastination is a significant problem when doing online coursework.
Events
Our consultants discussed TEL evaluation methods such as those described in our Evaluation Methods guide in our recent TEL short courses and at meetings with our DMC faculty fellows and TEL Grants Program participants.
Campus Projects
Information about the following campus educational technology evaluation projects is available on our site.
- View University of Minnesota faculty and student technology surveys and reports about the results.
- Read profiles of the 2003-04 DMC faculty fellows and learn how they plan to evaluate their projects.
- Find out about the outcomes of the TEL Small Grants Program projects in our The 1992-02 TEL Small Grants Program: Broadening Conversations about Teaching and Learning (PDF) report and how the 2002-03 TEL grant award winners plan to evaluate their projects in their project proposals linked from the Award Winners page.
- Read an article in the latest issue of EDUCAUSE Quarterly about the results of our efforts to evaluate the impact of the TA Web Certification Program. It was written by two of our consultants, Brad Cohen and J. D. Walker and Bill Rozaitis from the Center for Teaching and Learning Services.
- Learn about an online nursing curriculum evaluation project conducted by staff members in the School of Nursing and our consultants.
- Learn about a joint DMC/College of Human Ecology evaluation project.
Bibliography
We recommend these additional evaluation sources:
Angelo, Thomas A., and K. Patricia Cross. Classroom Assessment Techniques. 2nd ed. San Francisco: Jossey-Bass, 1993.
CATs are structured activities designed for use in formative evaluations of what and how students are learning in the classroom. Many of these techniques can be adapted for use in the evaluation of the impact of educational technology materials.
Chickering, Arthur, and Stephen Ehrmann. "Implementing the Seven Principles: Technology as Lever." AAHE Bulletin, October 1996, pp.3-6. Reproduced at http://www.tltgroup.org/programs/seven.html.
Chickering and Ehrmann's principles can be used to select classroom goals.
Doughty, Gordon, and University of Glasgow TILT Project Group E. "TILT Group E--Evaluation." University of Glasgow TILT Project. http://www.elec.gla.ac.uk/TILT/E-Eval.html.
Outlines a fairly comprehensive evaluation approach and includes links to further references.
Draper, Steve. "The Prospects for Summative Evaluation of CAL in HE." ALT-J 5, no. 1 (1997): 33-39. Reproduced at http://staff.psy.gla.ac.uk/~steve/summ.html.
Draper calls summative evaluation "the consumer view" because its point is to guide a decision about whether to "buy" the program being evaluated.
Eder, Douglas. "Teaching Goals Inventory." In Classroom Assessment Techniques, 2nd ed., Thomas A. Angelo and K. Patricia Cross. San Francisco: Jossey-Bass, 1993, p. 13-23. Reproduced at http://www.siue.edu/~deder/assess/cats/tchgoals.html.
Provides an easy way to clarify your teaching goals.
Lane, David. "HyperStat Online Textbook." davidmlane.com. http://davidmlane.com/hyperstat/.
A complete statistics course and book.
Neal, Ed. "Does Using Technology in Instruction Enhance Learning?" The Technology Source, June 1998. http://ts.mivu.org/default.asp?show=article&id=464.
A response to Jerald Schutte's 1996 study of educational technology. Read it with Schutte's article to understand some of the potential and pitfalls of comparative studies.
Niles, Robert. "Statistics Every Writer Should Know." RobertNiles.com. http://www.robertniles.com/stats/.
Statistical concepts for beginners.
Schutte, Jerald. "Virtual Teaching in Higher Education: The New Intellectual Superhighway or Just Another Traffic Jam?" Northridge, CA: California State University, Northridge Department of Sociology. http://www.csun.edu/sociology/virexp.htm.
Read it with Ed Neal's response to understand some of the potential and pitfalls of comparative studies.
University of Missouri-Columbia. "Effects and Perspectives of WebCT." University of Missouri-Columbia. https://courses.missouri.edu/info/MUstudies.shtml.
Online examples of student and faculty surveys geared toward the evaluation of technology-enhanced learning and information about a few small-scale studies.
University of North Carolina at Chapel Hill Center for Teaching and Learning. "Evaluation Issues." University of North Carolina at Chapel Hill Center for Teaching and Learning. http://www.unc.edu/depts/ctl/he1.html.
Guidelines for questionnaire construction.
U.S. General Accounting Office. "Quantitative Data Analysis: An Introduction." U.S. General Accounting Office, May 1992, GAO/PEMD-10.1.11. http://www.gao.gov/policy/10_1_11.htm.
A good general introduction to quantitative data analysis.
