An Experimental Evaluation of the Instructional Effectiveness of Student Response System: A Comparison with Constructed Overt Responding

              Teacher enhanced formative assessment (TEFA) is a learning environment that allows students to be engaged in the teaching and learning process.  Student respond to questions posed using a student response system (SRS).  These can vary from simple multiple choice single response answers or a text response of a word, phrase, or short answer.  The gathering of this response data is immediately retuned to the students, often in a histogram.  Students can observe their peers responses.  This allows students to become familiar with the fact that answers and thinking can vary.  If appropriate, grand conversation, can take place, offering students an opportunity to conduct in-depth analysis and synthesize ideas.   Finally, the correct answer is presented to anchor a concept or fact offering immediate feedback to students’ initial response to the question. 

The steps described here involving TEFA are firmly grounded in the social constructivist theory of Lev Vygotsky, (Ormond, 2003), which believes teachers should create many opportunities for students to learn with teacher and peers in co-constructing of understanding.  With the use of TEFA, students become cognitively aware of what they know and do not know by receiving immediate feedback. 

Additionally, with the use of TEFA, students become social learners through the initiation of grand conversation of a concept, which lends itself to the theory of Benjamin Boom’s Taxonomy of Learning Domains (Ormond, 2003), as students evaluate each other’s ideas in relation to the concept.  Research has presented data offering insight to this phenomenon in learning with TEFA. 

Empirical Study

In a paper presented to the International Journal of Teaching and Learning (2009), Knapp and Desrochers determined from their research that TEFA does positively affect students’ learning.  To gather this data, Knapp and Desrochers performed a real time empirical study. 

The sampling included 85 students of an introductory psychology course at a New York City college.  There were 58 female students and 26 male students. There were 79 Caucasian students, 4 African American students, one student that was identified as other, and one student that did not identify a race.  The average age of the students was 18.5 years.  The majority, 90% were in their first or second year of college.  69% of the students had a grade point average between 2.6 to 3.5 on a 0-4 scale.  

To determine which group a student was placed, the students were selected randomly by one of four researchers.   One at a time, a researcher would alternately and randomly selected a student and escorted them to one of four instructional method classrooms.  The results represent a true evaluation of TEFA for this group.  The reliability was ensured by video taping instruction and assessment.  Then reviewed for reliability. 

First, formative assessment was conducted during learning were students in one control groups could respond to questions using TEFA with SRS and in a second control group students used TEFA were students received immediate feedback.  Feedback included class responses displayed in a histogram, so the students could see how everyone answered.  Then the correct answer was presented to the students. 

After the lesson all students were given a posttest.  The posttest gains of the students that had TEFA of SRS condition (M = 64.3%, SD = .14, SE +/- .03) were significantly different from the posttest scores of the passive condition (M = 55.9%, SD = .11, simple contrast, F (1.80) = 6.0, p = .01), which experienced the same lesson with out any TEFA, (simple contrasts, F (1,80) = 5.0, p < .05).  The gains of the students using TEFA were determined by identifying answers given incorrectly during formative assessment at the time of instruction and then identify the answer given correctly for the same question on the post assessment by that student. Then, the gains of the control groups using TEFA were compared to the passive control group, which did not use TEFA.  This comparison identified a statistical difference between students taught using TEFA and students that are taught without TEFA.  These gains make evident that TEFA does improve learning.

The sampling procedure used in this study was very small with just 85 students.  The students were mostly female, simple by chance, but not very divers.  The students do not represent a diverse ethnicity with a predominantly Caucasian group of students.  Additionally, this study was conducted at one New York college one time.  Nor does it represent any geographical cultural differences.  

Finally, the conditions of the experiment were not conducted in a traditional classroom.  The conditions were controlled instruction delivered step by step to insure reliability and consistency.  The results may vary in a traditional classroom. 

To expand on these findings more research would need to be done with a more diverse population in a variety of geographical areas to avoid bias.  

Knapp, F., & Desrochers, M. (2009). An experimental evaluation of the instructional effectiveness of a student response system: A comparison with constructed overt responding. International Journal of Teaching and Learning in Higher Education, 21(1), 36-46.