Best Poster Award Nominee
101 – Implementation and evaluation of changes to the PEER assessment process
Debra Barnette
Ohio State University
Background: For 5 years our Pharmacology and Therapeutics course has utilized Team Based Learning (TBL) for our disease-state workshops. Each 60-student workshop is structured with an individual and team Readiness Assurance Process, followed by application mini-cases. An essential component in the TBL pedagogy is the completion of PEER assessments to maintain student accountability to the team.
Methods: During the second semester of the past year we implemented a different PEER evaluation process. Our original process was based on open ended questions with minimal guidance (i.e., no specific criteria or rating scale) for completing the PEER evaluations. Our goal was to improve the PEER assessment process by 1) incorporation of objective criteria with rating scales to guide student peer to peer quantitative evaluation, 2) require constructive feedback statements for open ended questions and 3) require self-assessment (reflective) evaluation. Self-reflection was presented as key to personal growth in the course. In response to overwhelming student input, we also implemented an option for each student to reward team members for outstanding contributions with additional”bonus”; points. The changes and incentivized system for PEER evaluations were compared to the standard open-ended four question traditional process used in the previous semester by the same students.
Preliminary Results: The revised peer to peer evaluation form with specific objective criteria significantly improved the PEER evaluation process as assessed by overall number and quality of student responses. The optional “bonus” points for team members were extremely well received and utilized by over 90% of the students. Overall, students perceived the “bonus” system as more equitable and less punitive compared to the first semester process. Students commented that the objective criteria provide a better framework for students to quantitatively compare each student’s team contributions. The self-assessments were enhanced as more students provided detailed comprehensive self-reflections.
Conclusion: The PEER review process may best be performed with a structured form that provides guidance for the students to provide feedback in TBL pedagogy. Future work should focus on developing reflective skills and assessment of the benefits in the TBL classroom.
102 – Team-Based Learning in Communication Courses for Accounting Graduates: Investigating Student Engagement, Accountability and Satisfaction
Judith Ainsworth
Fox School of Business, Temple University
Studies of Team-Based Learning (TBL) have found positive effects on learning outcomes, higher academic performance, learner engagement, and information comprehension and retention. Students believe that participating in TBL classes has helped develop more effective learning styles and increased confidence in their knowledge and decision-making (Oldland, 2015). However, the majority of these studies have been carried out in health education. TBL is less commonly used to teach business (Bobbitt, Inks, Kemp, & Mayo, 2000; Hernandez, 2002; Roebuck, 1998), rarely to teach accounting (Caldwell, Weishar, & Glezen, 1996; Cottell & Millis, 1993; Van der Laan Smith & Spindle, 2007).
Therefore, this study investigated the impact of TBL on student engagement, accountability and satisfaction in writing communication courses for accounting graduates. We hypothesized that students would:
1) be accountable to their team members and prepare better for the Readiness Assurance Tests (RATs)
2) participate actively and engage with team members during Team Application Exercises
3) meet the challenges of TBL classrooms and be satisfied with their learning outcomes
The study was carried out in two sections of the graduate course Professional Writing for Accountants. The course used the three-phase TBL method with a flipped classroom. Application exercises included discussing examples of accounting and audit reporting, revising and rewriting poorly written messages, reporting analyses to the class, and writing professional accounting documents.
We collected data from mid-module and end-of-module team peer evaluations (Michaelsen, Davidson, & Major, 2014); Team-Based Learning Student Assessment Instrument (Mennenga, 2012); individual (iRAT) and team (tRAT) scores, including item analysis of individual multiple-choice questions to determine Difficulty Factor and Discrimination Index (Mott & Peuker, 2015); and brief end-of-module team interviews.
Our analyses indicate that the majority of accounting graduates felt accountable to themselves and to their team, showed a preference for TBL compared to lecture instruction, and were satisfied with TBL and the course.
103 – Exploring Computer-Based and Paper-Based Perceptions of Team-Based Learning’s Readiness Assurance Process
Bridget Moore, Jennifer L. Styron, and Ronald A. Styron, Jr.
University of South Alabama and Eastern Virginia Medical School
BACKGROUND: Team-based learning has proven to be an effective instructional strategy in physical classrooms. As a result, there is a growing in interest of utilizing team-based learning in online learning environments. With the increased demand for technology integration in the classroom combined with the growing interest for tools that facilitate team-based learning in an online environment, a number of technologies are emerging to assist faculty in the TBL process. The purpose of this study is to determine whether a student’s (1) computer attitude score or (2) preferred learning modality impacts their preference of computer-based and paper-based Readiness Assurance Tests.
DESCRIPTION: Students from an undergraduate nursing course were solicited for participation in the 2017 summer semester. This class is a required course for undergraduate nursing students and is taken in the professional component of their curriculum (junior or senior level years). Students were introduced to the study at the beginning of the semester and asked to complete the pre- and post- collaborative and critical thinking survey regarding the effectiveness of the project. Students received an informed consent document along with a copy of the Computer Attitude Survey (CAS) and Visual, Aural, Read/Write, Kinesthetic (VARK) tools. The CAS and VARK tools were used to identify computer attitudes and learning styles and to determine if their differences exist in RAP learning preference (computer-based or paper-based) based on these items.
RESULTS: Data for this study will be entered into the statistical analysis program called SPSS and analyzed using this software. Descriptive statistics and frequencies will be reported for each survey. To determine differences among variables of interest, researchers will utilize a one-way ANOVA for (1) pre- and post-test collaborative and critical thinking scores based on progression in a team-based learning course; (2) RAP scores based on delivery format; and (3) RAP preferences identified based on students’ self-perceived computer attitudes and learning styles.
To explore student’s perceptions of team member effectiveness one open ended question focused on the learning experiences and student perceptions of team member effectiveness will be analyzed by using a selective coding technique to develop topical categories for each qualitative response set and a nominal ordinal method recording the relative frequency for each response category to quantify responses (Trochim, 2006). Data will be transcribed and coded to determine if any themes exist, thus providing insight to the phenomenon surrounding responses. These responses will also be compared with current literature associated with team-based learning and effective instructional strategies. Scores on each of the five domains (contributions to the team, keeping on track, expecting quality, and having knowledge/skill/ability) will be analyzed to determine differences and correlations of team member attributes.
CONCLUSION: The primary benefit of the study will be a greater understanding of students’ preferences in the readiness assurance process utilized in team-based learning. Additionally, results of this study will help inform decisions pertaining to which instructional strategy is most effective.