302 – Forming Team-Based Learning Teams with Kolbe A™: The Effect of Team Construction and Training about Conation on Team Satisfaction
Staci N. Johnson, Eliza D. Gallagher, and Claire L.A. Dancz
Southern Wesleyan University and Clemson University
The ability to work in teams is an essential skill for college graduates to possess as they enter the job market. Team-based learning (TBL) course design attempts to capitalize on the benefits of team work while providing frameworks intended to decrease the likelihood and severity of social loafing. Evidence suggests that this course design is not sufficient to overcome all negative practices and opinions for students working in teams. Student attitudes toward team work and assignments within TBL courses have been previously shown to be positive5, but research findings have shown that there is still room for improvement.4 This study, conducted during the Fall 2017 semester, will examine the impact of utilizing Kolbe A™ Index scores for team formation, along with training about conation, on student perceptions of their team. This research will be conducted as a mixed-methods study with a concurrent triangulation design. The quantitative portion of this research will use quasi-experimental test-retest design. Since the objective of this study is to determine if the awareness of and training about student Kolbe A™ score affects the perception of team work, students will be surveyed prior to and following the release of results and the training module utilizing the “Feelings About Teams” survey. The qualitative portion of this study will consist of ethnographic case study data collected from a reflective writing assignment near the end of the semester. This data will allow for triangulation to provide a more clear and thorough understanding of how students are perceiving their team experiences. Based on previous research utilizing Kolbe A™ in academic settings, we expect improved student perceptions of team work throughout the semester and additional gains following team training. The complete results from this study will be available at the conference.
Best Poster Award Nominee
303 – Student characteristics and their association with attitudes toward team-based learning, team productivity, and learning outcomes: An exploratory study
Heather J. Hether
University of California, Davis
Background: Research suggests multiple variables are associated with student perceptions of TBL and its effectiveness in facilitating learning; however, more research is needed to validate and extend previous findings into new contexts and disciplines. In particular, research is needed across a broad range of disciplines beyond health and medical programs from which many studies have originated.
Aims: This study examines how gender, ability, motivation, self-efficacy, and extroversion are associated with Communication students’ attitudes toward TBL. The study also examines how these factors are associated with students’ perceptions of team productivity and how team productivity is associated with student attitudes and grades.
Methods: After receiving IRB approval, students in two Communication courses completed an online survey. Each course was taught by the same instructor and applied TBL practices. The sample size is N = 101 students, with a response rate = 91%. Independent variables included gender, GPA, motivation, self and team efficacy, extroversion, and team performance. Dependent variables included student perceptions of TBL and learning outcomes. Statistical analyses were conducted using SPSS v. 20.
Results: Self-efficacy, intrinsic motivation, and extroversion were significantly positively correlated with student perceptions of TBL, while student ability was negatively correlated with perceptions of TBL. Gender had no significant associations. Team performance was positively correlated with self-efficacy, collective efficacy, extroversion, motivation and perceptions of team-based learning. Team performance was not correlated with higher grades.
Conclusion: Significant associations were found between student characteristics and positive attitudes toward TBL and team productivity. Findings suggest faculty should reach out to multiple student sub-groups, such as those who are extrinsically motivated and higher ability, to demonstrate the value of TBL and to cultivate team performance. This study also indicates the potential of TBL as a pedagogical strategy for faculty in the social sciences.
304 – A Student-Centered Approach to Peer Evaluation in Team-Based Learning
Lauren R. Brannan and Hannah D. Szatkowski
University of South Alabama
BACKGROUND: One important component of Team-Based Learning (TBL) is the process of peer evaluation. This process ensures that each team member is held accountable for contributing to team activities and discussions. After applying several approaches to peer evaluation, including paper-based and CATME, we still received complaints from students about the grading procedures being unclear or unfair. We expanded on the idea of having students set their own grade weights and sought to remediate the issue by empowering the students to create their own instrument for peer evaluation. The purpose of this presentation is to inform the audience of an alternative, student-centered approach to peer evaluation in their TBL courses.
DESCRIPTION: This study was conducted over one semester with 36 preservice teachers enrolled in two sections of a junior level reading methods course. On the first day of class, students worked in their teams to develop the instrument to be used for peer evaluation in the course. In their permanent teams, students first discussed the criteria that makes an effective team member; then as a whole class, compiled the list of criteria on the board and eliminated any duplicates. Once again in their teams, students discussed the set of criteria on the board and proposed an official list for the instrument. Finally, as a whole class, teams discussed and determined the official criteria and the rating scale to be used for evaluating their peers. The professor recorded the official decision, then converted it into a Google form for each student. The link for each student’s evaluation was posted in our learning management system for each team to complete. This approach to peer evaluation placed the responsibility in the hands of the students to generate criteria that were meaningful and grading that was transparent to them. At the end of the Fall semester, students will participate in an electronic survey that will ask them to compare their experience with peer evaluation in this course, to their previous experiences with peer evaluation in a TBL course.
CONCLUSION: Conclusions will be drawn upon completion of data collection and analysis (Fall 2017).
305 – A Study of the use of Team-Based Learning to Deliver a Consultation Skills Course
Simon Tweddell
University of Bradford
Background: In 2012 Bradford School of Pharmacy introduced Team-Based Learning (TBL) to deliver a final year Consultation Skills course to MPharm students to motivate them to prepare for classes and engage them in higher level critical thinking during class. TBL is form of collaborative learning that uses a special sequence of individual work, group work and immediate feedback to create a motivational framework in which students increasingly hold each other accountable for coming to class prepared and contributing to discussion, It is grounded in constructivist educational theory with students engaging with one another to solve authentic problems (Hrynchak & Batty, 2012)
Method: Whilst teaching the course, student feedback was collected from 75 students (85%) relating to their accountability, preference for, and satisfaction with TBL using the Team-Based Learning Student Assessment Instrument (TBL-SAI) (Mennenga, 2012). The end of module assessment results for 2 cohorts of students studying TBL was compared with those from 2 cohorts of pre-TBL students. Finally a student-led focus group of 12 students recruited from the cohort was conducted to determine student’s opinions on TBL.The results were transcribed and analysed using thematic analysis to identify common themes.
Results: Results from the TBL-SAI instrument showed student preference for and satisfaction with TBL as a method for the delivery of teaching. Additionally, the results also showed that students developed accountability to their team; a key pedagogical principle of TBL. A comparison of their assessment marks showed an increase of 13% in the cohorts learning using TBL (n=192) compared with those being taught pre-TBL (n=173). Results from the focus groups were positive with students enjoying the active learning, interactions, and challenging activities. Suggestions to improve included managing timings and better facilitation.
Conclusion: TBL was well received by Stage 4 MPharm students and it looks a promising pedagogy for delivering MPharm courses.
306 – Student Experiences with Team-Based Learning Over Time: A Pilot Study
Ron Carson and Heidi Mennenga
Adventist University of Health Sciences and South Dakota State University
BACKGROUND: Despite evidence showing that team-based learning (TBL) is effective and well-accepted by students, there is a gap in the evidence regarding how students’ experiences with TBL may change over time.
AIM: This study explores changes in accountability, satisfaction, and preference for TBL or traditional lecture among occupational therapy students using TBL across one year of occupational therapy education.
METHODS: A cohort design was utilized to determine changes in students’ experience with TBL. Students were asked to complete the “Team-Based Learning Student Assessment Instrument” during their first term and at the end of their third term of occupational therapy education. Data was collected and analyzed from two cohorts, with a total of 41 paired data sets.
RESULTS: Overall, students reported they were generally accountable with TBL, satisfied with TBL, and indicated a preference for TBL over traditional lecture. Cohort 1 was significantly more satisfied with TBL (p= .021) at the end of their third term. Cohort 2 was significantly more accountable (p= .027) and had an overall more favorable experience with TBL (p= .019) at the end of their third term. When the data from the cohorts were analyzed together, no significant differences were found.
CONCLUSION: Significant differences were found with individual cohorts regarding student satisfaction, accountability, and the overall TBL experience over time, however, no significant differences were found when the cohorts were combined. These results may indicate that individual cohorts of students experience TBL differently over time. Additional research should be done to review student experiences with TBL over time, perhaps over a longer time period, and/or look at additional variables, such as gains in student knowledge. The generally positive experience with TBL that students initially reported is encouraging for instructors who are newly implementing TBL and continued validation for experienced instructors already using TBL.
307 – Does Class Size Affect Student Perceptions of Team-Based Learning?
Heidi Mennenga and Marit Ostebo
South Dakota State University and University of Florida
BACKGROUND: An assertion that has fueled increased interest in team-based learning (TBL) is the idea that it creates an interactive and engaging learning environment even in large classes with several hundred students. However, limited research exists regarding how class size may impact the success of TBL.
AIM: This study explores differences in accountability, satisfaction, and preference for TBL or traditional lecture among students using TBL in small, medium, and large classes.
METHODS: Students in a large religion class (n=120), a medium population health nursing class (n=69), and a small human rights class (n= 34) were asked to complete the “Team-Based Learning Student Assessment Instrument” and open-ended questions. Quantitative results were analyzed and compared; qualitative results were also analyzed.
RESULTS: All three classes were generally accountable with TBL, satisfied with TBL, and indicated a preference for TBL over traditional lecture. The small sized class was the most satisfied with TBL, preferred TBL over lecture the most, and overall had the most favorable assessment of TBL when compared to the medium and large sized classes. Qualitative responses confirmed overall student satisfaction, and reflected particularly strong support for the social and interactive aspects of learning.
CONCLUSION: While TBL can be used successfully in large classrooms, it may be beneficial for educators to understand that the student experience may differ based on class size. Educators should recognize that students’ perceptions of and experience with TBL may be influenced by the relationship formed with the instructor, which is often affected by class size. In smaller class sizes, the instructor is often able to develop closer relationships with students versus in large class sizes. The relationship between educator and student and the impact on the TBL experience may be something to consider in future research.
308 – A New Evaluation Tool to Assess Facilitators in Team-Based Learning Classrooms
Ruth E. Levine, Peggy Hseih, P. Adam Kelly, Lisa Carchedi, Jennifer Gibson, Paul Haidet, Paul Koles, Lindsey Pershern, Dawnelle Schatte, Brenda Talley, Dwight Wolf, and Britta Thompson
McGovern Medical School, Tulane University School of Medicine, University of Maryland, Upper Chesapeake Health Systems, Penn State College of Medicine, Wright State University Boonshoft School of Medicine, University of Texas Southwestern School of Medicine, University of Texas Medical Branch, University of Texas Health Science Center at San Antonio
Background: Measurement of teaching quality is important for providing faculty feedback and improving instruction. Many tools exist to evaluate lecture quality and small-group instruction. However there are no published, validated tools to date developed specifically to evaluate the quality of instruction in Team-Based Learning (TBL) classrooms.
Summary of work: We convened TBL experts and students to craft an 11-item student survey which we administered in TBL classrooms at seven School of Medicine sites (n = 2,840 students). We also administered a facilitator questionnaire to each facilitating faculty assessing self-confidence, experience with, and enjoyment of TBL and demographics (n = 32 faculty).
Discussion: Students in large year 1 and 2 classes rated their facilitators significantly lower than students in small year 3 classes (p < .01, ?2= 0.098). Qualitative comments on the surveys were quantified and results reinforced quantitative findings, with year 1 and 2 student evaluations having equal numbers of positive and negative comments (46% to 53%) and year 3 evaluations having predominately positive comments (74%) compared to negative comments (25%).
Conclusion: Our survey provided a reliable tool for evaluating the quality of Team-Based Learning (TBL) facilitation, with different findings in large vs small classes. A variety of potential reasons may account for these findings including gender differences among facilitators, content differences between type of subjects being taught, student maturity, distraction associated with competing priorities, and class size. Our conclusions were that many factors may have contributed to our findings, but in particular, class size may be particularly important. Further analysis is needed to understand the difference in performance of this instrument in different settings.
309 – Promoting clinical decision making and teamwork in undergraduate nursing education: A mixed methods evaluation of team-based learning in applied pathophysiology
Jonathan Branney and Jacqueline Priego-Hernandez
Bournemouth University
Introduction: It is important for nurses to have a thorough understanding of the biosciences such as pathophysiology that underpin nursing care, but they can be difficult to learn. Team-based learning (TBL) is emerging as a strategy for enhancing learning in nursing education due to its promotion of individual learning as well as learning in teams.
Aim: The aim of this study was to evaluate the use of TBL in the teaching of applied pathophysiology to undergraduate student nurses.
Methods: In a year two undergraduate nursing applied pathophysiology module circulatory shock was taught by TBL, while all remaining topics were taught by traditional lectures. Students were then invited to complete the Team-based Learning Student Assessment Instrument (TBL-SAI) which measures accountability, preference and satisfaction. They were also invited to focus group discussions (FGDs) to gain a more thorough understanding of their experience with TBL. Exam scores for answers to questions based on TBL-taught material were compared with those from lecture-taught material.
Results: Of the 197 students enrolled on the module 167 (85%) returned the TBL-SAI, and the results indicated a favourable experience with TBL. Most students reported higher accountability (93%) and satisfaction (92%) with TBL. Lectures that promoted active learning were viewed as an important feature of the university experience which may explain the smaller majority (76%) exhibiting a preference for TBL. Most students wanted to make a meaningful contribution so as not to let down their team and they saw a clear relevance between the TBL activities and their own experiences of teamwork in clinical practice. Exam scores on the question related to TBL-taught material were comparable to those related to lecture-taught material.
Conclusion: Most students preferred, and reported higher accountability and satisfaction with, TBL. Through contextualisation and teamwork, TBL appears to be a strategy that confers strong pedagogical benefits for teaching applied pathophysiology (bioscience) to student nurses.
310 – Get Out of the Lecture Hall! Assessment of Student Learning and Communication Skills Moving from Lecture to Team Based Learning
Cynthia Standley and Paul Standley
University of Arizona College of Medicine-Phoenix
BACKGROUND: Active learning is an instructional method where students work collaboratively with peers to discuss information in a way that enhances self as well as group learning. Team-Based Learning (TBL) is a structured form of active learning that emphasizes student preparation out of class and application of knowledge in class. At UA COM-P, moving from a traditional, lecture-based scenario to a more active learning TBL paradigm enabled research to measure the impact on student outcomes. We hypothesized that moving from lecture to a combination of independent learning modules (ILM) plus TBL improves short-term medical knowledge mastery and immediate and long-term gains in interpersonal and communication skills.
DESCRIPTION: Our College delivers almost all instruction via a combination of lectures, case-based learning, simulations and patient panels in the preclerkship curriculum. Curricular seat time is limited to 26 hours weekly (via policy), with an approximate national average lecture time equating to about 55-60%. In AY2016-2017, the Curriculum Committee instituted a directive to course directors to convert 5% of lectures in the fall semester and 10% of lectures in the spring semester to active learning sessions, along with increases planned for subsequent years. While many of these active learning sessions are non-mandatory, students are expected to master the content by the time it is assessed via multiple choice exams, whether they were in attendance or not.
RESULTS: Institutional Review Board approval was granted in 2016 and to date we have consented 64% of Year 1 students and 76% of Year 2 students for a total research cohort of 112 students. We have acquired attendance data beginning this year using a card scanner, and built the data file to examine outcome performances on exams and correlating it with attendance. Currently we are working with each of our 9 course directors to convert lectures to TBL experiences, thus building on the lecture-to-active learning conversion.
CONCLUSIONS: We have observed increased student participation at non-mandatory TBL sessions versus historical lecture attendance and students are substantively more engaged with this instructional method switch. In addition to measuring the effects on course exam questions, we are building the processes necessary to measure how this switch affects interpersonal and communication skills as assessed in clerkships and other Years 3 and 4 experiences.
312 – What is the Correlation Between Reading Completion and RAT scores?
John Chapman
Southern Virginia University
New learning data available from online textbook platforms can lead to improvement in RAT question development. A first step toward understanding and eventually improving the quality of RAT questions is to examine the relationship between RAT scores and the levels of student reading completion.
This research incorporates data from an online textbook platform which calculates the amount of text a student reads based on how many paragraphs were displayed to the student. The platform can distinguish between scanning and reading activity based on time-on-paragraph measures. Using paragraph-level data the platform records both the quantity of reading completed by the student, and the quantity of time the student spent reading.
The platform displays to the instructor a dashboard detailing student reading times and amounts for each chapter. Each chapter can have up to 12 different sections. And each section is a separate page in the online textbook. This research will calculate correlations of reading percentages and reading times at three different levels, including a) the chapter level, b) the section level, and c) the paragraph level with the corresponding student’s RAP individual and team score. It is anticipated that higher reading time and reading amount will lead to improved RAP scores. This research will identify correlations at the chapter level, the section level, and the paragraph level for both reading time and reading amount.
Best Poster Award Nominee
313 – Improved Growth Mindset in Statistics for Psychology Students Following Implementation of a TBL Curriculum
Sarah Strand
California State University, Sacramento
INTRODUCTION: Undergraduate Psychology students at California State University, Sacramento (CSUS) are required to take an introductory inferential statistics course (PSYC 101: “Statistics for Psychology”). Historically, standard lecture-exam curriculum was used and resulted in high fail rates. To reduce the high fail rate, I flipped the class using team-based learning (TBL) curriculum.
AIM: Students in the standard curriculum focused on memorization (which is easy and familiar) rather than critical thinking and quantitative analysis. My goal was to use TBL to scaffold the development of skills that students were reluctant to practice on their own. In so doing, I predicted students would show improvements in critical thinking and increased confidence in their hard, albeit uncomfortable, efforts to succeed in statistics.
METHODS: During the Fall 2016 semester (duration of 16 weeks), three assessments examined efficacy of TBL (N = 41 students) compared to standard lecture-exam curriculum (“Control,” N = 37 students). A pre-post survey developed by James Stigler assessed Quantitative Reasoning. A pre-post survey developed by Carol Dweck assessed Growth Mindset. The final exam assessed retention. The surveys were given during the first and last week of the semester. The final exam was administered during finals week.
RESULTS: In the Growth Mindset survey, the “time” (Pre-Post) by “section” (TBL vs Control) interaction was statistically significant (F1,76 = 4.011, p < 0.05), with the TBL section showing an increase and the control section showing a decrease in growth mindset. There were no significant differences in the Quantitative Reasoning survey or on final exam performance. However, in the Quantitative Reasoning survey, the TBL section showed 3.5x greater increase in scores compared to the control section (TBL: increased 0.83 points; Control: increased 0.24 points).
CONCLUSION: TBL significantly increased growth mindset and showed a strong positive influence in quantitative reasoning skills compared to standard lecture-exam curriculum
314 – Applying Team-Based Learning in a General Physics Course
Jo-Chi Jao
Department of Medical Imaging and Radiological Sciences, Kaohsiung Medical University, Taiwan
Introduction: Team-Based Learning (TBL) is known as an effective student-centered teaching method for large size of classroom and has been widely performed on courses of medical or health sciences. General physics is a fundamental course in college. The content of general physics is broad and the textbook of general physics is thick. It is a challenge for both teachers and students to accomplish general physics within limited teaching hours. The aim of this study was to investigate the effectiveness of TBL on academic performance in a general physics course.
Methods: Three TBL modules about Mechanics were performed over the fall semester in 2014 in a medical university of Taiwan. Students were asked to read materials from a textbook and a MOOC: Easy to learn Mechanics on Ewant platform. An individual review test (iRET) was performed at the end of the class. The scores of individual readiness assurance test (iRAT), team readiness assurance test (tRAT), and iRET of forty-one students on the topic of “Oscillation” were evaluated with paired t-test statistical analysis.
Results: The mean score of tRAT (97.9) was significantly higher than both the mean scores on iRAT (51.6) and iRET (93.4) (p < 0.05). The mean score of iRET was much higher than that on iRAT (p < 0.05).
Conclusions: The results demonstrate that students has positive learning outcome on the topic of “Oscillation”. Students can learn more from team discussion. Some misconceptions can also be realized and corrected through teamwork. This finding will help the instructor of the course to have better applications of TBL on the general physics flipped classroom in the future.
Best Poster Award Nominee
315 – TBL Based Integration of Clinical Situations to Enhance Nursing Student Perceptions of the Elderly: A Mixed-Methods Study
Theresa F. Wright, Colleen M. Lynch, Debra Swanzy, Kimberly Jordan, and Julie M. Estis
University of South Alabama
Background: The rapidly growing elderly population increases the demand for nurses prepared to care for the aging population. Care of the elderly is often not chosen as a career path by nursing students due to negative attitudes toward the elderly and thoughts that providing care to the elderly is boring and lacks challenge.
Purpose: The purpose of this research was to determine the impact of Team Based Learning (TBL) pedagogy applied in an elderly community-based setting on nursing student perceptions of the elderly.
Methods: A mixed method, pretest/posttest design was utilized in this pilot study of 63 nursing students enrolled in a Bachelor of Science in Nursing program. Students completed an IRAT and TRAT based on pre-assigned material, participated in a 4S exercise, and traveled to local community senior centers to complete application activities. Data was gathered using the Image of Aging Scale, course evaluations, and writing prompts. Qualitative data was analyzed for themes and linguistics.
Results: Students reported a decrease in negative perceptions of the elderly (p<.0001) and an increase in positive perceptions (p<.0001). The focus on physical, mental, and physiological declines associated with aging were lessened in the post experience. These qualitative findings were supported with linguistic outcomes. TBL themes were positive: students enjoyed teamwork; improved team and communication skills; and improved learning, comfort, and enjoyment related to working with the elderly. TBL themes were supported with linguistic analysis revealing an increase in confidence (p=.0005); improved quality of relationships (p=.017); and increased positive emotions (p=.019).
Conclusions: Teamwork and application aspects of TBL were found to be most beneficial in positively impacting student perceptions of the elderly and the TBL pedagogy. TBL should be considered as a viable method to foster a desire for nursing students to care for the elderly and enhance learning outcomes.
Best Poster Award Nominee
316 – The Impact of TBL on Sociology Undergraduate Students: Does TBL Minimize the Gender Gap in Comfortability with Research?
Aya Ida, Todd Migliaccio, Patricia Morris, Yusuke Tsukada, and Dylan Baker
California State University – Sacramento
BACKGROUND: Research methods class has been often identified as one of the most difficult, intimidating classes by undergraduate sociology students. With quantitative skills often being associated with masculinity than femininity, a gender gap could exist in perceived difficulty of the class. Based on the idea of stereotype threat, this study aimed to test the hypothesis that men would have more comfortability with research than women at the beginning of the class, but such gender gap would disappear by the end of semester in TBL class, but not in traditional lecture class.
METHODS: Using quasi-experimental design, this study collected data from undergraduate sociology students who were enrolled in research methods classes in fall 2016, spring 2017, and fall 2017 (n=157). In each semester, data were collected on the first and last days from two separate classes; one of the classes adopted permanent teams and team activities in spring 2017 and implemented full TBL in fall 2017, while the other class remained lecture-based. We modified an existing questionnaire measuring comfortability with statistics to capture comfortability with research methods (53 items).
RESULTS: Based on the data collected in fall 2016 and spring 2017, the study found that men reported significantly higher comfortability with research than women at the beginning of the semester. However, compared to the class without teams, the class with teams not only closed the gender gap, but also increased women’s comfortability with research in greater magnitude thereby exceeding men’s comfortability by the end of the semester. Including data from fall 2017, this presentation would provide the additional findings comparing traditional lecture class, class with teams, and full TBL class.
CONCLUSION: Findings indicated that TBL could not only reduce the gender gap, but also help female students feel more comfortable with research potentially due to more opportunities to challenge stereotype threat.
317 – Increasing student engagement and grade differentiation in an elective course on Developmental Biology by introducing TBL
Jolanda Mol, Peter G.M. de Jong, A.J. de Beaufort, and Beerend Hierck
Leiden University Medical Center
Introduction: In the third year of the Biomedical Sciences curriculum at Leiden University Medical Center (LUMC) students are offered a series of elective courses to prepare for biomedical research practice. These electives typically take 3 to 6 weeks and use a traditional teaching model. In previous runs of the 3 week course “Research Applications of Developmental Biology” student engagement and differentiation in final grades were both low. The course has been transformed into TBL format to address both issues.
Course description: The revised course ran in July 2017. In total 4 TBL modules of 3-4 days each were implemented. Each module included preparation, iRAT/tRAT, appeal and seminar, application sessions and peer feedback/assessment. The final grade was the weighted average of 20 partial marks. On the first day of the course the teaching method was introduced as the students had no prior experience with TBL. 11 Students signed up for the course, 9 female and 2 male, aged 22.5 (+/-1.36). 3 Teams were engineered with maximal heterogeneity in gender, age, background, and mean grades. Experiences of students as well as teachers were collected by survey.
Results: Students reported the course to be extremely engaging and good for their understanding of the course content. Workload increased to 120 hours, compared to 75-90 hours in previous runs. The teaching staff reported a high quality of understanding in the students at the end of each module, exceeding the level in previous runs. Students used peer assessment strategically to compensate for low iRAT marks. The final grades were 8.4 (+/- 0.2) on a scale of 0-10.
Conclusions: Student engagement increased significantly. The TBL format did not lead to more differentiation in final grades, partly due to the ineffective peer assessment. The TBL format will be further improved for future runs of the course.
318 – Determination of whether implementation of mini-application exercises can enhance individual student performance in a TBL setting
Ruth Vinall and Eugene Kreys
California Northstate University College of Pharmacy
Introduction: A common concern of TBL is a tendency of some students to depend on team members to answer application exercise questions resulting in inadequate preparation for higher Bloom’s taxonomy questions on exams. The objective of this study was to determine whether implementation of individual applications (iBATs) improves student performance.
Methods: Ninety-eight first year pharmacy students taking the Cell and Molecular Biology & Biochemistry course participated in an unblinded, randomized, controlled, cross-over study. The course consisted of three blocks: block 1 served as a control, iBATs were administered for the two randomized groups in a cross-over manner for only blocks 2 and 3. An independent t-test was used to compare average block exam scores between groups. A paired t-test was used to compare average group z-scores between block 1 exam results and results of the block coinciding with iBAT implementation. A subgroup analysis based on the Bloom’s taxonomy of exam questions was performed. A questionnaire assessed students’ impression of iBAT impact after each block exam.
Results: Exam scores for block 1, 2, or 3 were not significantly different between randomized groups (p-value = 0.35, 0.11, and 0.82, respectively). Subgroup analysis revealed similar results. Implementation of iBATs resulted in an increase of 0.091 z-scores, but was not statistically significant (p= 0.35). Limited to critical thinking questions, an increase of 0.102 z-scores was observed (p= 0.36). Majority of students reported that iBATs increased their ability to identify knowledge gaps (88%), determine their proficiency of critical thinking questions (91%), and increased studying following poor iBAT performance (83%), while few felt that iBATs increased stress (30%), or reduced enjoyment of the course (46%).
Conclusions: Implementation of iBATs did not significantly improve student performance on examinations or specifically on critical thinking exam questions. Generally students felt that iBATs had a positive impact on their learning.
319 – A Modified TBL Format Used to Improve Knowledge Acquisition and Retention of Histology Content
Dana Peterson, Fayez Safadi, and Lisa N. Cooper
Northeast Ohio Medical University
Introduction: In 2014, a curriculum re-design project provided an opportunity for faculty to incorporate a modified TBL format for histology content presented to first-year medical students. Implementation of this modified TBL format for the last three years has yielded higher student scores on the National Board of Medical Examiners (NBME) Histology and Cell Biology subject exams.
Aims: To modify the traditional format and delivery of TBL methodologies to allow 160 students to participate in multiple one hour-long sessions focused on foundational histology content that could be facilitated by a single faculty member. To assess the effectiveness of the modified TBL format on student’s performance on a nationally-normed, standardized subject exam for histology and cell biology.
Methods: Major modifications to the traditional TBL format that were incorporated in this histology-focused curriculum re-design project include:
Students are individually assessed on the pre-assigned content with a 15 question quiz that uses a student response system (“clickers”). IRAT questions are written to capture the first and second learning/teaching categories of the revised Bloom’s Taxonomy.
In addition, students are allowed to self-select team members to collaborate with during the final activity of the session – the GRAT. This too, is a 15 question quiz where students select their group answers using clickers. However, these questions are designed to challenge students to apply, synthesize and evaluate their foundational knowledge.
Results: Students scores on the NBME Histology and Cell Biology subject exam administered in 2014 and 2015, are the highest scores ever reported at our institution.
Conclusions: A modified TBL format designed for large group participation, delivered using a student response system and facilitated by a single faculty member has resulted in higher student achievement on the NBME subject exam.
320 – Impact of improving online learning methods in an online team-based learning (TBL) interprofessional education (IPE) course.
Colleen Catalano and Lisha Bustos
University of Colorado
Background: To describe the impact on student learning and perceptions in an online TBL-delivered IPE course after implementing improved online education methods.
Description: Our institution delivers an interprofessional course to medical, dental, pharmacy, and nursing students over two semesters. The course is modelled after concurrent face-to-face (F2F) sections of same course, and contains 16 online modules on three content domains (teamwork/collaboration, values/ethics, and safety/quality). In 2015, online course activities were designed to closely “mimic” F2F application activities. In 2016, we aimed to diverge from “mimicking” F2F, while achieving the same learning outcomes as F2F using modified online learning strategies. Modifications were designed to improve course collaboration methods (e.g., use of wiki-type documents and teleconferencing; reduced threaded discussions) and streamline application exercises. We compared student learning data [individual readiness assurance tests (iRATs) and final exams], and perception data (session evaluations assessing perceptions of knowledge- and application-based learning and teamwork, and final course evaluations assessing overall course perception) from the two semesters of each cohort, 2015 and 2016.
Results: Students made improvements in their mean iRAT/final exam scores with the implementation of new online education methods. Mean session scores on knowledge- and application-based learning, and teamwork increased as well. Faculty also observed an improvement in overall course evaluation scores and students reported more comfort with the TBL process.
Conclusion: Using online best practices increases the level of interaction among students and between students and instructors and is particularly important in online instruction. Modifications to improve online learning strategies in an online TBL IPE course maintained learning, and improved perceptions on course learning activities and teamwork. Changes also led to earlier comfort with the online TBL process, we believe allowing our students to focus on interprofessional, collaborative learning.
321 – TBL on a University-Wide Scale: Implementation and Impact
Julie M. Estis
University of South Alabama
Background: Since 2013, the University of South Alabama implemented TBL as a campus-wide Quality Enhancement Plan (QEP) with aims of improving student learning outcomes, achieving higher levels of critical thinking and problem-solving skills, enhancing collaboration and communication, and applying course content to real-world situations.
Methods: TeamUSA QEP implementation focused on high-quality faculty development in TBL and ongoing support for interested faculty. An early adopter model was used to identify and train faculty from varied disciplines who became project champions. More than 300 faculty completed intensive TBL workshops focused on essential elements of TBL: preparation, IRAT, TRAT, 4S application activities. Approximately 85 faculty utilized TBL each semester. Pre/post-semester student surveys, course grades, withdrawal rates, faculty surveys, and student learning outcome reports are collected each semester to evaluate program outcomes in a mixed method approach. TBL outcomes were compared to courses not utilizing TBL.
Results: To date, student collaboration (p<.001, n=4382) and critical thinking (p<.001, n=3172) measures significantly increase from pre- to post-TBL course. TBL courses yielded significantly fewer D’s and F’s and more A’s and B’s than courses that do not use TBL. Persistence is also higher for TBL courses, with a withdrawal rate of 3.85% for TBL courses compared to 7.98% for courses not using TBL. Faculty consistently report increased critical thinking, student collaboration, and engagement in their courses. Higher level student learning outcomes are reaching mastery levels of 89% or higher for TBL courses. On a scale of 1= strongly disagree to 5=strongly agree, the mean level of faculty satisfaction in using TBL across 4 years is 4.4 (SD=0.78).
Conclusions: TBL effectively improves collaboration, critical thinking, and student learning across various disciplines and levels of instruction. This degree of project success is directly linked to thoughtful change execution, effective communication, experiential professional development, and meaningful assessment.
322 – What happens when you try something new in the classroom? Student Impressions of Team-Based Learning
Julie M. Estis and Cecelia Martin
University of South Alabama
Background: Team-Based Learning (TBL), the focus of the University of South Alabama’s (USA) Quality Enhancement Plan (QEP), has been used since 2013 to improve student learning by increasing critical thinking and collaboration. Faculty considering a new pedagogical approach are often uneasy about student perceptions and wonder if teaching evaluations will be impacted. Faculty express concerns over student evaluations of teaching (SET) for two basic reasons. SET are often the only method of teaching evaluation, impacting promotion and tenure decisions (Miller & Seldin, 2014). Second, graduate education is typically focused on scholarly practices with limited training on pedagogical strategies that differ from how they were taught (Oleson & Hora, 2013). Understandable concerns regarding how students might perceive an innovative instructional strategy emerge.
Method: SETs were conducted via university-wide electronic survey protocols at the completion of each semester. A retrospective analysis of SETs for TBL courses was completed. The data set of 930 surveys consisted of SETs from the first three years a sample of faculty used TBL.
Results: SETs for TBL courses were high overall, with means ranging from 4.19 to 4.44 (scale 1-5, 5= “very strongly agree” with positive statements) across the survey questions. Between-year differences were observed with SETs significantly increasing from Year 1 to Year 3 of utilizing TBL p=.005). Results for individual survey questions will be presented, along with individual faculty patterns of SET results before and after implementing TBL.
Conclusions: By the third year implementing TBL, faculty earned significantly higher SET scores compared to their first year. Though SETs were high for TBL faculty in their first year of implementing TBL, students and faculty may take time to adjust to a new pedagogical approach.
323 – A Structural Equation Model of Student Satisfaction with Team-Based Learning: An Analysis of the TBL-SAI
Danhong Chen, Foy D. Mills, Jr., Shyam S. Nair, and L. A. Wolfskill
Department of Agricultural Sciences, Sam Houston State University
Background: The Team-Based Learning Student Assessment Instrument© (TBL-SAI) was used to assess agribusiness students’ receptivity to TBL. Results showed that three of eight accountability measures, four of sixteen preference questions for lecture or TBL, and an individual’s contribution to the team influenced overall satisfaction with TBL.
Description: TBL-SAI was administered in two undergraduate agribusiness courses (n=371). Though the original instrument was designed to assess accountability, preference for TBL or lecture, and satisfaction, based on review of the instrument we combined some questions.
Evaluation: Confirmatory factor analysis demonstrated the reliability of our assertion. A Structural Equations Model (SEM) was used to assess the factors associated with satisfaction towards TBL consisting of attitude and effectiveness scales. Preparation, contribution, TBL/lecture distraction, and TBL/lecture recall are latent constructs measuring students’ preparation for the class, individuals’ contribution to their team, potential distractions in TBL and lecture settings, and how TBL and lecture help with recall and retention of information, respectively. Students perceived lower distraction with TBL. SEM results indicated that higher contribution, lower level of TBL distraction and higher perceived TBL recall were each significantly associated with a more positive attitude and higher effectiveness (p<0.05). Higher level of lecture distraction was positively associated with attitude (p<0.05). Previous positive group-work experience, higher expected grade, and making a close friend through TBL all contributed to higher overall satisfaction. Higher GPA was associated with lower satisfaction.
Conclusions: As a classic “adverse selection” problem, students who indicated that they prepared more or had higher reported GPAs were less satisfied with TBL. Yet, contribution to group work was considered essential by students (and employers) and was positively related to their satisfaction with TBL. Since students reported better recall and being less distracted in a TBL course, this promotes an atmosphere for higher engagement and better learning.
324 – Visualizing Mental Models in a TBL-Taught Exercise Psychology Class
Karla Kubitz
Towson University
BACKGROUND: As they learn, students form mental models of course content and the quality of their models impacts their ability to apply that content. Mental models can be visualized using online card sorting techniques. Visualizing students’ mental models in a TBL-taught Exercise Psychology class may reveal hidden confusion in application of course content.
DESCRIPTION: Fifty-four students wrote an essay-based Final Exam in a TBL-taught Exercise Psychology class. The Exam challenged them to write a 2-page essay describing an intervention designed to optimize exercise behavior. In order to visualize the students’ mental models, essays were analyzed using an online card sort technique (i.e., http://www.usabilitest.com). The sort was a closed sort with three categories, including ways to motivate, mental health benefits, and mechanisms of change. The sort included 15 cards (i.e., 5 cards per category). Analyses included the calculation of grouped percentages, distance matrices, and a multi-dimensional scaling analysis (MDS).
RESULTS: The results indicated that: (a) all five ways to motivate cards were grouped together 56-85% of the time; (b) three of the mental health benefits cards were grouped together 57-70% of the time; and (c) four of the mechanisms of change cards were grouped together 50-67% of the time. Two of the mental health benefits cards (i.e., decreased stress reactivity and increased cognitive functioning) and one of the mechanisms of change cards (i.e., social interaction) were considerably less likely (i.e., 22-28%) to be correctly sorted.
CONCLUSION: Students’ mental models of their TBL-taught Exercise Psychology course content were consistent with expectations. As predicted, they revealed several areas of confusion in application of course content. Card sorting provides a novel way to visualize students’ mental models of course material and identify areas in need of additional instruction.
325 – Students Perceive Team-Based Learning Enables Acquisition of Professional Skills
Judy Currey, Josh Allen, Julie Considine, Gabby Burdeu, Stephanie Sprogis, and Elizabeth Oldland
Deakin University
Background: Higher education is designed to teach discipline-specific skills, but professional skills commonly known as soft skills are vital to employment opportunities on graduation. Team-Based Learning may assist in acquisition of these skills.
Aims: To describe student perceptions of how learning via Team-Based Learning contributed to the acquisition of skills defined by the university’s Graduate Learning Outcomes (GLOs). These outcomes are: discipline-specific knowledge, and the professional skills of communication, digital literacy, critical thinking, problem solving, self-management, teamwork and global citizenship.
Methods A descriptive exploratory design using survey was used. Following ethics approval, first year critical care specialist nurses in the Master of Nursing Practice in 2016 and 2017 at Deakin University were invited to participate. Participation involved provision of demographics, a 300-word written reflection about TBL’s contribution to acquiring GLOs and a GLO ranking exercise.
Results: Of 165 eligible participants, 128 provided complete data for analysis; 78% response rate. Most respondents were female (87%), aged 25 to 34 (67%). Participants made 1805 positive mentions of the university’s GLOs in written reflections. Communication received the highest number of positive mentions (20.4%), followed by self-management (18.8%), and teamwork (14.9%). Digital literacy had the lowest number of positive mentions (0.5%). For GLO rankings, participants perceived TBL contributed to their acquisition of critical thinking skills the most (55.2%), followed by problem solving (34.5%), and teamwork skills (39.7%). The least ranked was global citizenship.
Conclusions: Beyond discipline specific knowledge, TBL can positively contribute to the acquisition of professional attributes highly valued in all industries. Critical thinking, problem solving, teamwork and communication skills are embedded in TBL processes. Thus, without explicitly been taught these highly valued professional skills, students believe they acquired them through TBL classes
326 – Assessing the Outcomes of Team-Teaching on Student Learning and Faculty Satisfaction
Luma Munjy and Christopher Foley
California Health Sciences University
Problem: In order to deliver a successful TBL session faculty must be effectively trained in the basic principles of TBL design and execution. Faculty workload for the first year of TBL classes is documented to be greater than traditional teaching methods. This can be especially challenging for new educators transitioning from lecture-based delivery models into TBL. Faculty at California Health Sciences University (CHSU) investigated team-teaching strategies and are currently in the process of assessing the effects on student outcomes and faculty satisfaction.
Aim: The purpose of this presentation is to present a team-teaching strategy and provide results from student and faculty feedback that demonstrate that team-teaching is at least as effective as a single-facilitator model.
Description: Two faculty members were provided with approximately 10 contact hours to facilitate a hypertension course for second year pharmacy students at CHSU. Faculty members co-designed the hypertension curriculum which included readiness assurance, 4S application and inter/intra team discussions that focused on drug information, pharmacology, clinical practice guidelines as well a thorough assessment of the literature.
Results: A total of 38 students participated in the post-survey. Results showed that 87% of students agreed or strongly agreed with the statement, “The team-based approach gave clearer and more concise explanations that improved my learning as compared to the single instructor approach.” A total of 84% of students agreed or strongly agreed that, “The team-based approach was more effective in promoting active learning as compared to the single instructor approach.” Faculty satisfaction data is currently being assessed.
Conclusion: Team-teaching appeared to have a positive effect on this group of students. We are currently experimenting with different instructor teams, classes of students, and topics to determine if instructor and classroom dynamics influence the success of team-teaching for novice TBL facilitators.