201 – Interdisciplinary Faculty Learning Community to Support Team-Based Learning across the Disciplines
Sandra Sgoutas-Emch, Moriah Meyskens, Lark Diaz, and Keith Macdonald
University of San Diego
Background: Faculty learning communities (FLC) are commonly used to help supportInnovation in teaching and enhancement of student learning. These communities give faculty the space, time and resources needed to enrich their teaching techniques and develop new or modify existing ways of teaching. The University of San Diego’s Center for Educational Excellence (CEE; our faculty development center) has organized a variety of these communities with topics such as active learning, universal design learning, and social justice in the classroom. The purpose of the FLC on Team-Based Learning (TBL) was to enhance faculty skills in supporting the need for students to build their collaboration skills and addressing the demands for 21st century solutions to the world’s complex, integrated problems.
Description: A community of faculty from a variety disciplines, not traditionally associated with TBL, spent a year researching, discussing and practicing the essentials of TBL. Faculty participated in monthly meetings, completed readings and homework assignments and attended conferences such as the TBL Collaborative. The faculty also developed workshops and resource materials that were used to educate other faculty at the university on TBL. Faculty included professors in management, biology, political science, psychology, and theology and religious studies.
Implementation: Faculty developed and implemented TBL across a diversity of courses using a number of assignments and activities. For example, TBL was used to enhance a community engagement project for a health psychology course; group quizzes, Lego building to define strategy of company and group analysis of a company was added to a management course; a 3-minute Biology paper was used in a biology course to help with team selection; and developing a timeline of the history of Christian marriage where teams decide how to present their timeline via skits, butcher paper, discussion panel for a theology and religious studies course.
This presentation will discuss these approaches and have participants think about how they may develop their own approaches in their disciplines.
202 – Spreading the Knowledge: Implementing a Medical Student Facilitated Infectious Disease TBL in a High School Classroom
Victoria Lucia, Rose Wedemeyer, and Matthew Drogowski
Oakland University William Beaumont School of Medicine
Background: A team of medical school staff, faculty, and students collaborated to develop a series of medical student-led activities designed to reinforce high school (HS) biology topics. Using train-the-trainer and near-peer teaching frameworks, we partnered with a HS biology teacher to provide her students an educational experience promoting critical thinking, while providing medical students the opportunity to enhance their teaching/communication skills and their appreciation for the role of non-health professionals in promoting healthy behaviors.
Description: A TBL on influenza/influenza vaccine was incorporated into the infectious disease unit of two biology classes in a local underserved HS. Under the supervision of a certified TBL Facilitator, a medical student developed a complete TBL module, including facilitator guide. Medical students administered the iRAT/tRAT and facilitated the entire TBL session.
Evaluation: HS students found TBL questions appropriately challenging and informative, promoting understanding of vaccines, including the myths surrounding influenza vaccines. They also found medical students to be helpful in exploring solutions to Application Exercise questions and connecting with course material. The HS teacher reported that subsequent student presentations demonstrated deeper levels of understanding and critical questioning about vaccines compared to classes that did not experience the TBL. Medical students had difficulty ensuring continued HS student engagement, but felt they utilized proper body language and non-verbal communication skills.
Logistical Lessons Learned: Provide an orientation session for medical students to discuss strategies for engaging/interacting with HS students; a 50-minute class session can accommodate the iRAT/tRAT and two Application Exercise questions; enhance the session summary and promote dialogue among the HS and medical students by discussing higher education and careers in health profession.
Conclusions: TBL can be successfully delivered in the HS setting to reinforce science-related topics. Including medical students in this process exposes HS students to careers in health professions they may not have previously considered.
203 – Use of engaging review application exercises to enhance understanding of critical concepts prior to summative examinations
John Cusick, Parto Khansari, and Leo Fitzpatrick
California University of Science and Medicine and California Northstate University, College of Pharmacy
BACKGROUND: Review application exercises were created in an Immunology class for third-year pharmacy students that utilized Team-Based Learning (TBL) as its pedagogy. The goal was to enhance student understanding of critical concepts prior to summative examinations.
DESCRIPTION: Three distinct exercises were utilized prior to summative examinations. For applications #1 and #2, important topics were divided amongst the teams, to ensure that all topics relevant to an upcoming exam were covered by posters that would be displayed for students to view. Application #1 was conducted over several class periods while applications #2 and #3 were conducted in a single class period. Application Exercise #1: Teams created engaging and informative posters describing assigned subject matter utilizing superheroes or other caricatures. A gallery walk was conducted the day the final posters were due, in which teams of students reviewed key concepts on posters created by fellow students, and graded the posters according to rubrics that emphasized the posters should be engaging, accurate, informative, and most importantly, were effective as a teaching tool. Application Exercise #2: Teams created mnemonics, songs or haikus to highlight assigned material and displayed their creations on a poster. Approximately half of the teams performed their song creation to the rest of the class. Application Exercise #3: A Jeopardy® review game utilizing team clickers and a team leaderboard created a fun and friendly competition in which student teams collectively discussed and answered practice exam questions.
CONCLUSION: The review application exercises were well received by students as indicated by the course evaluations, with many students indicating that the review exercises greatly helped them solidify their knowledge in a way that was fun and engaging. Additionally, the posters served as excellent recruiting tools during the interview process, as the student creations generated positive feedback from visiting applicant candidates not familiar with TBL.
204 – An Innovative TBL Module for Teaching Ethics in Human Subjects Research
Chrystal L. Lewis
University of South Alabama
BACKGROUND: While planning an evidence based practice course, nursing faculty expressed concern that students were not engaged, had poor understanding of key concepts, and complained the unit was not relevant. Students presented overconfident attitudes regarding vulnerability to ethics violations. Our purpose was to promote student application of research ethics principles to realistic situations in an engaging TBL context.
DESCRIPTION: A new TBL module utilizing 4S application activities was developed regarding human subjects research ethics. Students read about the Belmont Report and ethics principles prior to class. During class, teams critically examined the Tuskegee Syphilis Study, identifying ethical violations. Teams then simultaneously reported the number of ethical violations identified, a principle from Belmont Report not violated, and which ethical principle violation was the most serious in their opinion. Discussion was facilitated across teams, justifying decisions with clear rationales and evidence. Students completed a “minute reflection paper” on their personal understanding of ethics in research and a peer evaluation.
While the module was originally developed for nursing students, it was expanded to Occupational Therapy and Speech Pathology students. The Readiness Assurance Process was added to improve student preparation for the application activity.
RESULTS: Anecdotally, faculty and students impressions of the module were positive and worthwhile of the time. In addition, students showed increased awareness of ethical violations and engagement in discussion was improved. Students informally reported shock and dismay when they realized how ethical violations are still occurring both in practice and research.
CONCLUSION: While this ethics in human subjects research TBL module was developed for nursing, using the Belmont Report and Tuskegee Syphilis Study it can easily be transferred to any field participating in human subjects research.
205 – Overcoming peer evaluation pitfalls by utilizing faculty and student feedback
Charlotte Ricchetti, Dan Berlau, Peter Clapp, Peter Cogan, and Christopher Malarkey
Regis University School of Pharmacy
Background: Regis University School of Pharmacy utilizes Team Based Learning (TBL) across the first three years of the curriculum, and students complete one formative and two summative peer evaluations each semester. The three-part peer evaluation included forced team ranking, and provided students with both quantitative and qualitative data. Faculty were not providing the results to students in a consistent manner, and students felt pressure to provide each member of the team with equal ranking. A faculty task force was created to determine what changes to the peer evaluation tool would reduce these unintended problems.
Methods: The task force developed two surveys. Students were asked seven questions about perceived peer pressure, the value of each of the three sections, and the importance of anonymity when providing feedback. Students were also given the opportunity to provide open-ended comments. Faculty were asked three questions to assess consistency for distribution of feedback and to evaluate the importance of deterring students from intentionally manipulating the results of the peer evaluation.
Results: Eighty-five (85) students and seventeen (17) faculty participated in the survey. Students were strongly in favor of removing forced ranking section from the peer evaluation instrument (68/81). A significant number of students felt some pressure from teammates to fill out the peer evaluation in a specific manner (68/79). About 50% of students were opposed to removing anonymity from the qualitative portion of the peer evaluation at any time, but 44% (37/84) were open to removing anonymity by the third year. A majority of faculty felt the peer evaluation instrument should deter gaming to some extent.
Conclusion: Faculty utilized the survey results to develop a new peer evaluation tool that removed forced ranking and increased the use of behaviorally anchored statements.
206 – Increasing student engagement and learning outcomes with technology enhanced team-based learning (“TBL”) in a Mergers & Acquisitions guest lecture at the National University of Singapore
Brian O’Dwyer and Miguel Soriano
Embry-Riddle Aeronautical University and National University of Singapore
Background: One co-author was approached to conduct a guest lecture for a Mergers & Acquisitions class which aimed to cover deal rationale, company valuation, deal structuring, letter of intent and due diligence priorities. Students had gone through the topics in different lectures but had not applied the content together. The co-author was interested in conducting class like a “hackathon”, with students working in teams to apply knowledge and come up with solutions, instead of a passive guest lecture.
Hypothesis: The use of a technology enhanced Team-Based-Learning to conduct the lecture would lead to an engaging experience for students.
Method: Students were sent review materials for the course to date. The co-author gave the class a 20-minute briefing on the M&A situation. Then three application exercises were conducted covering different phases of M&A. After each exercise, team responses were reported simultaneously with technology followed by a facilitated discussion amongst the teams. After the exercise, students were surveyed about their experience.
Results: Student survey results showed students were confident in their ability to meet the learning objectives with a rating of 8.2 on a scale of 0 to 10. End of session net promoter scores (“NPS”) were calculated which showed a NPS for the format of 68, and an overall NPS of 78. For reference, a NPS of over 50 is considered good and over 70 is considered world class. Stated differently, on a scale of 0 to 10, students rated the format an 8.9 out of 10.0, the use of technology an 8.8 out of 10.0 and the overall session 9.2 out of 10.0.
Conclusion: The use of modified Team-Based-Learning methodology with technology could be an effective way to review and apply content of a whole course in a realistic guest lecture format.
207 – Using TBL to Facilitate an Interdisciplinary Collaboration between Medical and Physician Assistant Students
Gisella Newbery and Wendy Lackey-Cornelison
Western Michigan University, Homer Stryker MD School of Medicine
Background: The Interdisciplinary Interest Group (IIG) is a joint venture between medical students at Western Michigan University Homer Stryker MD School of Medicine (WMed) and allied health professional students at Western Michigan University (WMU). The goal of this student-run group is to establish effective collaborative practice between health sciences students. Team-Based Learning (TBL) provides a valuable tool for such cooperation.
Description: TBL, alongside professional discussions and hands-on workshops, provides important opportunities to promote teamwork and enhance IIG events. Medical students at WMed created and facilitated an interdisciplinary TBL application exercise, which was completed by students in PA and MD programs. This TBL exercise required integration of several disciplines in order to illustrate the different roles in patient care across disciplines. Implementation of the TBL highlighted the benefits and challenges of interdisciplinary TBL.
Results: Student creation and facilitation of this TBL allowed for those invested in the TBL to pursue the goal of becoming better healthcare professionals by working effectively with their peers in different allied health programs. Challenges for implementation included the group’s interdisciplinary nature and varied schedules between campuses.
Conclusion: Student-created and facilitated TBL application exercises enhance not only student motivation to participate in these types of events, but also their understanding of the TBL process and its goals. Creating opportunities for students to develop and facilitate TBLs in allied health professions curricula benefits both students and faculty using TBL. These experiences provide valuable learning opportunities that cannot be attained from attending classes alone.
Best Poster Award Nominee
208 – The Effects of Technologically-based Pre-class Preparation Activities on iRAT Scores
Tamara Bories
Western Illinois University
Background: Student success on individual Readiness Assessment Tests (iRATs) largely depends on the quality of pre-class preparation. However, students sometimes lack the requisite study skills or the motivation to prepare optimally on their own. To improve performance on the iRATs, technologically-based pre-class preparation activities were developed. Students were offered either: (a) regular pre-class reading assignments; (b) narrated, audio only, PPT mini-lectures; or (c) narrated, audio and video, PPT mini-lectures. The latter were recorded in a Chroma key room and paired with a content-related TED-Ed lesson.
Methods: Students from seven different classes completed one or the other of the pre-class preparation activities prior to taking their iRATs. Three classes served as a control condition and completed their regular pre-class reading assignment. Two classes completed the audio mini-lecture (AML) pre-class preparation activities. Two classes completed the video mini-lecture (VML) pre-class preparation activities along with the TED-Ed lesson. The effects of the pre-class preparation activities on iRAT scores were examined.
Results: The technologically-based pre-class preparation activities seemed to improve performance. The average scores (for five out of the six iRATs) were 1.38 points higher in the AML classes than in the control classes. Likewise, the average scores (for four out of the six iRATs) in the VML classes were 0.72 points higher than in the control classes. However, it did not seem to matter whether the technologically-based pre-class preparation activity was audio-based or video-based. The average scores for the AML and VML classes were similar. That is, the average scores (for all of the six iRATs) were only 0.13 points apart (i.e., VML>AML).
Conclusions: The purpose of the TBLInnovation described was to improve performance on the iRATs. Although the findings are based on a relatively small number of classes, they provide support for making technologically-based pre-class preparation activities available to students.
209 – Examining the Effectiveness of Team-Based Learning in Undergraduate STEM Statistics Courses
Cynthia G. Campbell, Kathrine E. Johnson, and Teresa Z. Taylor
Boise State University
Background: In many STEM courses a significant portion of learning is focused on the attainment of procedural knowledge. Application choices generally have a limited choice of method and one correct answer, which differs from how Team-Based Learning (TBL) is implemented in Social Science classes. We modified facets of TBL to function more effectively in this context and assessed the effectiveness through examining students’ responses and performance.
Methods: Modifications involved daily use of the readiness assurance process based on team responses. Daily application exercises provided students with a high level of support in mastering the skills. We used smaller teams (4-6) focused on ensuring a safe learning environment. To assess TBL effectiveness, data was collected on students’ perception of learning, their self-efficacy across the semester; and transfer of knowledge from introductory statistics to a subsequent course.
Results: Three sections of Introductory Statistics (n = 221) and two sections of Statistics Methods (n = 200) were compared to courses taught with traditional methods. Students in the TBL courses had more consistent daily attendance than students in other courses. Students’ ratings of confidence in their statistical abilities increased across the semester. Students also rated teams and other elements of TBL as important to their learning. To assess transfer of knowledge, data was collected from a next level course (Research Methods, n = 84). On a knowledge assessment conducted in the Research Methods course there was a statistically significant difference with TBL students outperforming students taught in traditional ways.
Conclusions: TBL can be used effectively in courses in which the application context varies. The modified TBL provided structure for instructors to create an active learning environment for students to engage in STEM course content which resulted in the improvement of several important academic indices.
Best Poster Award Nominee
210 – Usefulness and Validity of the Team Development Measure for Assessing Teamwork in Interprofessional Education
Wendy Madigosky, Janice Hanson, and Monica McNulty
University of Colorado Center for Interprofessional Education and University of Colorado School of Medicine
Background: The Team Development Measure (TDM) was created for clinical teams, however team development assessments are needed in the interprofessional education setting. This poster presents validity evidence for TDM use in a classroom-based interprofessional education course for students in dentistry, medicine, nursing, physician assistant, physical therapy, and pharmacy programs.
Design: We gathered evidence in four of five domains of validity set forth in the 2014 American Educational Research Association Standards for Educational and Psychological Testing: content validity, response processes, internal structure and consequential validity.
Results: Content validity: Information about TDM development and item selection was reviewed (Stock, Mahoney, & Carney, 2013). Relevance to our IPED course was established by identifying a close match between TDM components and IPED goals and objectives for the ‘Teamwork and Collaboration’ content domain. Response process: Faculty reviewers found TDM instructions clear and items easy to answer. The Likert-scale structure of item responses was familiar to students in the course. Student feedback on course evaluations indicated that many students found the TDM to be a valuable platform to express how their team was developing. Internal structure: An increase over time in mean team scores indicated that TDM measures growth in team functioning. A spread of responses across teams each time the tool was administered suggested that the TDM discriminates between teams that functioned differently in our IPED classroom setting. Cronbach’s alpha calculation supported internal consistency. Consequences of using tool: Data from the TDM helped IPED teams identify strengths and weaknesses. It additionally prompted teams to identify areas for improvement which they addressed during the remainder of the course.
Conclusion: Content, response process, internal structure and consequential validity data builds a strong case that using the TDM in an interprofessional classroom setting is valid and helpful.
211 – TBL: Connecting metabolism and biochemistry with real life events
Joanne Lind, Matthew Robson, Morwenna Kirwan, and Christine Chiu
Macquarie University
Background: Undergraduate biochemistry courses are a challenge to teach as students struggle to connect the complexity of the science with the real world. This can result in lack of student engagement and poor content retention. TBL and gamification have been shown to increase student engagement with student learning. This paper describes the design of a new Nutrition and Metabolism module that used TBL to connect principles of biochemistry and metabolic processes with historical experiments on dieting and nutrition.
Methods: The Minnesota Starvation Experiment was used as the basis of the online learning module within the Bachelor of Clinical Sciences program at Macquarie University. The content stepped students through real-life images and data where they complete activities (quizzes, hot spot, and wikis) connecting the underlying biochemical processes with the evidence presented. The weekly face-to-face tutorials consolidated the learning with IRATs and TRATs. Gamification was introduced via an online TBL leader board and prizes for the top performing student and team.
Results: A total of 26 final year students completed the module. Student participation was high, with over 90% attendance at all tutorial classes and all students passing the module. Students were confidently able to explain the scientific basis of weight loss and weight gain both within tutorials and the end of module assessment. Several students expressed a preference for the competition aspect of the TBL, citing it as a “major motivator to prepare for class”. Data collected from the IRATs identified topics that required further explanation, and enabled early intervention for students at-risk of failing.
Conclusions: In conclusion, Macquarie University has developed a TBL module for the study of metabolism and biochemistry that connected real world examples with scientific content. Students were motivated to learn and enjoyed the gamification built into their team based learning.
212 – Utilizing Team-based learning (“TBL”) data to analyze student performance, predict student outcomes and optimize learning outcomes.
Brian O’Dwyer
CognaLearn
Background: Data has been described as the “new natural resource” and is transforming many fields from business to sport. Team-based learning (“TBL”) with its frequent assessments has the potential to generate over 100,000 data points in a course. With technology being more frequently used in TBL, data can be more easily tracked and analyzed. It is an opportune time to identify approaches to using TBL data.
Hypothesis: TBL data can be used to analyze student performance, predict outcomes and optimize learning.
Method: Three case studies of how TBL educators use TBL data.
Results: Case one compared IRAT, TRAT and Final Examination scores. Three effects were identified: i) an increase in TRAT versus IRAT scores of over 20%; ii) a narrowing of the range between the highest and lowest scores between the IRAT and the Final Examination and iii) Final Examination scores closer to TRAT than IRAT scores.
Case two attempted to predict student final course scores by analyzing the impact of various assessments on final grades and found that IRAT and midterm performance was one of the best indicators of final course grades.
Case three analyzed RAT performance. RAT scores and discrimination index at the item or question level were used to identify tough questions or concepts. Once identified, the tough questions were repeated on subsequent RATs so that students would have more practice with tough concepts.
Conclusion: There is enormous potential to use TBL data to analyze, predict and optimize outcomes. However, care must be taken not to blindly apply analysis from one set of learners to another. What could be done is to develop several frameworks for TBL data analysis that could be easily tested, customized and applied in various populations.
213 – Implementing Team-based learning (“TBL”) to provide continuing professional education for practicing accountants.
Brian O’Dwyer
CognaLearn
Background: The author was approached by a partner from the accounting firm of KPMG about using Team-based learning (“TBL”) to provide training to practicing accountants. KPMG requires each accountant to complete 40 hours of continuing professional education per year. KPMG selected a 3.5-hour training session for newly promoted audit managers in the very technical area of the Financial Reporting Standard 36 (“FRS 36”) which covers impairment valuation.
Hypothesis: TBL can be used to effectively train practicing accountants in technical matters.
Method: The author and KPMG spent about 20 hours over a three-week period to adapt the existing lecture style materials. The existing materials had multiple choice questions and cases. The training was planned with an overview of TBL and why FRS 36 was important to KPMG. This was followed with an Individual Readiness Assurance Test (“IRAT”), Team Readiness Assurance Test (“TRAT”), Clarification Session and Application exercises. At the end of the training, the 53 learners were given a feedback survey using a five-point Likert scale.
Results: Learner survey results showed 94% of learners felt that the TBL approach was effective and 98% were interested in other training events with TBL. From a learning objectives perspective, 83% of participants reported that the five learning objectives for the training had been met. During the training session, the IRAT and TRAT were conducted as planned. However, the Clarification Session took longer than planned and the intended mini-lecture evolved into a two-hour lecture. There was insufficient time to conduct the application exercises.
Conclusion: TBL can be an effective way to train practicing accountants. Even with only using some elements of TBL (IRAT, TRAT and Clarification) the session was quite effective. In the future, including the application exercises could be even more effective.
214 – Thinking Critically about Critical Thinking: A TBL-style Course for Pre-clerkship Medical Students
Paul Haidet and Daniel Wolpaw
Penn State College of Medicine
Purpose: Physicians increasingly need to critically think through their decisions. We designed a course to foster habits of mind related to critical thinking.
Methods: We designed a 6-week course for first-year medical students, based on the idea that critical thinking enables an appropriate “toggle” between fast and slow thinking (Kahneman, 2011), and that five habits of mind (curiosity, open-mindedness, intellectual humility, balanced skepticism, metacognition) foster this. We used a TBL-variant called Case-Based Collaborative Learning (CBCL, Acad Med 2016;91:723-9). Students completed writing assignments designed to stimulate practice of the habits of mind. We used a peer-review process to foster students’ conceptual depth and understanding.
Results: 145 students completed end-of-course evaluations (100% response). 46% found the CBCL cases and 37% found in-class discussions to be very- or extremely-useful to their understanding the five habits of mind. 71% rated the habits of mind to be very- or extremely-relevant to their future practice of medicine, but only 45% rated them as such for board-type exams. 73% reported practicing the habits of mind at least occasionally. One student noted: “On the [cardiology block] exam, I practiced the technique of listing reasons that support/don’t support my [fast-thinking] hypothesis “¦ I was able to physically see why one choice was better than the other.” Out of six possible physician attributes, students rated critical thinking highly.
Conclusions: Students found the “habits of mind” framework to be useful for clinical work, but less so for board-type exams, although qualitative comments suggested that they were using course concepts to aid progress in concurrent biomedical science-oriented courses. The CBCL framework, while creating high degrees of interaction, was perceived by students as only moderately successful. Our future work will focus on course framing and perceived relevance of the CBCL activities, as well as explicitly tracking practice of the five habits.
217 – Promoting Team Development Through Faculty Coaching and the Team Development Measure (TDM)
Amy Nordon-Craft and Wendy Madigosky
University of Colorado Anschutz Medical Campus
At the University of Colorado Anschutz Medical Campus, learners from 7 health professions participate in a 2 semester TBL course emphasizing teamwork/collaboration, patient safety and quality improvement, and values/ethics. One of the desired outcomes of the course is for teams to learn how develop. To support this team development, we introduce the four components of team development (cohesiveness, communication, role clarity and goals & means clarity.) Additionally, we identified the need for our faculty facilitators to become team coaches over the course of the semester.
In order for teams and faculty coaches to assess their progress, we used the Team Development Measure (TDM). TDM was developed in the clinical settings and is an objective, evidence-based tool. It provides data on team performance and can serve as a process improvement tool.
Teams complete the tool after the 2nd session (once they have learned about the 4 components of team development) as well as after the 8th session, which is at the end of the first semester. Prior to the beginning of the second semester IPE course, the results were shared with both facilitator/coaches and student teams. Teams were provided an opportunity to reflect and discuss the results and were asked to develop strategies to improve their performance. To maximize effective team dynamics, faculty coaches encouraged and challenged the teams to identify areas of strength and areas for growth. TBL report outs were used to share each team’s top strength and a stretch goal for the remaining 7 class sessions. Teams complete a finale TDM after the 16th session (end of the second semester).
In this presentation, we will describe how to introduce teams to the TDM four components using TBL, and how to assess team development over time using the TDM. Finally, we will explain the importance of expanding the traditional TBL facilitator role to that of coach.
218 – Impact of tailoring Team-based learning (“TBL”) train-the-trainer workshops from academic to training environments
Brian O’Dwyer and Sandy Cook
CognaLearn and Duke-NUS
Background: Duke-NUS routinely offers training programs on TBL. The audience is usually university educators across many disciplines. Given the diversity of the audiences, the materials used are more generic. In 2016, the Public Utilities Board (PUB) in Singapore requested Duke-NUS to teach their trainers to use TBL to train employees in project management.
Later the Australian Employment Training Solutions (“AETC”) asked to teach trainers develop TBL modules to provide employability skills training to disadvantaged youth using TBL. The question was would the training be more effective if the more TBL training material was tailored specifically to the audience requesting the training?
Method: Average IRAT, TRAT scores and “recommend course” from programs were compared. There were 28 trainers from PUB and 20 trainers from AETC. The PUB program was 2 ½ days with 4 workshops covering the fundamentals of TBL and ½ day reviewing their efforts at converting modules into TBL. There were 3 RA tests with 27 questions. AETC program was 2 days, 6 workshops, with 5 RA tests and 34 questions.
Results: The overall recommendation rate for the AETC workshop was 100% compared to 92% for the PUB workshop. The average IRAT scores for AETC and PUB were 60% and 61% and TRAT scores for both groups were 82%.
Conclusion: While not statistically different due to the small size, clearly more specifically tailored version appealed to all the participants AETC. There was no difference in the RAT performances ““ suggesting relatively equal levels of difficulty in the two versions. The challenge for TBL trainers is the time and effort required to customize each training program to the specific audience. It will also be important to follow-up with the two groups to see the success and impact of the TBL programs that they each develop.
219 – Implementing Team-based learning (“TBL”) in an online synchronous environment for a career skills workshop
Brian O’Dwyer
CognaLearn
Background: The author developed a series of five workshops to teach: Career Strategy, Pitching, Resume Writing, Networking and Interviewing. The workshops are typically delivered in a face-to-face formation using team-based learning (“TBL”). In one-year master of management programs students have very little time to settle into graduate school before the recruiting process begins. The author wanted to deliver these workshops in an online format before students arrived on campus.
Hypothesis: A workshop delivered with TBL face-to-face could be delivered effectively in an online format.
Method: Accepted students at Duke University’s Fuqua School of Business were invited to participate in the online career skills workshop. Students answered questions during registration which was used to pre-assign teams. The workshop was conducted using the www.zoom.com web conferencing tool. It began with an overview delivered using Microsoft PowerPoint. This was followed by an Individual Readiness Assurance Test (“IRAT”), Team Readiness Assurance Test (“TRAT”), Clarifications and Application Exercises which were delivered using the www.intedashboard.com online TBL tool. Teams were broken out into separate web conference rooms during the TRAT and Application Exercises.
Results: Post-event learner surveys showed that 88% of learners would recommend the workshop or a 4.4 rating on a 5.0 scale. This compares to a face-to-face version of a similar workshop which had a 100% recommendation rate with a 4.8 rating on a 5.0 scale. An asynchronous (non-TBL) online version of the workshop is available using www.udemy.com and has been rated 4.1 on a scale of 5.0. Of the 40 learners that registered for the course only 20 students participated and 13 completed the feedback survey.
Conclusion: It is possible to conduct a workshop using TBL in an online synchronous format. However, the sample size is small and more work could be done to understand the effectiveness of online TBL
220 – Enhanced technology for Team-based learning – impact of a third generation of TBL software
Brian O’Dwyer
COGNALEARN PTE LTD
Background: The author, also a faculty member at Embry-Riddle Aeronautical University, began teaching with paper-based team-based learning (“TBL”) and was impressed with its efficacy. However, the TBL administrative process was overwhelming. To overcome this challenge the presenter started with an existing TBL software platform developed at Duke-NUS Medical School and completely rebuilt a newly redesigned “second generation” TBL software application (www.intedashboard.com). The TBL software application provided functionality to implement, individual and team readiness assurance test, application exercises and peer evaluation.
The author then conducted an 18-month beta test with 20 participating institutions. Results of the beta testing were presented as a poster at the 2017 TBLC conference. Beta testing faculty were surveyed about their experience with the www.intedashboard.com TBL software which reported: 75% of faculty would recommend the TBL software; 81% felt real-time data increased learning outcomes and 54% reported that technology save administrative time; 36% identified concerns about academic integrity; 18% identified concerns about usage difficulties and 9% thought it was not as much fun as paper based methods.
The author then incorporate the feedback from the beta test into another completely redesigned third generation of TBL software which is slated for release in late 2017.
Hypothesis: A third generation version of TBL software will improve
Method: Faculty and student surveys will be used to compare the second generation of TBL software with the newer third generation of TBL software.
Results and Conclusion: Pending completion of analysis.
221 – Developing Emotional Intelligence and Moral Competence in Health Professionals Using Team-Based Learning
Luma Munjy and William Ofstad
California Health Sciences University
Problem: The Association of American Colleges and Universities developed the LEAP Essential Learning Outcomes as competencies that should be gained from contemporary higher education. These outcomes include ethical reasoning, cultural competency and civic engagement which align to development in the affective learning domain. These outcomes require higher levels of reasoning, emotional intelligence and moral behaviors, all of which are critical competencies necessary for compassionate patient-centered care and central to ensuring practice ready health professionals. Unfortunately, current research suggests that current health education, which is primarily focused on knowledge and skills, tends to diminish the learner’s ability to behave morally, empathize with patients and engage effectively in the affective domain. Team-based learning (TBL) provides a platform for intra- and inter-team discussion which can facilitate the development of emotionally intelligent and morally competent healthcare professionals.
Aim: The purpose of this poster is to present a model for the effective development of attitudes, values and behaviors of higher education using a modified version of the Lind’s Konstanz Method of Dilemma Discussion adapted to a TBL framework.
Methods: The role of readiness assurance, 4S application and inter/intra-team discussion along with facilitator led closure is detailed by this model to effectively display how TBL can facilitate this learning strategy. Assessment tools including the Moral Competence Test (MCT), the Defining Issues Test (DIT) and the Schutte Self-Reported Emotional Intelligence Test (SSEIT), are summarized and presented as methods for documenting learner growth.
Results: Authors share findings utilizing this model in a pharmacy school curriculum. Results will be presented as student satisfaction, improvement in outcomes, changes in student behavior, and impact beyond the classroom.
Conclusion: This model provides a systematic method for educators to effectively develop emotional intelligence and moral competence in their students.
222 – Assessing Student Performance in your TBL course with Standards-Based Grading
Drew Lewis
University of South Alabama
Imagine a class in which students ask you to give them another exam; in which students know exactly which skills they have mastered and which they need to practice; in which students walk in to your office and say “Professor, let me show you I know how to…?” instead of begging for partial credit. This is the environment created by a grading system called Standards Based and Specifications Grading (SBSG). We describe Standards Based Grading and Specifications Grading and show how you can incorporate them into your TBL class. We will discuss the behavioral shifts we see in our students in SBSG courses. You will also discover how Specifications Grading allows you to eliminate false equivalences created by traditional (weighted average) grading schemes, and keep measures of student learning independent of behavioral incentives such as peer evaluations and readiness assurance tests. You will also learn how Standards Based Grading inherently produces data measuring how well students are meeting your TBL course’s learning outcomes, which can be used to guide and evaluate future pedagogicalInnovations in the course.
223 – Multi- Level Mentoring Using TBL in Faculty/ Graduate Student/ Undergraduate Student and Teaching Center Teams
Janet Johnson, Holly Bender, OP McCubbins, Luke LeFebvre, and Foy Mills, Jr.
Iowa State University and Tennessee Technological University
Mentoring is no longer top down in the contemporary academic setting. The purpose of this session is to inspire TBLC participants to enhance their mentoring of undergraduates, graduate students, and faculty in multiple ways. What happens when an attempt to optimize TBL with a team in your course results in a much richer experience of reciprocal mentoring? A panel of experts from four institutions will share varied experiences with multi-level mentoring including a partnership between a faculty member and a teaching center to train TA’s to teach an 800 student class via TBL. Another panelist will describe how a partnership between graduate students and the Preparing Future Faculty program had unexpected outcomes for multi- level mentoring of faculty course supervisors. A third will describe how he applied TBL in departmental teams of faculty and graduate students to teach departmental courses. A fourth panelist will describe how he as a graduate student taught TBL to his faculty mentors to transform departmental teaching. The panelists will describe experiences implementing TBL in face to face, blended and online settings to explore the transformative power of TBL as they stepped into an active and engaged role of mentor.
225 – The Use of Virtual Reality for Distance Team-Based Learning
Jody K. Takemoto, Rachel A. Sharpton, Brittany L. Parmentier, Thayer Merritt, Leanne Coyne, and Rebecca Fernandez
Department of Pharmaceutical Sciences and Department of Clinical Sciences, Ben and Maytee Fisch College of Pharmacy, The University of Texas at Tyler
BACKGROUND: Implementing the core features of team-based learning in a distance learning environment can be difficult. Cutting edge technology in virtual reality now allows people to meet ‘in person’ regardless of their physical geographical locations. In this pilot study, we explored the usefulness of virtual reality as a platform for distance team-based learning.
DESCRIPTION: Participants met their team members and were provided step-by-step instructions and a short reading assignment. Participants went through the readiness assurance process, and were escorted to their individual virtual reality rooms. After providing participants with an application in the virtual reality environment and a debriefing about the key concepts from the activity, a survey about the virtual reality team-based learning experience was administered.
RESULTS: Preliminary results suggest that the technology was easy and comfortable to use. There were no reports of uneasiness to communicate through virtual reality to their team members. If given the choice, participants were more likely to choose a virtual reality team-based learning environment over current online methods.
CONCLUSION: This preliminary study demonstrated that using virtual reality to conduct a team-based learning module warrants further exploration. Optimization to ensure ease of use, and long-term comfort is necessary before full implementation. Additional tools to facilitate team-based learning, such as a virtual readiness assurance process is also of interest.
226 – Online Web Enabled Peer Evaluation System
Richard Lindquist
University of Connecticut Health Center
INTRODUCTION: Peer Evaluation is an essential component of TBL. It promotes effective team interaction and minimizes loafing. An online web enabled peer evaluation system was developed using Uniform Server web stack (Apache, MySQL, PHP.), which requires no installation and runs from a thumb drive or any USB storage device or Bitnami web stack, which can run on Windows, Mac OS or Linux environments as well as Vmware or VirtualBox vitualized environments, and popular cloud platforms such as Amazon Web Services, Microsoft Azure and Google Cloud platform.
DESCRIPTION: In creating peer evaluations, faculty enter criteria/questions for database storage. Faculty create stock peer evaluations by selecting questions from the database, or edit questions or stock evaluations. On activating one of the database stored stock peer evaluations faculty can select whether the peer evaluation uses free text response, Likert, or divide the points formats.
The system allows configuring the peer review start & end dates and whether self evaluation contributes to overall grade, faculty grading of reviewers, & anonymous reviewers are used.
In addition to reading text responses faculty can view brief & detailed results with or without variance by teams & by individual team members, see how each team member evaluates other team members & how each individual is evaluated by team members, individuals who have or have not evaluated his/her team members & individuals who have not been evaluated, individuals with low evaluations or giving low evaluations, and criterion with low assessments. If self evaluations are enabled, learners & and faculty can compare and contrast self evaluations with the team’s evaluations. Faculty feedback to reviewers, if required, is enabled.
RESULTS & CONCLUSION: Student survey reveals that 50% prefer this online web based system over paper peer evaluations, 30% had no preference and 19% preferred paper over online web based peer evaluations. Thus the system was considered a success.
227 – Successfully tranisitioning your in-class TBL to online TBL through improved team cohesion and accountability
Annetta Dolowitz and Ken Gunnells
University of Alabama at Birmingham
According to Babson Survey Research’s 2015 annual report on online learning in U.S. Higher Education, 28% of students take at least one distance course. In their 2014 report, 2.85 million of the 5.8 million students enrolled in fall were taking all of their courses online, and 2.97 million were taking some courses online (Online Learning Consortium, 2015). The Babson study found that like RW3 CultureWizard survey, students often take online courses for flexibility and geographic convenience (Online Learning Consortium, 2015). Part of successful adaptation to this education reality is graduating students who are ready to enter the work world with “real-world skills [that] are needed to bolster employability and workplace development” (Adams Becker, 2017, p. 2).
This poster’s purpose is to share our innovations, insights and techniques in a model for translating a face-to-face TBL class to an online setting. Some TBL elements, such as the RAP and application exercises, are relatively easy to move online with few modifications by utilizing a school’s LMS. However, orientating students to TBL and team formation, as well as peer evaluations/team accountability require substantial modifications to work effectively for online settings.
Our recommendations highlight two specific areas for online TBL class design changes; 1) the need for smaller online team sizes to accommodate difficulties with team communication, and tools to increase team cohesion, both of which enhance team success, and 2) more granular and frequent team accountability checkpoints to discourage freeloading that sometimes results from delayed punishment and a belief by some online students that they can hide their inaction from the instructor. Examples of changes include a short video introduction to teammates, setting an iRAT threshold below which a student’s iRAT grade is used in place of the team’s tRAT grade, and listing only contributing teammates on team projects for graded assignments.