Will TBL™ Work For Me?

1. Are there any instructors for whom team-based learning will not work?
2. Should I try to use team-based learning with my subject matter?
3. Should I try to use team-based learning with my class size?
4. Should I try to use team-based learning if I am in a classroom with inflexible seating?
5. Should I try to use team-based learning wif I teach in a 50-minute time slot?
6. Will team-based learning force me to invest a lot more time in my teaching?
7. Do I need any special skills to use team-based learning successfully?
8. How does TBL relate to “Flipped Learning”?

Problems with Group Work

1. Can I cover the required course content if I use class time for group work?
2. Will costs related to free-riders outweigh the benefits of group work?
3. Will students be motivated to prepare for group work?
4. Can I resolve interpersonal conflicts that will inevitably arise among group members?

“How to” Implementation Questions

1. How should I form teams?
2. How big should the teams be?
3. What should I do if students have difficulty with the required pre-class reading?
4. How do I set up an effective grading system?
5. How can I provide grade-based incentives for group work and still avoid giving higher grades than the weaker students deserve?
6. Should I give a RAT every class period (or every week)?
7. How much should the RATs count??
8. What should I do when students are absent?
9. Should I use peer evaluations on all projects??
10. How should I handle the scoring of appeals?
11. How should I give feedback on multiple choice questions?
12. How can I provide immediate feedback on the RATs if I don’t have a scoring machine?
13. What are the benefits of using a “split answers” format on the RATs?
14. Are there alternatives to using multiple-choice questions on the RATs?
15. What kinds of questions are good questions for the RATs?
16. Can I use a writing assignment for the application phase of team-based learning?
17. Is it OK to use group presentations?
18. If I can not use group papers or presentations, what can I use?

 

Part I – Will Team-Based Learning Work For Me?

Question #1 – Are there any instructors for whom team-based learning will not work?

Based on our experience, there are four groups of instructors who should not try to use team-based learning. These are instructors who: 1) are not yet certain what they want students to do beyond the goals of “learn and remember”, 2) feel threatened by frequent student challenges, especially when the challenges come from students who are united in groups, 3) really enjoy the “performing” aspect of the teaching role, or 4) are unable to invest the time needed to redesign their approach to teaching.

Going beyond knowing: One of the greatest advantages of team-based learning is also a limitation. The advantage is that it shifts the focus of instruction from learning about concepts and ideas to learning <strong”>how to use them in meaningful ways. The limitation is that it will not work unless the instructor is ready to provide students with an opportunity to, and to reward them for, learning how to use the basic content in a meaningful way. Thus, if the instructor’s primary focus is on “covering” content and he or she fails to use application-focused assignments, students are likely to rebel at having to invest time and energy learning what seems to them to be an endless string of meaningless details.

Comfort with being challenged by students: This is another situation in which an advantage of team-based learning turns out to be a problem for some instructors. Team-based learning promotes a climate in which students feel free to challenge the instructor. In most cases, this is an advantage because it promotes both learning and instructor satisfaction. For some instructors, however, being challenged can be an anxiety-provoking experience. Thus, we would not recommend using team-based learning until you have a firm grasp on the material you will be teaching and are intellectually and emotionally “up” to being challenged by your students.

Satisfaction from the “performing” aspect of teaching.Although the majority of instructors who adopt team-based learning will not even consider going back to their old way of teaching, it can be very unrewarding for others. Due to the decreased role of lectures, faculty members who really enjoy preparing and delivering lectures end up feeling as though team-based learning robs them of the aspect of their teaching that they enjoy most.

Time needed to re-design your course” Shifting from a traditional approach to team-based learning does require a significant time investment for the developing of RATs and application-focused activities. Although the time required for the redesign process can be kept to a minimum (see: “Will team-based learning force me to spend a lot more time in my teaching” below), you are much more likely to have a positive experience if you delay your first team learning start up until you have time to plan out what you are going to do.

Return to questions

Question #2 – Should I try to use team-based learning with my subject matter?

This question is most frequently asked by instructors with exactly the opposite concerns. On one end of the spectrum, some instructors worry that team-based learning will not work in courses that have a great deal of factual material to cover and that lend themselves to “right or wrong” answers such as chemistry, physics and many of the natural sciences. On the other end of the spectrum, some instructors, mainly in the humanities, worry that team-based learning will not work because there are no right or wrong answers.

In both cases, the answer as to whether or not team-based learning is appropriate for the subject matter is an unequivocal “Yes” and the key for both groups is having a clear picture of what you want students to do with the material. In both cases, the most common pitfalls in implementing team-based learning are related to the Readiness Assurance Process. However, the nature of the pitfalls is quite different for courses with lots of factual material than for courses with no right or wrong answers.

Courses emphasizing factual material: In this kind of course, the key is focusing the RATs on key concepts, not on details or computations. That is, the questions should focus on making sure that students understand the material in the table of contents and not on the material that is only in the index. If students get a clear “big picture” from the Readiness Assurance Process, they will add (and retain) the details as they complete application-focused assignments. If the RATS focus on details or computations, students are likely to feel that they are being punished for not memorizing everything. In addition, if the RATS focus on details, the potential value of the group RATs will be lessened by the fact that the discussion will be brief and will focus on what the reading says as opposed to what it means. Further, if the RATS emphasize doing computations rather than setting them up, there is seldom very much discussion because one member will simply act on behalf of the group.

Courses with no right or wrong answers: The key in this situation is focusing the Readiness Assessment Process on ensuring that students understand the criteria that you will apply in grading their assignments, i.e., the RAT questions should assess their understanding of the models that you want students to use to complete their assignments. For example, a Sociology instructor could use a RAT to ensure that students understand concepts related to socio-economic status, then test their ability to use the concepts by assigning them to use the concepts to discriminate between: effective and ineffective campaign strategies in a presidential election, interactions among characters in a movie clip, the focus of television ads during different prime-time television programs, etc.

Return to questions

Question #3 – Should I try to use team-based learning with my class size?

This question is most frequently asked by instructors with either very large or very small classes. In both cases, the answer as to whether or not team-based learning is appropriate is an unequivocal “Yes.” In both cases, the key is using decision-based assignments and procedures that enable immediate comparisons with other teams. In large classes, this means the teacher needs to ensure that all the teams work on the same problem and then provide a way for teams to simultaneously reveal the choices they have made.

In classes that are too small to divide into two teams (i.e., fewer than 8 students), then the best option is using a single team and, whenever possible, providing them with ongoing comparisons to teams in similar classes, e.g., their RAT scores vs. average RAT scores from previous classes.

Return to questions

Question #4 – Should I try to use team-based learning if I am in a classroom with inflexible seating?

Faculty members often worry that team-based learning won’t work unless they can get their class scheduled into a classroom with round tables or movable chairs. Based on our experience, this fear is largely unfounded. Although the physical layout of the classroom can be somewhat of a problem, team-based learning can be used effectively in virtually any classroom. The only essential condition is providing a space in which members have the possibility of having eye-contact with each other. For example, in tiered classrooms with fixed individual seats, instructors should make seat assignments such that members on one row can turn around and be face-to-face with the rest of their team. As long as teams have the possibility of doing so, they will develop procedures for overcoming the physical barriers inherent in the space to which they have been assigned.

Return to questions

Question #5 – Should I try to use team-based learning if I teach in a 50-minute time slot?

Although team-based learning can be used effectively in 50-minute class periods, doing so usually requires making some adjustments. For example, Readiness Assessment Tests must be either very short or the Readiness Assurance Process has to be spread across more than one class period, i.e., the individual test on the first day and the team test, appeals and instructor input on the next one. Further, while a lecturer can simply end class when the time runs out and start again at the beginning of the next with very little disruption of the flow of the class, using blocks of time for in-class group work makes timing issues much more important. As a result, longer class periods help in two ways. One is that they allow more efficient use of time. The other is that they provide more flexibility for designing and managing in-class group assignments.

Return to questions

Question #6 – Will team-based learning force me to invest a lot more time in my teaching?

This question is not an easy one to answer. On one hand, successfully implementing team-based learning does require most instructors to invest a significant amount of time to restructure their courses and develop effective team assignments. On the other hand, this task is largely a one-time effort. Once the course is up and running, the time required for operating and maintaining a team learning course is not substantially different from the time required to effectively teach a course in any other format. In addition, most teachers are able to streamline the course-restructuring process by using a variety of resources and by sharing ideas and assignments with other team learning users. Further, most team learning users find that their first course restructuring effort is the most time consuming and that the conversion process is much easier with subsequent courses.

Given the significance of the start-up costs, we have three recommendations. One is starting with the basics and adding sophistication over time. At a minimum, this means starting off each unit with a RAT that prepares students for one or more appropriate application-focused assignments. “Appropriate” here calls for all groups to be working on the same problem and making (and simultaneously reporting their) specific choices. Second, we recommend a strategy of implementing team-based learning in one course at a time. Finally, it is best to start with a course that has as many advantages as possible. These would include: familiar subject matter, a moderate number of students (20-40), at least a 75-minute time slot and, a classroom in which students do not face significant physical barriers to working face-to-face with each other.

Return to questions

Question #7 – Do I need any special skills to use team-based learning successfully?

The personal characteristics of any teacher have an impact regardless of the teaching approach that he or she uses. For example, being organized and having a flair for the dramatic are two keys to being an effective lecturer. However, while getting organized is something that anyone can learn to do – it simply takes a commitment to do it – learning how to make spellbinding presentations can be very difficult for a large percentage of would-be lecturers.

Based on our observations, the majority of experienced teachers already have most of the skills required to effectively implement team-based learning. Most of us have already learned how to organize material, create and give tests and assignments, and provide feedback on student performance. The major change, which can be a difficult one, involves thinking differently about what should be happening in our classrooms. Instead of thinking about how we should be teaching, we have to focus on what we can do to enhance student learning. Beyond that, the one new skill that appears to provide the greatest challenge to new team learning users is developing the ability to design effective group assignments.

Return to questions

 

Question #8 – How does TBL relate to “Flipped Learning”?

Flipped Learning is defined by the Flipped Learning Network as “a pedagogical approach in which direct instruction moves from the group learning space to the individual learning space, and the resulting group space is transformed into a dynamic, interactive learning environment where the educator guides students as they apply concepts and engage creatively in the subject matter”.

Although Flipped Learning and TBL are not synonymous, both are forms of student-based learning. Flipped Learning can be regarded as a component of TBL in so far as it is based on a pre-learning approach, and focuses on student-centered learning activities in the classroom.

TBL goes beyond Flipped Learning, as it includes structured approaches for in-class learning activities. TBL also incorporates teambuilding as a necessary activity for collaborative learning, by establishing fixed teams. Students learn teamwork skills that are important for solving a variety of complex problems. TBL also includes a process for students to provide constructive feedback on team contribution to other team members.

In summary, TBL provides a structured but flexible framework that guides the educator in developing pre-learning materials, in-class tests, and in-class activities. A substantial body of research shows that this framework engages students and achieves improved learning outcomes.

Return to questions

Part II – Anticipated Problems with Group Work

Question #1 – Can I cover the required course content if I use class time for group work?

The greatest fear of most instructors is that if they use class time for group work, they will not be able to cover enough content. For the most part, this fear is based on the observation that, even in the face of threats such as pop quizzes, only a small minority of students even try to prepare for class and even fewer come to class actually well prepared. The fact that students rarely appear to be prepared for lecture-based classes, I contend, results from a powerful combination of disincentives for completing pre-class reading assignments and a lack of positive incentives for students to reveal what they know when they are prepared.

Most students have learned that, as long as they attend and pay attention, they will either be able to do well on the tests based on the lectures alone or they will be adequately warned if additional reading is really necessary. As a result, such things as the costs of buying the text, using discretionary time to read it, and then being bored during the lectures, end up being powerful disincentives for pre-class preparation. Further, when students do prepare, they are often reluctant to speak up for two reasons. One is that, if they are wrong, they worry about looking bad. The other is that, even if they are right, many worry that they might be seen as “one-upping” their peers.

Team-based learning, however, provides multiple incentives for pre-class individual study and for students to speak up when they do have information to share. One strong incentive is that the individual Readiness Assessment Test (RAT) scores have an impact on their grade. In addition, because teams cannot do well unless their members are prepared, interaction during the team RATs provides several other powerful incentives for both pre-class preparation and for sharing information within teams. Students learn quickly that their peers will hold them accountable for pre-class reading because, every time a disagreement occurs, members will be asked about the reasons for their choices. As a result, there is both immediate social pressure to study and the realization that failing to do so is likely to result in the negative consequence of receiving a low peer evaluation at the end of the term. In addition, over time, members are increasingly motivated to study to ensure the success of their team.

Given this combination of factors, the vast majority of team learning users have found that, because students are willing to take responsibility for completing pre-class work, they are actually able to cover more material. In addition, the efficiency of the Readiness Assurance Process allows them to shift the primary focus of in-class work from covering content to decision-based team assignments that focus on developing higher-level thinking skills.

Return to questions

Question #2 – Will costs related to free-riders outweigh the benefits of group work?

Faculty members who ask this question tend to have one of two situations in mind. By far the most troublesome is when one or two members end up being saddled with doing far more than their fair share of the group work. This type of free-rider problem is almost always the result of poorly-designed group assignments that can be (and usually are) completed by individual members working alone. Because one member doing a poor job can “sink” the group, better students don’t trust others to do the work to their satisfaction and will do a disproportionate share and then resent (and complain about) having to do it.

Based on our experience, free-riding based on parceling out assignments to group members occurs when either of two conditions exists: 1) assignments involving tasks that can be completed by an individual member (i.e., group interaction is in fact not needed), or 2) assignments involving a great deal of writing. Because writing is inherently an individual activity, the only real group activity will be deciding how to divide up the work.

For a variety of reasons, this kind of free-riding is virtually never a problem with team-based learning. One is that the RATs so powerfully illustrate the value of give-and-take discussion in tackling intellectual problems (e.g., over 98% of the teams will outperform their own best member. Another is the multiple incentives for completing pre-class assignments. The most important reason, however, is that application-focused assignments provide both incentives and opportunities for face-to-face interaction because they are designed around reaching decisions (not producing a lengthy document) and are largely conducted during class time.

The other problem that many instructors view as free riding is when students get higher grades than their individual scores would seem to merit. In most cases, this results from using a traditional grading scale (i.e., 90% = A, 80% = B, etc.) when a significant part of the grade is based on team work. Since the teams do such good work (average team RAT scores are typically at least 15-20% above average individual scores), a low-scoring member of a high-performing team can end up with a relatively high grade. The best way to avert free-riders of this type is to make sure that the mechanics of the grading system take into account the fact that group scores will normally be higher than individual scores (see grading system discussion below).

Return to questions

Question #3 – Will students be motivated to prepare for group work?

This concern typically results from a combination of instructors’ lack of faith in students to complete pre-class assignments and the frequency with which they have to deal with complaints about free-riders. What the teachers lack, however, is an understanding of how much their own failure to create conditions that foster individual and group accountability has contributed to the problem.

With team-based learning, a combination of factors, many of which are listed in the answers to the two previous questions, encourage members to be individually prepared for class. Further, because group performance is a major component of the course grade, students are motivated to prepare for and participate fully in the team assignments. They realize that team members will “sink or swim together.”

Return to questions

Question #4 – Can I resolve interpersonal conflicts that will inevitably arise among group members?

This concern also typically results from a lack of understanding of the tremendous negative impact of poorly-designed group assignments. Most of the really troublesome cases of conflicts in learning groups are associated with assignments in which students are able to parcel out the sub-tasks of the assignment. In some instances, the conflict is over either which member gets to make the decisions as to who does what or who gets the easiest (or hardest) part. In other cases, the conflict arises when members are impatient with members’ failure to agree on and/or meet deadlines. Occasionally, the conflicts are content-based and result from differences of opinion about the quality of each other’s work.

With team-based learning, lots of disagreements occur, but neither process nor content issues are significant problems because of the frequent and immediate feedback on team work. For example, although each RAT provides multiple opportunities for both process and content conflicts to occur, disagreements of both types are quickly resolved by feedback resulting from the scoring of the exams. As a result, team learning instructors almost never have to spend front-end time coaching groups on to how to resolve conflicts or mid-course time adjudicating problems that arise from group melt-downs.

Return to questions

Part III – “How to” Questions on Implementation

Question #1 – How should I form the teams?

In forming groups, we recommend trying to do three things. One is spreading assets and liabilities (i.e., background factors that are likely to make a difference in students’ performance in this course) across the teams. Assets are often such things as attitudes toward and/or performance in previous course work, course-related life experience; liabilities often include such factors as no (or poor) preparation in related courses, language barriers, etc. Thus, the team formation process should be criterion-based. A second objective of the team formation process is to avoid pre-existing, cohesive sub-groups (e.g., a group of three students from the same fraternity and three students who did not previously know each other would probably struggle). For these two reasons, teams should not be self-selected. Third, the process you use for team formation should foster the perception that none of the teams was given a special advantage. Thus, we recommend using a very public team formation process.

As a practical matter, I literally have students form a single line, in order of a set of categories with the “most rare but important” category first. For example, in undergraduate Management Principles courses the rare and important category is usually full-time work experience, so that is the first category. (By contrast, I rarely use work experience as a category in graduate courses because most students have worked for at least a year or two.) The next category for undergraduates is typically “Those who were born and raised outside of Oklahoma and the immediately adjacent states.” Students array themselves from most to least distant; this makes it possible to spread out English-as-a-second-language students without specifically singling them out. Next, I have accounting and finance students join the line (they tend to see things in “black or white”) followed by management and marketing students (who are more comfortable with various “shades of gray”).

In graduate classes my most important but rare category is typically Ph. D. students and/or students from programs outside of business undefined e.g., I often have several law students and want them in separate groups. As a result, I start off by asking students to raise a hand if they are from outside the college of business. Then I have them line up alphabetically by discipline. Then I invite the “born and raised outside of Oklahoma and the immediately adjacent states” category to come next and the final set of categories has to do with undergraduate majors.

When the entire class is standing in line, I then have them count off by the number of groups I want. When the counting-off process is finished, I have the groups assemble themselves in designated locations in the classroom.

Return to questions

Question #2 – How big should the teams be?

The size of the teams always represents a compromise between being large enough to have sufficient intellectual resources to complete the assignments and small enough to develop into true teams. Historically, we have found that if teams have at least five members, they usually have the intellectual resources to complete the team assignments. On the other end of the spectrum, we have found that groups larger than 7 tend to have difficulty in the team development process. Hence the optimum size for team-based learning is 5 to 7 students.

Return to questions

Question #3 – What should I do if students have difficulty with the required pre-class reading?

In some cases, students’ reading limitations are a very legitimate cause for concern. These include: difficult subject matter, poorly written reading materials, insufficient reading skills, limited [English] language skills, and physical and/or mental handicaps. Although the answer as to what to do about it depends, to some extent, on the reason(s) that students are having difficulty, the severity of the problem can be greatly reduced by one or a combination of the following:

Return to questions

Question #4 – How do I set up an effective grading system?

An effective grading system for team-based learning contains three essential components: individual performance, team performance, and peer evaluation. The individual performance component provides a basis for student accountability to the instructor and to each other. The group performance component provides incentives for the development of group cohesiveness and justifies putting effort into group work. The peer evaluation solves two important motivational problems. One is providing an incentive for individuals to participate in group discussions. The other is removing the students’ fear that they will have to choose between getting a low grade on the group assignments and having to “carry” the group work (if other group members fail to do their fair share).

The final weight of each of these three components should be a balance of three factors. First, each of the components should be given enough weight so it is clear to students that the instructor thinks it is important. Second, the instructor must be personally comfortable with the relative weights in the chosen grading system. Third, the grading system must be responsive to student concerns for fairness and equity.

In our classes, we involve students in the development of the grading system through an exercise called “Setting Grade Weights”. This is an exercise in which we set maximum/minimum limits for each component of the course grade, and then representatives of the groups negotiate a mutually acceptable set of weights for each of the grade components. Over the years, it has proven to be a highly effective way to create a grading system that is both acceptable to all the parties involved and supports the individual and team behaviors that are necessary for learning and team development.

Return to questions

Question #5 – How can I provide grade-based incentives for group work and still avoid giving higher grades than the weaker students deserve?

A relatively common problem with new team learning users is finding a way to count group work in their grading system without giving higher-than-deserved grades to some of their students. The apparent dilemma is created by the fact that most groups will score over 90 on most tests and other assignments. Further, even though team-based learning increases both attendance and pre-class preparation, users sometimes find that when a large percentage of the grade is based on group work, locking themselves into a 90 = A, etc. grade distribution forces them into “giving away” grades to students who really do not deserve them.

One way to at least partially avoid the problem is to not use percentages as the foundation for determining students’ final course grades. Even though I carefully explain my rationale to the students, I still get some initial resistance. But it dissolves as students come to realize that you will be fair with them. (Note: Students are almost universally convinced that there is something magic about a 90. In my experience, however, it is really an arbitrary cut-off because most instructors try to peg the difficulty level of their tests in order to give the “appropriate” number of A’s. If the test turns out to be too difficult, they curve it up, give an easier test the next time, or both. If the test is too easy, they give a harder one next time.)

As a practical matter, I generally work with an overall 1,000 point base and also “1,000-point within category” base. Thus, if group analysis of a movie or novel is worth 30% of the group part of the grade, I treat it as though a “perfect” score would be 300. I also have just over 100 questions on my total of 6 RATS and the Group RATs are 30% of the group grade. At three points/question, that adds up to be about 300 points for the groups. As a result, I simply have my spread sheet add up the total of the raw scores on the six RATs.

At the end of the semester, I multiply the subset totals by the weights determined by the class (e.g., if students have set the weight of the group part of the grade at 65%, I simply multiply the total of the raw group scores by .65). Then I sum the weighted points from the three major grading components (individual scores, group scores and, peer evaluations). This gives me an overall score for each student. I then have my computer sort the students into a distribution based on this weighted total score and look for “break points” for A’s, B’s, etc. at approximately the level that I think represents a “fair” overall grade distribution. I typically take three factors into account. These are: 1) grade norms for similar courses [I use this as a base.]; 2) my personal impressions about how well the students have performed relative to other classes that I have taught, and; 3) extenuating circumstances (e.g., textbook problems, weather problems, etc.).

Return to questions

Question #6 – Should I give a RAT every class period (or every week)?

This answer is an unequivocal NO! In fact, when someone says they tried team-based learning and they got a negative student reaction my first question is: “How many RATs did you give?” At least 80% of the time they will answer that they have given at least 12-15. If you give too many RATs, you are likely falling into a trap of pushing students into memorizing details that are not really significant to either you or the students.

The primary purpose of the RATs is to assess student readiness to engage in related application-focused activities, all of which are open-book. The RATs should focus on ensuring that students have an understanding of the basic concepts that is thorough enough to allow them to: 1) tackle the application-focused assignments, and 2) use their assigned readings as reference material to digest additional detail as they work. Thus, the test questions should focus on the concepts that would be found in a typical Table of Contents, not just in the Index. In my judgement, focusing on details is counterproductive for a variety of reasons. First, it limits the amount of material that students are willing to be responsible for, because they feel like they have to memorize everything. Second, if your goal is long-term recall of the material, then you are better off really reinforcing the fundamental concepts (the details are not all that important – partly because they are always changing – plus, students will never retain them anyway). Third, they can pick up – and are more likely to retain – the details when they are encountered in a meaningful context as they are working on the applications.

Return to questions

Question #7 – How much should the RATs count?

A primary benefit of the individual RATs is that they make members accountable for their individual preparation. Students realize that, because of the give-and-take that occurs during the group test, they will not be able to hide a lack of individual preparation for the group RATs. Thus, regardless of how much the individual scores actually count toward the course grade, they make members accountable to their group. As a result, as long as the group RATs and the peer evaluation both have a significant impact on the course grade, it is not essential that individual RATs count very much at all. On the other hand, even though we typically develop a grading system using an exercise that gives students the option of not counting the individual RATs at all, they rarely choose that option. Instead, they typically set the weight for the entire set of individual RATs at about 10% and the group RATs at about 15% of the overall course grade.

Return to questions

Question #8 – What should I do when students are absent?

In general, I try to create a situation that is similar to what students will experience in the workplace. Thus, I say something like:

Then, if they do need to miss a RAT or some other graded activity, I give them the team score and also allow them the opportunity to take the individual test using one of two options. One is to take the test early so that I can let their team have access to their individual answers during the team test. The other option is to take the test later. Some professors are uncomfortable with the idea of giving make-up tests. They worry that allowing some students to take a late exam will give them an unfair advantage over other students but see the task of creating comparable but different exams as extremely time-consuming. It turns out that neither of these problems are as big as they seem.

In my case, I do not create an alternative exam. If a students misses a RAT when it is given in class (and few do), I leave a copy of the regular exam with the departmental secretary. Then, any student who needs to take a make-up RAT contacts her to make arrangements to take the exam in a nearby room. In the many years I have done this, the vast majority of students taking such make-up exams get scores that are lower than their average on the RATs they take at the normal time. This means I can simply use the same exam for the make-up exam, and doing so does not give students any significant, unfair advantage.

If allowing make-up RATs turns out to be too much of a problem, another option is to set up the grading system that allows students to drop one individual RAT score. When students calculate their overall score for the individual RATs, for example, they would only count the best 5 out of 6. If they have no absences, they get to drop their lowest score. If they have one absence, they simply have a “0″ for that score and drop it. If they have more than one absence, they get to drop one of those but they have to calculate the other “0′s” into the total. The advantage of this approach is that it reduces the hassles associated with giving make-up exams. The disadvantage, and one that I personally feel is important, is that it tends to eliminate the opportunity to reward students who are willing to put in the effort to be prepared on every test.

Return to questions

Question #9 – Should I use peer evaluations on all projects?

Although it is essential to have a peer evaluation that counts at the end of the course, we see arguments both ways with respect to every project or even mid-course peer evaluations. The strongest argument in favor of frequent peer evaluations is that they give members earlier feedback on their behavior and hence an earlier opportunity to change it. Some students are honestly unaware that some of their behavior is seen as counter productive by their peers and are willing to try very hard to change, when they learn that change is needed.

There are, however, two arguments against frequent peer evaluations as well. One is that having members evaluate each other tends to disrupt the development of group cohesiveness. The other is that we have seen a few instances in which an assertive member received high evaluations for taking charge on early assignments. This encouraged the person to dominate the group even more on later assignments but led to the person receiving the lowest peer evaluations in the group at the end of the course.

Thus, if you choose to do peer evaluations along the way, we recommend: 1) providing a mechanism for giving anonymous comments; 2) making sure that members give each other both positive and negative (i.e., “room for improvement”) feedback; and 3) counting the later feedback the most.

Return to questions

Question #10 – How should I handle the scoring of appeals?

There are several issues in relation to handling score changes that result from appeals. However, in every case we think it is important to make sure that we are not reinforcing a behavior that we really do not want. For example, we make it clear that any team has the opportunity to submit appeals; but if they choose <strong”>not to appeal, we hold them responsible for their choice (i.e., an appeal granted for one team will not improve the scores of other teams). In doing so, we also make it clear that it would be unethical to teach in a way that implied that it is OK for students to sit back and let others do their work for them. Similarly, we only accept appeals on the questions the teams have missed; allowing individual appeals would eliminate the incentive for members to stand up for their point of view during the team test. However, if we grant a team appeal, we also change the scores of each member who had the same answer as the team. Finally, we do not take away points from a member who had correctly answered the question in the first place.

Return to questions

Question #11 – How should I give feedback on multiple-choice questions?

There are two aspects to the feedback process. One is the actual scoring of the tests. The other, which is by far the most important, is providing students with information on how their overall performance compares to other teams and whether or not each of their answers was correct. The simplest way to score the tests is using a portable test-scoring machine. We use a Scantron 888P+ scoring machine that we actually take into the classroom. This allows us to score the individual tests while students are taking the team tests.

We provide immediate feedback on the team tests in one of two ways. The most effective method is using IF AT answer sheets (Epstein, 2000, http://www.epsteineducation.com) to provide both immediate feedback on a question-by-question basis and partial credit if students have partial, but imcomplete, understanding of the concepts on which the questions are based. With IF AT answer sheets, students scratch off the covering of one of four or five boxes in search of a mark that indicated that they have found the correct answer. If they find the mark on the first try, they receive full credit. If not, they scratch until they find the mark, but their score is reduced with each unsuccessful scratch. The other method is having a member score their answer sheet on the Scantron scoring machine as soon as the teams complete their test. We then have the team member post the score on the board or on an overhead transparency (to provide the inter-team comparisons) and pick up the team folder containing the individual (already-scored) answer sheets.

Having immediate access to both the individual and team answer sheets provides immediate feedback on two key aspects of their work. One is their level of mastery of the assigned readings. The other is whether or not they effectively incorporated the input that was potentially available from each team member. Further, because the feedback is literally instantaneous, the IF AT answer sheets have a particularly powerful and positive effect on the teams for two quite different reasons. One is that they very rapidly learn the value of voicing and listening to each others’ ideas. As a result, they are able to develop and benefit from effective interaction patterns in a fraction of the time it would take in a less feedback-rich environment. The other is that, even if they don’t find the correct answer until the second or third scratch, members experience, and are motivated by, a series of immediate successes from working together.

We have also discovered a way to promote team development by limiting the feedback we provide in machine scoring the answer sheets. If we give information on which answer was correct (which is one of the automatic scoring options) teams often fail to discover situations in which the team missed a question that was answered correctly by a team member. On the other hand, when we limit our feedback to simply identifying the incorrect answers, their first question to each other is, “Did anyone get this one right?” Thus, with no coaching at all, teams obtain the feedback that enables them to develop a decision-making process that is effective in obtaining input from even their most timid members.

Return to questions

Question #12 – How can I provide immediate feedback on the RATs if I don’t have a scoring machine?

In reality, actually scoring the individual tests isn’t essential for either learning or team development. What is important is providing immediate feedback. Thus, in large classes, if we do not have a scoring machine, we do not actually score the tests until later. Instead, we have team representatives turn in their team folder containing the individual answer sheets before they start the team test. When the teams finish their tests, we provide an answer key that they use to “score” their team test as soon as they turn in their answer sheet. Further, as long as members record their answers on the test itself (which they have to retain to take the team test), providing feedback on the team tests also enables members to “score” their own individual tests

You can also use class discussion to let students “discover” the correct answers. This involves having teams alternate in giving their answer to the individual questions and, when teams disagree, moderating the discussion between teams. Further, because the teams are challenging each other (and not the teacher), it is also possible to use the discussion to grant credit for an alternative answer in the same way as would normally be done with written appeals (i.e., declaring more than one “winner”).

Another alternative is quickly hand-scoring the individual tests using an answer sheet with companion answer key in the form of a transparency overlay with clear windows (for correct answers) and shaded boxes (for incorrect answers). Then you can let the teams use the overlay to quickly score their own tests.

Return to questions

Question #13 – What are the benefits of using a “split answers” format on the RATs?

Early on, most team members are somewhat timid about challenging each other out of fear of being perceived as overly aggressive. As a result, newly formed groups always look for way to incorporate everyone’s input without having to risk a confrontation between members with differing views. With normal true/false and multiple choice questions, their natural inclination is selecting answers by a majority vote after a very brief discussion. However, voting with minimal discussion tends to hamper both learning and team development because members tend to withhold information that would stimulate give-and-take discussion. Further, the fact that voting often produces the correct answer tends to obscure the importance of obtaining input from everyone in the group.

Once we understood the dynamics of the process, we were able to develop a procedure that largely solves the problem. Our solution is providing a way for teams to incorporate everyone’s input without having to risk a confrontation between members with differing views. This procedure highlights, instead of obscures, the importance of getting input from every team member. With this system, we have each question count for three points. If we are using machine-scored forms, we have students answer each RAT question three times (i.e., as though it was three separate questions and number the questions accordingly). Thus, the first question would be numbered 1-3, the second would be 4-6, etc. and we enable them to “split” their answers by filling-in up to three different answers to each question. For example, if they want to “hedge their bets” on the first question (which we would have numbered 1-3), they might want to answer A on answer spaces 1 and 2, and B on answer space 3. In this case, if the correct answer was A, they would get 2 of the three possible points. If the correct answer was B, they would get 1 point and they would get no points at all for a C, D or E. If we are hand-scoring the questions, we have students distribute three points in the boxes on each row of the answer sheet.

Early on, most groups choose politeness over rightness by splitting answers to smooth over member differences. By allowing split answers, however, we focus their attention on the fact that they incur a cost (i.e., the loss of a point) every time they reach a decision without exploring the reasons behind members’ choices. This, in turn, provides a powerful incentive for members to start talking out differences instead of sweeping them under the rug. As a result, teams quickly develop to the point that give-and-take discussions are the norm and they are rewarded by increasing their ability to incorporate members’ input into their decisions.

Return to questions

Question #14 – Are there alternatives to using multiple-choice questions on the RATs?

The critical attribute of any RAT question is whether or not it can be used to provide immediate feedback on both individual and team performance. Although multiple choice questions are particularly well-suited for providing immediate feedback, other types of questions will work as well. For example, one faculty member hands out a sheet containing “fill-in the blank”, short answer essay, and other questions that are designed to test concept understanding by completing simple computations. When students complete their individual RAT, they slip it into a photo album page so that it is visible on both sides but they won’t be tempted to write on it during the team test (same questions but on colored paper). When they finish their team test, the teacher gives them a copy of the answer key that his graduate assistant will use to correct the test. After comparing their test to the answer key (which provides immediate feedback on both the individual and team tests), students have the opportunity to fill out an additional form on which they submit any “appeals” or comments that they would like to have considered in completing the grading process.

Return to questions

Question #15 – What kinds of questions are good questions for the RATs?

In addition to providing the basis for immediate feedback on both individual and team performance, the test questions should also emphasize key concepts (i.e., avoid asking questions about inconsequential details) and enhance learning. With respect to enhancing learning, one characteristic is that at least some of the questions must be difficult enough to stimulate discussion. Otherwise, teams will simply defer to their best member. In addition, using related questions that require increasingly complex levels of understanding are particularly helpful for two reasons. First, if the questions are correctly chosen and sequenced, students can learn from the questions themselves while they are taking the RAT. For example, by asking one or two recognition-type questions followed by a question that requires synthesizing the concepts from the two earlier questions provides students with the opportunity to develop a deeper understanding of the concepts themselves. Second, questions that require higher level thinking skills are far more likely to stimulate the kind of discussion that promotes peer teaching.

Return to questions

Question #16 – Can I use a writing assignment for the application phase of team-based learning?

In many ways using “good” in relation to “writing assignments for groups” is an oxymoron. Writing is inherently an individual activity. As a result, group writing assignments typically set in motion a series of events that: 1) tend to inhibit learning, and 2) often foster very negative attitudes toward group work. Learning suffers because when groups are assigned to create a lengthy document, the only rational way to accomplish this overall task is to divide up the work so that each member independently completes part of the assignment (usually the part that he or she already knows the most about). The negative attitudes are caused by the fact that if any member does not do a good job on his or her share of the writing, his or her peers will be forced to accept a lower grade or engage in a last-minute attempt to salvage a disaster. In fact, high-achieving students often express the feeling that getting an acceptable grade on a group term paper feels like having crossed a freeway during rush hour without being run over.

Return to questions

Question #17 – Is it OK to use group presentations?

Group presentations have both a positive and a negative side. The work leading up to a group presentation is usually more productive than writing a group paper. Because they realize they will all be in the public spotlight at the same time, members tend to spend more time working together to make sure they are as least somewhat familiar with the major content issues. As a result, the preparation phase of group presentations often enhances both learning and interpersonal relationships. Unfortunately, the actual presentations are often close to a waste of time. For listeners, it is just another lecture except that, they usually have serious doubts about whether or not their peers are expert enough to be taken seriously. As a result, the energy level is so low that, in far too many cases, the only ones who benefit from the experience are the presenters.

Return to questions

Question #18 – If I can not use group papers or presentations, what can I use?

In general, the value of group assignments is largely a function of the extent to which they cause members to engage in give-and-take, content-related discussions. As a result, the best group assignments require groups to make decisions that involve complex applications of course concepts. The thinking required is simply too complex to be handed out to individual members working on their own. Further, in addition to having the teams make a specific decision, having them all: 1) work with the same problem or case, and 2) simultaneously report their decisions dramatically increases the value of the group assignments for two quite different reasons. One is the resulting give-and-take discussions between teams produces timely, content-related feedback on students’ concept understanding. The other is that it builds commitment to and enhances the problem-solving capabilities within the teams.

Return to questions