Dave: [00:00:00] Today in Episode 165 of the Teaching in Higher Ed podcast “Teaching Lessons from Course Evaluations” from Bonni Stachowiak.

Production Credit: [00:00:09] Produced by Innovative Learning, maximizing human potential.

Dave: [00:00:18] Welcome to this episode of Teaching in Higher Ed. This is the space where we explore the art and science of being more effective at facilitating learning. We also share ways to improve our personal productivity so we can have more peace in our lives and be even more present for our students.

Dave: [00:00:46] Well hello to everyone. This is Dave Stachowiak and I am having the privilege of getting to speak with someone you’ve probably heard a bunch on the show, Bonni Stachowiak. But we’re turning the tables today right Bonni.

Bonni: [00:00:58] We are. Speaking of listening. I’ve heard from a couple of our listeners who have listened to every single episode of the Teaching in Higher Ed podcast. One guy told me he thought it was the Bonnie and Dave show in the beginning it’s the first five episodes or just the two of us so well you sound familiar to that guy but the rest of them you might be the person they haven’t heard as much from lately.

Dave: [00:01:16] It is always fun to come back and as I listen to every episode of the show and every time I learn such new things new ways to look at things some of the recent episodes you did on Wikipedia and using YouTube videos and all that just fun ways of utilizing technology in the classroom. It’s just I takeaway something each time from it. It’s fun.

Dave: [00:01:36] But today we’re going to turn the tables on you a bit. And I know one of the things that you think about a lot of course is the feedback that you get from students and how you can utilize that feedback effectively. And there’s also been a number of episodes you’ve had in the past where you’ve talked about the way to navigate handle respond to course evaluations and also some of the best practices around that. And it’s changed your thinking on how you handle this and so I know one of the things your interest in doing is going through it actually making this really what’s the word tangible specific relatable to exactly which you’ve been receiving as far as feedback so I think we’re going to look at some of the feedback we received from evaluations this past semester and just talked to what are some of the what are some of the ways you’re processing that and handling that especially in the context of what you’ve learned hosting the show over the last couple of years.

Bonni: [00:02:29] I want to mention that there is an excellent episode with Betsy Barre and she goes into the research on course evaluations and I’m going to link to that in the show notes which will be at teachinginhighered.com/165 because if what you’re looking for is research on them is there bias. Are they good measures. She does a phenomenal job and there’s a lot of links in that episodes as well so this is much more of a reflective episode because yes we can read all the research and work that’s out there but they’re talking about us and they’re talking about our teaching and they’re talking about things that if you’re going to try to do well you’re going to be vulnerable and pursuing and we thought we’d take the risk and do something a little bit more personal on this show.

Dave: [00:03:11] So before we get into a lot of the details I am curious on a 30,000 foot level how do you handle evaluations as far as your own process and just the process of how your university handles it too.

Bonni: [00:03:21] One of the things that I do from a practical standpoint is they have a place for them. My evaluations are now done electronically although they were not always done that way so sometimes they get the paper copy and then I’d scan them. But there is a place that all of my course evaluations are located at least by institution. I teach occasionally as an adjunct but I’ve got you know at my main institution I have one folder where I can go to and get all of my evaluations for one from one place and that makes it really easy then if you’re ever trying to go up for tenure or promotion or if you are looking to apply to teach somewhere else then you’ve got all the information you need as far as Course evaluations because it’s very likely you’ll be asked for them in either of those processes.

Dave: [00:04:01] And what are some of the techniques that you’ve found helpful or useful in perhaps that you’ve learned from this show where you’ve taken and utilize but I’m also curious just over time and teaching now for 10 11 years of full time. What techniques have you found have been helpful.

Bonni: [00:04:18] Well there’s a couple of things in terms of just what’s not helpful. Let me start there. What’s not helpful is to just see that the e-mail came in with the course evaluations and you know if perhaps this is if I blocked out time to look at e-mail for an hour or whatever I’m in the mode of processing email.

Bonni: [00:04:38] It’s not helpful for me to just open those up right when they come in and be expected to process them actually glean anything useful from them and then I’m off with my day. So I have to separate it I put them in the place that that I’ve designated for them. I try to pick the quote unquote right time to review them I have to be in a good space to do this. And I will say it. I should say from the get go. I get very good Course evaluations and I’m not saying that to be egotistical. I’m saying that because despite the fact that I get very good Course evaluations there can be one comment that will just sit with me and I’ll grieve over it. You know what I mean like all in all they’re hard for me.

Bonni: [00:05:23] They’re really really hard for me despite the fact that time and time again I mean they’re just I receive wonderful feedback from students so it’s it’s like I have to be in the right place to do it. And I actually have to sort of phases that I go through.

Bonni: [00:05:37] One is a quick dive which would be just so at a separate time not just when the e-mail comes in or they’ll say oh your course valuations are now available and you’re learning management system because that’s where ours are now placed and then. So I just download them put them away and then I’ll have the quick dive of like go in. Skim it. Are there any huge major things and then just kind of sit on it and then maybe a week or two late or schedule some time to really think through now what.

Bonni: [00:06:04] So I actually have two times that that I read them. There’s the quick glance and then there’s the OK here’s here’s the actual.

Dave: [00:06:11] When you do the quick dive. Is that literally when the e-mail comes in. Or if you set aside time to do a quick dive as well.

Bonni: [00:06:16] It depends on what space I’m in when I’m checking my email when I know that they’re available ya know how disciplined I’m being too because we want to take a peek. Right.

Dave: [00:06:24] Yeah yeah. So you do a quick dive and then there’s a second time. This is the part that I definitely have not done before but you take that’s that second time. Tell me more about what you do during that time where you block off to go in short to the deep dive.

Bonni: [00:06:38] Well I think the first time that I read them it’s emotional it’s personal it feels personal it shouldn’t. I mean I wish I wasn’t like that. It seems like I would be a better person if it wasn’t personal but it is. So then the second time I’m trying to be more objective about it what does this feedback actually say as opposed to you know what does it say about you as a person in your work. And I really like that like that just helps me do it by having a little bit more space there so I look at it both from a qualitative and quantitative standpoint.

Bonni: [00:07:07] I tend to find more value personally in the qualitative because one of the things I believe that Betsy said was that there’s not so much difference between the quantitative results between this is not her wording but this is my memory of like the good ones versus the not so good ones there’s not that much of a difference quantitatively speaking. And since I tend to get pretty good evaluations Anyway I don’t I can’t see that much. And what Sometimes I’ll see and we’ll talk about this a little bit more in these evaluations would be like 95 percent of people strongly agree. And then there’s one person neutral and you know how this goes. What am I thinking about that 95 percent of the results are only in Greek or the one person who said neutral. What does that mean. What is it like what is that.

Bonni: [00:07:54] And that’s why by the way I think separating them out helps. And then they have a more of an objective look. OK what what do we want to do what we want to take away. I wanted to mention my first manager when I graduated from college a man by the name of Kent North wonderful wonderful leader I’m so glad to have gotten to work in under his leadership for a year.

Bonni: [00:08:14] My first first first year of working professionally and I’ll never forget because I taught computer classes and I taught a group of twenty four people and there are these two women that were so incredibly rude and they talked the entire class and were just and this is back in the day when you actually taught 24 people Microsoft Word where people clicked along with you now click on the follow me and they were not clicking along.

Bonni: [00:08:41] And they both gave me straight fives and I was so used to getting straight 10s that were they were these calm smile sheets you know they were not very meaningful evaluations and I will never forget I went to his office and I don’t think I was crying. I don’t think I cried. I don’t remember ever crying but it sure felt like kind of you know what I mean. He must have known that I was upset and he had me actually physically hold these two evaluations in my hand.

Bonni: [00:09:05] They’re sitting on top of my hands and he says I want you to tell me is there anything worthwhile you can glean from this feedback is this helpful to you. And I’m holding them in my hands and no there’s nothing helpful about this feedback. And he took the two pieces of paper together and he lifted them up and he went into the trash can they went. Which by the way was a complete not what you’re supposed to do. I was like all the evaluations went into a locked box at the front of the it was like a very taboo for him to have done that.

Bonni: [00:09:42] And symbolically I’ve carried with me all the time and to picture myself like holding it in my hand. Is there anything worthwhile you can glean from this to make you a better teacher. No there’s not and then just the power of of that was very. Because I know that early in my career I taught computer training. How did I get evaluated. Very much on. Could I stay engaged energetic for eight hours.

Bonni: [00:10:06] I mean there was a lot talk about. So was definitely about the customer is always right and trying to be people pleasing then it wasn’t really about deep learning. It was can you get people to follow steps for eight hours so I probably had some bad training around that. But I always found that useful just in terms of holding them in my hands. And can you get anything worthwhile out of this qs a teacher.

Dave: [00:10:27] Well it illustrates I think another point too is that it is one data point to what you were just talking about as far as teaching computer classes. A lot of it was driven around how much people liked you I haven’t heard this term in a while but “entertrainment” right? So making it a fun experience.

Bonni: [00:10:43] I never heard that term before. Edutainment, yes, but no entertrainment.

Dave: [00:10:46] Where it was a fun experience, “I like the instructor.”

Bonni: [00:10:50] Yes.

Dave: [00:10:50] But doesn’t necessarily correlate with how much better I am for example that case and Microsoft Word so.

Bonni: [00:10:57] And now I’m laughing because I’m holding about six pages of notes.

Dave: [00:11:03] I was wondering how you were going to navigate that.

Bonni: [00:11:05] Now this is going to be interesting as we go.

Dave: [00:11:08] Since they’re now torn apart. So let’s get to some of the oven. I think you first heard that by the way of like how you process that. So let’s get to some of the just the logistics of what are the courses that you taught this pass semester.

Bonni: [00:11:22] So the ones that I’m mostly going to be talking about I taught two sections of an undergraduate Consumer Behavior class so this is an elective for our business majors whether they’re business majors or marketing majors yeah, Consumer Behavior. And then I also taught a section of a Leadership and Technology class for another institution and in their doctoral program.

Dave: [00:11:44] So two very different kinds of students. Oh probably couldn’t be more different in fact.

Bonni: [00:11:50] If there are more different students. I have not encountered them yet.

Dave: [00:11:54] So you made a distinction earlier between quantitative and qualitative and I think most people probably appreciate those who are listening to the show but just for those who this may be new language for the quantitative as the numbers pieces right the readings that you’ve got of you know on a scale of one to five those kinds of things the qualitative is the written responses in most evaluations I’ve seen. Most institutions have some combination of both. So first of all from a quantitative standpoint what are some of the trends that you noticed.

Bonni: [00:12:20] Well in terms of the way that these different institutions evaluate things. I just want to comment that that if any of you are ever in a position to have an influence on the questions that are asked. My goodness the more is not the merrier here.

Bonni: [00:12:38] And I find so much less value in the institution where I teach in the doctoral program. There’s just pages and pages and pages of quantitative and they just they don’t they don’t really provide good value. So I’m thinking of it as just just myself. How do I get the most out of these. But I was going to just mention that it’s hard to get a lot out of them if there’s so many granular questions because they’re sort of the you lose meaning in the data and then of course the students get exhausted.

Dave: [00:13:09] So one of the things I’m curious about in the second institutional where it’s page after page and page of information. So as a faculty member who is getting all that data like how do you approach that then what do you do to make some sort of like what am I going to pay attention to versus what do I get. What am I gonna set aside.

Bonni: [00:13:25] Well I’m going to separate it because it was different. So for the main institution where I teach the Consumer Behavior class one of the things that I found helpful in this particular instance is that the quantitative scores tended to map pretty well to the qualitative feedback and that’s not always the case because because I’m not always able to see you know where things happen and I’ll give you an example in the Consumer Behavior class and I’ve talked about this before on the show and have blogged about it but I did sort of a version of choose your own adventure assessment. And they got to you know decide what. And they called them development opportunities but they got to decide what kinds of things they wanted to do in order to earn their points. And then I also did a combination of being able to drop their lowest scores and also human development opportunities.

Bonni: [00:14:18] It wasn’t as crisp as I might like to be it was the first time I was trying it and I had I had an Evernote note where I had linked to it in the learning management system to this Evernote note and that ad development opportunity one and then the choices of four or five choices they could do development opportunity to four or five choices three or four etc. and then it just it got I didn’t fill out all of five of them as the semester started because I wasn’t didn’t have my act together enough when the semester started. So sometimes I would be kind of behind. And then since people could drop their lowest score not everybody was going to do four and five.

Bonni: [00:14:54] So just got to be like it wasn’t terrible but a couple of the neutral scores instead of the strongly agree were on things like had to do with being organized into that kind of thing or assignments were clearly described in those like like again still high scores but I love it to be perfect because that’s something that I should be able to do is get a perfect score on that I described the assignments well. So there was a couple of times where I was like that was kind of nice or I was like OK. And you know what else Dave is that if I were to go back and do it again I would still take the risk to not have everything as completely organized as I might normally.

Bonni: [00:15:35] For what it offered in terms of what it gave them or agency comes up in the qualitative rankings and in some of the other quantitative rankings too of of that the assignments were really helpful. They were returned in a reasonable amount of time that I was available for help outside class like all the things like that I was like yeah good. I mean it’s great that those things were there. Yeah.

Dave: [00:15:56] So if I may perhaps coach you for a second the a one of the concepts that’s really popular in the software world right now and and entrepreneurs is minimum viable product and there are creases and I forget what’s the name of the book Lean lean starts didn’t think. And so I think these concepts actually really helpful because I think about it like through the lens of like what James Lang teaches on his book Small Teaching of it likes starting to do something different and pushing yourself in you and I were talking about this around leadership earlier today of the importance of feeling uncomfortable and pushing yourself forward.

Dave: [00:16:32] And so it almost would be it would almost be like if the scores turned out perfectly. It’s like you hadn’t pushed herself to try something new. And so it’s if I can offer this good for you that you tried something new and yet didn’t go perfectly. But it would have been weird if it had gone perfectly. Yeah. And instead you tried something new you learned a bunch. You found a few things that didn’t work and then that’s the next iteration of how do I take this from being a minimum viable product to now a better product the next time and a better product the next time and you know the continuous improvement that you’ve talked about and somebody I guess talked about in every episode of How to do that for each iteration of a class.

Bonni: [00:17:08] I’m so glad that you brought that up and I’ll be put in the book and share notes because people might be interested in learning more about that but I haven’t actually read it myself but a number of people have referenced that and certainly read articles about it and that kind of thing.

Bonni: [00:17:19] It’s changed my entire thinking on how I approach business and projects. And I think there’s a lot for faculty that book even though it’s not written it’s not a book for faculty. There’s so much there are just that just that concept of thinking about small iterations versus trying to nail something the first time around.

Bonni: [00:17:39] The thing I was going to mention about the other institutions of this would be that doctoral students in the quantitative feedback. It was one of those classic a smaller class I think 12 or 13. And so I would have 12 strongly agrees and one neutral 12 strongly agrees and one knish all 12 strongly agrees on one neutral and that was I mean 12 people can I admit that I can’t help but think about who that one person might be. And pretty sure I probably know that. That’s a really hard demon to. I don’t know how I could ever get that out of my own head. And yet what was really cool. That I thought yes was 13. Instructor was respectful of the students. 13 showed enthusiasm and interest. 13 showed. Oh provided extra assistance if asked for. So it was kind of nice where you are like yep yep yep and this was the institution that asked 9 million quite like you know eight million ninety nine thousand of like everyone says strongly agree on one person neutral neutral neutral neutral.

Bonni: [00:18:50] But then when I got to like the things that are just core to us treating other people with respect and us you know showing up if you will for the work that kind of thing. I got that. That’s pretty cool that I could overcome the person’s concerns about my teaching or what have you that felt like. All right. Well all right. What I mean so let us call.

Bonni: [00:19:11] Yeah. So getting beyond just the quantitative numbers a qualitative. What are some of the themes you found and maybe first of the positive themes.

Bonni: [00:19:21] There’s just some really nice feedback. I tried to not just brush by these quickly because we need these we need this to nourish our soul. When you do the hard work that we do we’ve got to sort of be like a sponge and soak some of these extra kind of fine words up because we’re also going to hear some not kind and some discouraging words that will make us feel like we’re not having the impact that we want to have. So it was just nice I got you know she’s one of the best teachers Doctor B is the best. I mean it’s nice to see that I loved everything about this. So those were cool cool words to read.

Dave: [00:19:57] You’ve been doing this for a while now and you’re the best teacher I know. And I think as a parent to anyone who’s listening show for while you’re very talented at what you do from a teaching standpoint what are the what are the themes you pretty consistently hear in your evaluations that you are really good at as a teacher.

Bonni: [00:20:21] Being energized engaged in students caring for them showing that I care.

Dave: [00:20:26] All right. Well the other side. So what are some of the constructive criticism feedback maybe even things that were harder to read or hurtful that you heard in the most recent evaluations.

Bonni: [00:20:40] I wanted to talk about this today and I will. I want to start just by saying it’s hard and that I had done this deep dive and have been I think I did that maybe three or four weeks prior to us having this conversation and some of this stuff just has been walking around with me. And then as I was preparing for the episode yesterday I did sort of a second deep dive which I don’t normally do. And it was the weirdest thing because I have been carrying something around that isn’t in the evaluations.

Bonni: [00:21:16] So this is like you know they do those studies on how terrible we are as police witnesses because it’s so easy to have something be suggested. And then for us to accept that as what really happened and you know I know all of this from watching crime shows on television so it must be true that my my body is the classic for people where people see an accident there’s four different versions of the story.

Bonni: [00:21:43] So I need to tell a very personal story and that I will explain why I am telling this personal story because it involves why I carried something around these last three weeks that simply I cannot find in the evaluations that I thought was there. And this involves you and I asked permission for me to tell this story before it started because I thought.

Bonni: [00:22:02] So when I was in my mid to late 20s I had my biggest heartbreak of my life. And of course now I couldn’t be happier about this horrible heartbreak because if that hadn’t happened I wouldn’t be married to you. I would be married to someone else so thank goodness because you’re a really great husband. So we’ll start with that.

Bonni: [00:22:25] But at the time I didn’t know you I didn’t know you were coming along. So this was hard. So I was starting to be on my healing process and totally never see this person anymore like and a whole different part of my life. And someone says like oh do you ever talk to me. No no I don’t. Like why would I still talk to him. I ran. To him. He said he broke up with you because you were too nice.

Bonni: [00:22:50] There was something so raw about that that brought up all these feelings of like… First of all how horrible is that. I never got any kind of explanation. But now that’s like what we’re going to ascribe it to but then that this person was telling the other people I was just ashamed we just went through all this stuff. So all of this to say I have a trigger about being too nice.

Bonni: [00:23:13] That is still something that makes me feel shame when it really shouldn’t. And I know that it shouldn’t. But that’s not great so I thought three weeks ago when I had read this that somewhere in the evaluations and had said she’s too nice as one of the criticisms of my teaching and I’ve been like OK. I’m going to tell this story in the episode and you know I have to get days permission make sure he’s OK you know like this whole thing like I read it and I read it and I read it and I read and I read yesterday it’s not in their favor they’re not there.

Bonni: [00:23:48] Isn’t that weird. I would have told you it’s there and I still kept doing research in the text search wasn’t always working. But like in some cases where I couldn’t find anywhere there is a comment about wishing I was stricter. She should be more strict.

Dave: [00:24:06] Which is funny to me because when I think about you in the context of what I know about the institutions you teach your colleagues you’re one of the stricter people I know them as far as handling things and being very caring of students but also at the same time being very purposeful about evaluation standards and all that and being willing to engage in difficult situations so that’s a that’s an interesting thing that it comes up at all is interesting to me.

Bonni: [00:24:35] Yeah. And then there was also thinking that the class was too sheltered or something like that so it was kind of interesting. Interesting set of comments but it was a long story. But to move on to some of the other things that showed up as confusing hurtful or hard ones to process oh my gosh this next one I’ll read it word for word.

Bonni: [00:24:57] The only thing that disappointed me is that for the poster sessions you didn’t walk around and look at everyone’s posters. I know we all spent money on materials and put time in to them and would have liked you to at least see it in person. This one’s so hard. So I had combined both of the sections of the classes together and we put on an awesome event. We had I think almost a hundred people there and there were alumni coming in there were business professionals coming in we did a Facebook Live thing that I blogged about. That like went crazy bad enough to post about all my Facebook Live failures.

Dave: [00:25:33] Because it was 90 degrees – angled the wrong way…

Bonni: [00:25:37] Yeah but I just I saw this and I was so sad.

Bonni: [00:25:42] I was so sad because I thought I watched the whole video and I was so proud of all of you. I smiled from ear to ear as I watched every single clip of that video on its side. [laughing] And I did it in private and they wouldn’t have known that I would have put that much effort into celebrating their learning made me so sad… so sad… And then especially talking about them spending money and I know some of our students just don’t have a lot of money and that they wouldn’t have known that I treasured that. We had a student who’s a professional photographer. She took these amazing pictures and I just smiled ear to ear at just celebrating their learning.

Bonni: [00:26:34] But I wasn’t purposeful enough about getting to every single booth… I don’t even know if that would have been possible because there were so many of them and the event wasn’t really that long. So I just have to think about this for next time. I never want to read that on a comment again.

Dave: [00:26:53] You know there is a I’m not saying this to try to make you feel better necessarily but there’s an aspect of this. I mean I think about parenting and marriage the things you and I think about a lot of our relationships with each other and with our kids and like there’s times of both of us do things that we think a lot about and really touched about and the other person never notices or seemingly never notices. And that’s just part of the human experience you know and it’s really it really sucks when it happens.

Bonni: [00:27:29] There were a couple of comments for people that just to out of I think 40 to 50 evaluations have been like that. I hate group projects. I did the majority of it myself that way and I don’t think very much about it because they had a choice. Those was poster sessions that I just mentioned they could work as groups of two but they all could do it by themselves. And some people did. So I’m like hey that’s all on you. Like I’m not I’m not taking that one myself. If you chose to work as a group and you did the work yourself then like I. Yeah. Anyway.

Bonni: [00:28:03] And then I guess the yes the other one is that sometimes in teaching doctoral students the level of perfectionism is new heights that I’m not accustomed to dealing with generally speaking with undergrads if you have people that are very achievement oriented like I am. I care a lot about my grades. I know you do too. Dave that generally speaking that starts to happen when they get below an A cause that affects the GPA.

Bonni: [00:28:30] So if it’s an A-minus then like there’s lots of tension there but there’s sometimes comments that are confusing to me from the doctoral students because if they didn’t get a perfect score. And I am always thinking but but so you still got an A on the assignment and you still have an A in the class. So it’s hard for me to just wrap my hand head around so the best ways to address that.

Bonni: [00:28:52] So there was a comment my low scores were due to the syllabus and assignment expectations needing to be updated on blackboard. I feel that Dr B explained that change as well. But when we went to work on our own we all heard different information through our discussions we confused each other and this is something I won’t spend too much time on. But I had there was like a setting inside a blackboard where I thought I’d change the deadlines for things. But then there was another place you had to change them on spoiled by Canvas because you just change in one place and it changes everywhere else but there was you know the problem there and I know at my own reflection right when you have problems like that you’ve got to document it don’t wait to get the evaluations I already knew this was an issue but document. Oh remember when you change dates you got to change here and go change it over here so you don’t forget next time. So that one wasn’t that surprising but it is a little weird like but it is hard.

Bonni: [00:29:44] Thats always a harder one for me to spend some time thinking about and I think just going and finding other people that more regularly teach doctoral students can be helpful and also talking to other people that teach within that specific program because programs create their own cultures. And since I’m the third of the fourth class that they have they’ve already had a lot of cultural learning and norms established that I need to be aware of to be more effective in my teaching.

Bonni: [00:30:08] I’m thinking about this in context of what you were talking about earlier as far as the you know trying something new and making some mistakes and not going perfectly. Our higher education system at least here in Western culture has trained a lot of us who are high achievers of like A is good and anything else is you you messed up. If you get a B or C or in this case a hundred percent is good. A 100% is good for an A-minus like whatever it is. And so and that is not at all how the world works in any capacity. Right. I mean anything you’re applying for a job or even in higher education or starting a business it is a lot of 40 percent days. There’s a 70 percent or 60 percent you have a 90 percent day occasionally.

[00:30:51] It is learning and being challenged all the way in a a I almost I mean this is an episode about rethinking the entire evaluation system of the heart of higher education of course but I think it’s it’s to the extent that we can find a way as educators to make space in our classrooms under discussion was for 87 percent day or a 73 percent day and that to not be a horrible thing that yeah there’s a GPA correlated with that but that there’s also real learning that happens and I know that’s hard. I’ve I’ve I’ve navigated that myself. But I just I put that out there is something for us all to think about because that is the world that the students are going into all of the students. We all have had the privilege to work with.

Bonni: [00:31:38] I need to get Ken Bauer back on the show because he recently has been blogging about and tweeting about just doing away with all of that. It’s just it did it or you didn’t. We’ve talked before about spects specifications great too. But yeah there’s a lot to think about here for sure.

Dave: [00:31:51] You recently read Stephen Brookfield’s on Becoming a Critically Reflective Teacher would’ve been some of your reflections as you’ve reviewed the evaluations this time around in the context of his work.

Bonni: [00:32:04] One of the things that really came out was he has a critical incident survey and he administers these regularly at least once a week in all of his classes. And so one of the things that comes out is that for a couple of things it’s not major but for a couple of things I would have known about these things. Oh my gosh I just realized there was a big one that I didn’t mention a big one. Another failure was it was it was couched in a complement so it was like I love this class. It was awesome.

Bonni: [00:32:34] But when you play music when we’re doing quizzes my brain gets fuzzy and I don’t play music when they’re doing quizzes. But they were referring to quizlet let the flashcard app has a game that is awesome. And I’ve blogged about that. I’ll post a link and the show notes but it’s incredible.

Bonni: [00:32:53] And I realize that for some silly reason I hadn’t really thought about like music of course that’s cognitive load and if you have 100 percent musical intelligence like I do from the Gardner intelligence survey it’s going to be all your to be thinking about is the magnificence of the music you’re hearing in the lyrics and that’s going to take you on all of these memories and places that you’ve been so like of course I would never play music. Well students were taking the test or if there were some kind of real focus they needed to do. But in this case I sort of lost the comprehension of that because I thought oh we’re playing a game. Oh.

[00:33:30] So it’s a game that you play quiz live you play using the flashcards and the students by the way love love loved it. There was a huge hit. And even this student clearly had a good experience in the class. But I’m like whoops don’t play music while you’re playing live. Just because you’re having fun doesn’t mean you need to play fun music.

Dave: [00:33:47] The music isn’t part of the game. The music was something you added it. Got it.

Bonni: [00:33:51] Yeah. I just thought like oh fun background noise. I’m not going to do that again. So at any rate that particular piece of feedback I was also disappointed to hear that because I was like oh man I should know better than that. Lesson learned but I should have learned that earlier in the semester when we first started experimenting with the tool because had I asked using his critical and he has this on his Web site you don’t necessarily have to buy the critically reflective teacher in order to have access to it.

Bonni: [00:34:15] It’s on stephenbrookfield.com which I linked to on the show notes. But you can download it and just regularly administer you know what’s happening in the classroom what is contributing to people’s learning. And this is similar to Gardner Campell talks about an Apgar which Apgar is a medical term for measuring babies when they’re born you get an Apgar score was.

Dave: [00:34:35] One to 10 or something like that.

Bonni: [00:34:38] And then if it’s too low that it was invented by a nurse and they’re you know their name was after oh I didn’t didn’t I think I was like sure Apgar stood for something. No I think you’re right anyway. Yeah. So at any rate that would have been another tool that I linked to in the show notes so we could use to gage you know how things are going so you don’t have to wait until the evaluations come out and then the other thing that was sort of interesting for me to reflect on a little bit.

Bonni: [00:35:03] He’s such a brilliant writer and I love his transparency and his rawness and he inspires me in that way to be vulnerable as I’m talking with you today about these you know very very personal experiences have reflected on the work the mission the reason. You know one of the reasons why I’m on this planet is to teach.

Bonni: [00:35:22] And so he talks about he has kind of visceral reactions to people who have one teaching awards and teachers who are considered more charismatic and because he doesn’t consider himself that way although I think he is very charismatic but he talks about you know that we don’t want to be aiming for that kind of stuff because then we become sort of too perfectionistic we become too focused on our own people pleasing. Being like you know entertainment that kind of thing and that sometimes we’ve all experienced this that when you’re asking students to do that deeper learning you’ll have the resistance and they’re not necessarily going to be happy with you every day the class you know I’m exhilarated by your teaching that kind of thing.

Bonni: [00:36:11] And so it was sort of interesting for me to think about yeah I have won some pretty big Teaching Awards and I have I have struggled at times with people pleasing you know that just having that be hard. Like I feel like I can’t reach this person and that’s hard for me when I feel like I can’t. And so I guess how do I translate him. Because he doesn’t see himself that way with me like Oh I think I might be one of those teachers that you’re describing that you don’t really seem to care too much for.

Bonni: [00:36:41] I’m not phrasing it very well I should have probably highlighted a few quotes of him actually in his own words but I think I just when I think about my own role as a teacher but also as someone who coaches faculty I just wanted to reflect on people that really do get bad evaluations because clearly I had had some emotion in this episode which has never happened before in 2009 and clearly this is hard work.

Bonni: [00:37:06] But my gosh Dave what about the people that really get bad evaluations it’s not just one comment it’s the whole thing has gone haywire with a qualitative and quantitative and it’s hard to get stuff out. So I just wanted to say if that’s you that. So glad you’re listening to this podcast. I’m so glad that you’re trying to do things to enhance your teaching and that you really can get better at this stuff.

Bonni: [00:37:30] This is not something that’s unattainable and and that you’re always going to be learning and never feel like you’re done. But I’m wishing you the best. I guess the next time and hoping that this show provides some things for you for encouragement. Some things for you some practical things that you can do. And I’m just so glad that we’re all together in this community.

Dave: [00:37:49] And that leads us to the recommendations portion of our episode.

[00:37:52] My recommendation is quick and easy keep an encouragement file for yourself of really nice things that cards that students give you or words that they’ve written and both physical if they give you a physical card but also have it digitally so you can incorporate it into your portfolio if you need to do that. But I’ll tell you what that is like medicine to the soul.

Dave: [00:38:13] I have a e-mail folder for that I have a I think it’s titled Love and it’s just it’s not just love notes from you. Well some of those. But any time I get a really nice e-mail that’s about me I put it in there and I use that once in a while if I’m having one of those days I’ll go through and I’ll just look through a few of them and you know it really does. It really does help.

Dave: [00:38:33] So my recommendation is a concept that I think a lot of people have probably heard of and it relates right to our topic today of just a very what I found to be very simple but also very powerful way to engage evaluations during a course during the semester. It’s just called Stop-Start-Continue.

Dave: [00:38:51] And for those who weren’t familiar with it it’s there’s a lot of ways you could do this the way I have done it is I’ll usually get about 40 percent through a course. And I tend to when I’ve taught adjunct it’s tend to be more accelerated classes where there’s only five class meetings. So I would often do this either at the like the end of the second class meeting or maybe the beginning of the third classmate and so not quite halfway but enough time that there’s you know we can still tweak can change things. And so what we do is spend maybe 10 minutes in the class of and I’ll ask the students who will put up on the board.

Dave: [00:39:21] I’ll make three columns of what is something you see me doing that you’d like to see me stop doing what something that you don’t see me doing that you’d like me to start doing that will help you to get more out of this class. And then finally what do you see me doing that you want me to keep doing. Continue.

Dave: [00:39:35] You know we’ll spend a few minutes and I’ll ask everyone you know just and I’ve done it different ways I’ve done times I’ve left the room and I’ve let students put things up on the board. I’ve also done it where I’ve been there and you know I write the things that they say and I haven’t. For me at least I haven’t found a big difference between those so I’ve tended to just stay in the room and have that conversation with them versus having them put it up on the boards and then coming back to it. And then we talk about it and so we and what I find is you know there’s always someone who puts up something silly like you know I’d like you to stop or not have us do this next project or next assignment or whatever.

Dave: [00:40:10] Then once you get through a few of those then there’s always something that I’ve found that comes out of that that’s a really helpful comment in some way. And so when they’re all up on the board we spent a few minutes just kind of blasting them up and I attempt not to comment on them while people are putting them up so we’ll talk about stop and start and so just list them up. And then what I’ll do is I’ll go back and we’ll spend a few minutes.

Dave: [00:40:32] I’ll address each one and attempt to address each one as a one of three ways either I’ll say that’s really valid especially if it’s a stop or start. I will do that and I’ll make a commitment to do that. And of course the key is you have to really do it because if you don’t it it’s worse than if you hadn’t that if you had not done this at all because then you’re not you’re not looking like someone who’s been responsive to feedback you’re not going to hear it again from people.

Dave: [00:40:57] The second category of things is that’s interesting. Let me think about that and then the third category is and this is really important too is that’s not something I’m going to change and here’s why. And so that way there’s some explanation as to. Yes I found this to be you know a comment that’s helpful. But here’s here’s why I have to either not made that choice to do that in this class or I’m not going to take that advice. So that way it acknowledges the feedback that’s given even if you plan not to do anything with it because you don’t want to leave something up there not to comment on it.

Dave: [00:41:25] And then people three weeks later saying like oh we told you to do this and you didn’t do it because if you’re not going to do it just be honest why you’re not going to do it. And I found that to be really helpful and sometimes nothing comes up that’s really substantial within that interaction. But inevitably a week later someone will make a suggestion in class.

Dave: [00:41:42] You’ve set the standard now in the classroom. I’m willing to have a conversation about how I’m doing as an instructor outside of just the sheet you’re going to fill out after you don’t see me anymore because then I can do something with that and I always aim to find at least one or two things I can tweak immediately from that conversation. So students realize that I’m I’m learning to loner to grow and change and adapt.

Bonni: [00:42:06] Thanks so much for being on today’s show Dave and for advice and for all your coaching and good questions.

Dave: [00:42:14] Hey thanks for being on your own show.

Bonni: [00:42:16] Hey. I make a great guest.

Dave: [00:42:18] You do make a fabulous guest

Bonni: [00:42:19] I come very prepared with samples that.

Dave: [00:42:22] You should think about.

Bonni: [00:42:23] Theme music.

Dave: [00:42:23] You should think about hosting the show.

Bonni: [00:42:25] I really should. How next week?

Dave: [00:42:27] Sounds good. So if you found today’s conversation helpful I hope you’ll go on the teachinginhighered.com Web site because you can sign up to receive in your inbox each week a article that Bonni writes as well as the show notes with all of the resources that we mentioned each week you can go to teachinginghighered.com/subscribe. Plus you get a very helpful PDF guide with lots of educational tools that Bonni put together a while back. Lots of good stuff there and we’ll see you in a week, right?

Bonni: [00:42:56] Yeah see ya next time, we will see you next time if you want to consider joining the slack channel that’s there at teachinginhighered.com/slack to find lots of ways to get engaged. Thanks so much for listening.

Dave: [00:43:05] That’s a fun channel.

Teaching in Higher Ed transcripts are created using a combination of an automated transcription service and human beings. This text likely will not represent the precise, word-for-word conversation that was had. The accuracy of the transcripts will vary. The authoritative record of the Teaching in Higher Ed podcasts is contained in the audio file.