Bonni [00:00:00] Today on episode number 259 of the Teaching in Higher Ed podcast Dr. Natasha Jankowski shares about intentional and transparent assessment.

Production Credit [00:00:14] Produced by Innovate Learning, maximizing human potential.

Bonni [00:00:23] Hello and welcome to this episode of Teaching in Higher Ed. I’m Bonni Stachowiak and this is the space where we explore the art and science of being more effective at facilitating learning. We also share ways to improve our productivity approaches so we can have more peace in our lives and be even more present for our students.

Bonni [00:00:51] Today I am welcome in to the show Dr. Natasha Jankowski. She was introduced to me through my partnership with the Association of College and University Educators, or ACUE. ACUE’s courses and community site feature many of teaching and learning top experts, faculty developers, and practitioners to showcase evidence based teaching practices. For more than three years ACUE has connected me with great guests for the show.

Bonni [00:01:19] Dr. Natasha Jankowski is Director of the National Institute for Learning Outcomes and Research Assistant Professor with the Department of Education Policy, Organization and Leadership at the University of Illinois at Urbana Champagne. She is co-author, along with her colleagues, of the books Using Evidence of Student Learning to Improve Higher Education and Degrees that Matter: Moving Higher Education to a Learning Systems Paradigm. Her main research interests include assessment, organizational evidence use, and evidence-based storytelling. She holds a Phd in Higher Education from the University of Illinois, a Master’s in Higher Education Administration from Kent State University and a B.A. in Philosophy from Illinois State University.

Bonni [00:02:08] Natasha, welcome to Teaching in Higher Ed.

Natasha [00:02:11] Thank you so much for having me.

Bonni [00:02:13] It is great to be connected with you and especially because today’s topic is such an important one in our work. Yet I’m reminded of your criticisms that many of us faculty really mistrust this process- I know some of my colleagues will think like “why are they making us do this?” and “they don’t understand!” Who is the they here? And why do we have this mistrust of assessment? And by the way, you say it’s a warranted mistrust.

Natasha [00:02:38] Yes. Very very much so. I think I sort of feel bad for faculty on the one hand. I am a faculty member as well. The amount of things we’re asked to do and sort of the narratives around us is being stalwarts to change and raising important questions about why are we doing things that we’re doing? And I think for administrators if the value and worth is not clear, faculty will start asking you questions. But it’s sort of pushed off on as just jump in and do this. And why are you not interested in being a team player?

Natasha [00:03:09] But there is, I think, a good reason for the mistrust that faculty have around assessment. And part of that is faculty are constantly assessing their students learning in their classes. There’s consistently making changes. They modify the syllabi. They change an assignment in class when there’s a discussion going, they’ll switch it up. And none of that seems to count, that something else needs to happen and that the expertise and knowledge that they bring as educators is not enough to justify that students have learned that you need something else. And so I understand that while faculty constantly assess, they hate assessment of student learning as there’s a bureaucratic enterprise.

Natasha [00:03:50] And part of that is it started out as being externally required. So there was this movement in the 80s, regional accreditors that guess what- assess learning- faculty were like “I’m sorry, what was that?” And then you had a slapdash some learning outcome statements together that weren’t really about what you valued or really taking the time to think deeply about why do we want our graduates to do when they came out of a program? And then we had to find comparable measures to assess it.

Natasha [00:04:16] So you get in this sort of externally not well thought out statements of goals coupled with a history of assessment as this measurement tool where it has to be a standardized test or it needs to be a validated instrument. We can’t trust grades. There’s too much noise in them because it could be class participation and that’s not related to learning, that you need to have the same rigour as the research projects and very quantitative focused. So all of our qualitative or humanities faculty that just research in a different way, we’re going “where do I fit in? I don’t have the skill set. I wasn’t trained in this way.” And then that sort of perpetuated by the feedback we give as peer reviewers- so as a fact, like we peer review other programs in the accreditation process and we say “well you’re not doing it right,” as though there’s a right way to do it. And we need to see something else. And only certain types of evidence count. And so I think faculty feel that most administrators speak out of both sides of their mouth because they say find the values and things that you want to have learning, go about assessing it in any way that you want but only if it looks like this. And I can compare it to another institution. And so there is a good solid history there that I think is very warranted for the apprehension of the faculty.

Bonni [00:05:31] You really emphasize having an intentional approach in our assessment and I certainly have seen evidence that is not always so and we might describe it a little bit more as haphazard. And I think some of that comes maybe from the apathy, at least what I have seen in institutions from this mistrust and feeling like you’re just this cog in a wheel and it’s not it’s not meaningful work. So how do we become more intentional? What does that look like in terms of assessment?

Natasha [00:05:58] Sure. I think it’s about realizing that it’s not about the cycle process of completing an assessment cycle in a year. That’s really about assessment as a reporting structure. That I don’t care what you’ve learned, I just need you to have this data and you reported it and all this and we know that change doesn’t happen in a year but report on it anyway. Or maybe you’re still thinking about what this means. But report on it. And so some of it has been I think a realization that people need time to make sense and that if we don’t know what we’re going for, if we don’t have a collective shared understanding that we can’t assume that, then we really need to think about is this the learning that we desire? Do we have that shared understanding of what it is that that means? And do I understand how my course fits into the larger picture?

Natasha [00:06:39] And that type of how am I part of a larger whole? How do I set up to pass a student to you that can be successful in your class that you can build from? And thinking about it as that developmental journey, there’s not time and space for that. It’s the constant turning of the doing. And so finding time and space for reflection is to say if our assignment there maybe what we want our students to do, then how does that align with our learning outcomes? And do we need to change that? Or if our learning outcomes are these statements, is that really what we want? Or is that just the thing that was easy or measurable or whatever?

Natasha [00:07:14] But asking those hard epistemological questions of our design then we can actually say this is what I was going for. How do I build that in? And it becomes much more an instructional design and intentionality piece. So if I’m asking you to do something in my class, I can tell you why I’m asking you to do it and how it relates to something else. And that’s not to say that everything has to connect to some other thread. There are some things you do in your course that are unique to that course. There are tasks you do because it’s just fun. Or I just need to know if you do the readings so I can rethink how I’m going to teach that day. Then that’s not an assessment. That’s important things for my pedagogy and helping you scaffold your learning. But if we actually want assessment to count in those ways that we’re driving towards and making that clear for our learners, then we need to have the time, have reflective conversations in our practice in our educational design. So not a measurement piece in that sense but really a design conversation around assessment.

Bonni [00:08:09] You’re reminding me of some of the things that have been shared in the past. Michelle Miller when she was on, spread the heresy- I’m being sarcastic but she talked about- in particular this was about her book called Minds Online. And specifically in that that instant she was sharing that not every single thing that you might assign has to necessarily be graded. If we’re talking about a large group of people perhaps it’s just that you did it and perhaps there’s some spot checking. And I’m getting I’m not oversimplifying what it was she was trying to share. But it’s just reminding me of- you mentioned scaffolding- I think sometimes we might think we have to be assessing every single thing we might have a student do. But sometimes those things that students do are building up to the learning that can then be demonstrated in a more formal assessment. I think that’s a really important point.

Natasha [00:09:01] Yeah. I think I completely echo that because if that practice- how are we preparing you to be ready to demonstrate you’re learning at a particular level? And if I know that I have this culminating assignment or I want you to run this group project, what am I doing along the way to give you the chunks of knowledge needed? And I don’t need to assess that you’re on the right sort of developmental trajectory. I need to give you opportunities to practice and flex those muscles. And in that sense that learning curve movement through it. But what I really want to see is when you get to this point, can you do it? And how well? And where are you struggling?

Natasha [00:09:33] But I shouldn’t be surprised when you get there because I’ve been with you all along the way. And I think our prior sort of approach to assessment was we’d check at the end and be like “oops, 40% of you have no clue. 60% of you got it. Sorry. Have a nice life. We’ll make changes to the program for the next group.” And you were surprised by the results because it hadn’t been that ongoing look along the way. And so I think this is much more assessment WITH not TO students. And thinking about how this is an active part of their learning journey as much as it is ours to understand our design is crucial.

Bonni [00:10:06] Well and you’re describing what- I mean if we do have a failure, the ideal scenario is that we change our approach the next time. But that doesn’t even happen as much as I wish it would because sometimes it’s just then blame the students. “Oh they’re underprepared.” And it’s like well that’s kind of- it’s an assessment sometimes for the students yes, but also for us to be learning from because ideally we wouldn’t- like you said, that’s such an important thing of we wouldn’t be surprised by the results.

Bonni [00:10:34] We’ve talked some about what makes assessing learning so challenging. Are there other things we should address in terms of like why is this such a wrestle for us?

Natasha [00:10:44] Yeah. Well I think one huge piece that that always gets brought up in the assessment literature and the scholarship around it in the disciplinary field is the lack of use. So on the one hand, there’s the how do we help train and support people to answer questions that they have about their own practice? Like if I go to a hybrid approach, am I doing it in a way that they’re still getting the learning? Am I switching it so on one hand it’s changing teaching practices, wow is that impacting learning? On another it’s if I create this course sequence, does that help them attain a particular learning outcome in a better way?

Natasha [00:11:18] So being driven by questions of practice, it’s helpful to figure out how do we need to go about looking at that and assessing it? And that’s a skill set that is not part of your graduate program and training. So there is definitely the help in thinking through what that means and how to do that and what makes sense.

Natasha [00:11:35] But then there’s this other huge piece too about. You’ve done it. You’ve got all this data. What do you do with it? What do you change? There’s these theory that change around if something’s not working and your students are struggling, and you see that, how do you decide what lever to pull to get better learning occurring in your classroom? Is that multiple ones? Is a stackable blocks? Like how do you think about it? And that time to sort of say, “what do we think we need to change?” relates to the assumptions that we have about what our students can and can’t do, I think sometimes. And our designs.

Natasha [00:12:13] So we hear a lot “our students aren’t writing well.” I haven’t found a campus where they feel like students are great writers, everyone sort of can come together around students really struggle with writing. But there’s a lot of different ways that people respond to that. So we’ve seen some programs will say OK we’re going to add in a course on writing because there’s this idea that you learn through a course and if they’re not doing it, then they need to run into it in specific course dedicated to that to give them that skill set. Which is not right or wrong. It is a theory of learning and what students need and all of that.

Natasha [00:12:45] Another one we hear is when “we have a writing center. So we’ll alert our students to the writing center. We’ll tell them about it. They will know that they need to go. They’ll feel comfortable to go. They’ll go to the writing center and writing will improve.” But behind that are all the assumptions about student agency, a sense of self, and that I’m ok to go and help, that it is a useful writing center, that I’ve told them that they’re coming. And thinking through who’s responsible for what part of it.

Natasha [00:13:12] So while the actual doing of assessment and thinking about “is this actually a demonstration of the learning that I’m going for at the level I want?” Which is a really hard question in and of itself. Then what do I do if it’s not? And how do I change that? And maybe it’s I just change the way I think about it. Maybe it’s I actively involve students in helping me figure out what I need to do. Maybe it’s a design issue that I don’t have any control over as a faculty member. That can all be incredibly hard and make the larger process of it one where there’s a willingness and a desire to change but an inability to be timely in that or an uninsureness of what steps to make to really get to be supporting the student that you have in the way that you want.

Bonni [00:13:57] We’ve talked about the importance of intentionality but I know another big thing for you is transparency. Talk to us about the importance of that and what that looks like an assessment.

Natasha [00:14:08] Sure. So transparency for me is a big one. I tell this story. I travel a lot. We’re a national research institute. And while we’re housed in a university, it’s very easy for us to think philosophically about assessment and not actually be out talking to folks. So it’s very important that we’re not just national in name, that we actually get out and have conversations. I get tomatoes thrown at me and we have a lovely time. But I want to have those talks.

Natasha [00:14:35] And so I travel a lot. And while I’m traveling I do that short little chat with the people that you’re on the airplane with. Like are you traveling for work? Or pleasure? And I say “for work.”  And they’re like “what do you do?” And it’s really hard to say succinctly in an airplane what I do. Most people think I’m a lawyer because I have these giant stacks of reading. I’m a paper person. So they’re like oh you’re a lawyer. And I’m like no. I run a research institute that studies how do you know if students learn anything in college? Short airplane version. Without fail the people sitting next to me are like “oh my gosh. Yeah, I went to college. I didn’t learn a thing.” It’s heartbreaking. But then they go on to talk about how they’re wonderful creative thinkers, they use information resources really well, they can work in teams, they have wonderful leadership, they’re civically engaged. All of these learning outcomes it was going for. And what they’re talking about, they don’t remember specific content. They don’t remember the year something happened in a history course that they took. But they know how to find it and they know how to argue competing points around it, which is really the parts that we’re going for. But we never told them that that’s what we were going for. It was on a syllabus, maybe, on the first day of class when you’re also overwhelmed with everything in the whole schedule and you’re like “there’s no way I’m going to survive.”.

Natasha [00:15:45] So it’s completely gone. But that intentionality part we don’t unpack for our student where we say “this is what you’re gonna get out of it. This is why I’m asking you to do these things. When you’re reading this, it’s because I want this perspective.” And let’s unpack that intentional design for our learners so that they can really be aware. So also when they leave and they’re having conversations with employers or others, they can talk about not the course they took but what they know and can do and they’re aware of that. And like when I’m out doing this activity, that’s an instance of my writing in this way. And that transparency of the design is really lacking for our learners but incredibly crucial for adult populations.

Natasha [00:16:27] But also just for relevancy of why I’m asking you do it. Like we thought really hard, we’re not just doing that because we thought it was fun. I mean there are some things we do because it’s fun. But as faculty, we’re experts in our area and we’re being very thoughtful in how we go about providing the learning experiences that we do and why we ask students to do particular things. But we aren’t good at talking about it and telling our students about that so then they can go forth and talk about the value and worth of their education because they understand why they were asked to do what we were asking them to do.

Natasha [00:17:00] So transparency for me is more than posting it somewhere or being like I put it in the course catalog which I don’t know that last time you read a course catalog. Not the most riveting content. But that is actually something that we’re regularly communicating with our learners.

Bonni [00:17:16] Well and then the other thing too is what what comes up so much in terms of motivation and learning is the phrase “desired difficulties.” And that’s one of those things as a less experienced person would go like “ah.  They really don’t like this.  This is hard.” And self reporting can show you that I really don’t like this, that  I think that this other way works- like I’m thinking about retrieval practice, and oh highlighting works so much better for me. And actually it doesn’t, but we just think that it does because that’s how we learned or how we think we learned how to learn or what have you. So I’m thinking about people thinking what works. And what are the other things I’m hearing you talk about too and I guess I feel like I have to give a bias warning right now that a lot of my bias is about to happen in the next 60 seconds. And that is just that so much of the time we think the answer then is just to pour more in, we’ve got to cover more. You mentioned earlier, add a class on writing because our students don’t write well. And you said that’s not good or bad. In my head, I’m like “it’s bad! I just know.” But I mean in the sense of just trying to cram more in versus I’m not going to say I’m the expert at this, but I’m experimenting right now with less is more. And I am finding the richest learning can happen when I actually get real. One of the things I use is that Rice University, their Center for Teaching and something that starts with E- CTE- Anyway I will link to this in the show notes. They have a course workload estimater. And that thing just made me get real about reading to me I read really really fast and especially if it’s a book I’ve read many times before and I’m just refreshed from the course I’m about to teach. But you’ll put in there how many pages and how will it be new material for the learner or will it be something they’re already familiar with. I mean it has different factors you put in and then it’ll estimate. And I had to really get real. And I backed way up on the amount of reading I feel like it really increases the likelihood that they’ll do the reading. Of course you have to have other things in place too, not just not one factor I don’t think fixes it but that was a lot for me to say and I haven’t really asked a question yet. But I see you nodding your head. So it seems like you might have something to say.

Natasha [00:19:38] I love this whole conversation part because I think it’s being smart about how we get to what we’re really going for in ways that are beneficial for other faculty memebers. It’s not reading an extra assignment or trying to figure out how to do another course release or not. Should I bring in an adjunct to teach an additional course? But really think about this as a team sport or baton passing maybe in a relay race of learning for students. But one of the things too is we know how to navigate what we’re asking our students to do really well. Like we’re experts in it. You throw a research article down in front of a faculty member, you can analyze that thing in record time because we know what we’re looking for. We know what sections to go to. We know that they have standard sections of things to do. Our students don’t know that at all. So part of it is do we need to give them assignments in like an intro to a discipline course like saying “ok welcome to chemistry. How do you read like a chemist? What do you look for? How do you engage with a text? What does that mean?” Which is incredibly different from how you read a philosophical text where line by line actually matters. You can argue about a particular word but if I don’t know that going in, I don’t know how to optimize my time as a student reader and I’m unsure how to engage. Then I have to sort of figure out that skill set moving through instead of being intentionally like an assignment on how do you engage with a scholarly text? What does that look like? What does that mean? What does a scholarly text even entail? And thinking about it that way.

Natasha [00:21:07] But even we’ve seen some faculty start to realize that they assumed students knew how to study and that they needed to give them assignments on studying. They would take one course time and say here are the top ten ways that people engage in study- highlighting, note taking, all other sorts of things. Where is the effectiveness of this? Which is not very especially over deep retention. You just need to get through taking the exam maybe. But here’s five to try. And so the assignment in this one course which I love is actually students pick a class you’re struggling, like maybe you got a midterm and you were surprised at how you did. And it was not good. Pick two of these new approaches, apply it, reflect on it, we’re going to have touch in points. And then let’s see how that impacted your final in that course.

Natasha [00:21:57] So as a student I can see did this actually pay off better? But I can also try to approach it to say did this work for me? [Does it map to anything] about how I learn and what I need to be thinking about as a learner and how I need to spend my time and actually highlighting is a waste of my time. And so I might be like “but I spent so much effort and hours and that didn’t count.” Thinking about how can we be smarter but how can we help our students with things that they just don’t know and we can’t assume that they know and building those in as assignments to help them be successful so we can do other things with them in the class and in the readings. But thinking about that load of what’s going on the student is crucial. If we’re not thinking about that, then you don’t start getting these innovative assignments of people thinking about how do we help a student navigate.

Bonni [00:22:42] You’ve emphasized intentionality. You’ve emphasized transparency. Another thing I know is important is self-regulation, although that word is like a AH. I wrestle with that word because it just seems so much that we only use that to describe other people and somehow we’re above that. And I sometimes think we might use a dose of self-regulation, whatever that means. So please help us recognize what this means. I know you also think it can be a problematic phrase. And then why is it important in assessment as well?

Natasha [00:23:14] Sure. So one of the things around self-regulation- it comes from a wide variety of different sort of backward angles and some of it is looking at metacognition. So are you aware of how you learn? And so once you’ve done these assignments let’s say on study skills, can you apply those? And are you able to say “stop highlighting Natasha. Get out the flashcards, that works for you. But only if you do them in this particular way with visual pieces.” But can you actually start to change the behaviors on your own? Because there is this tension between how much can an institution or a program do versus how much can a student do to get to that learning.

Natasha [00:23:51] I can drag you across the finish line but how are you helping crawl? That’s what we’re doing. So there is this one part within self regulation on awareness of your role in the larger educational space. However, I think there is tension within self-regulation particularly around issues of equity for learners and assumptions that we have about our students. And it can become a sort of brush off of responsibility by just saying “well we need more self regulated learners” that are normed to look like the particular good type of student that sits in the chair and shows up on time and has done all the reading and has three reading questions ready to go. And that if that’s not happening then that’s on the student to engage with and do.

Natasha [00:24:37] But let me give an example of that. So we were working with a campus that had a high commuter population of students. And the students started showing up late for class. Large numbers of students, 10 minutes late for class regularly. The faculty were immediately like “they need to get on top of their stuff. If they aren’t motivated to be here, that’s not my problem. And I can’t teach people that aren’t motivated to learn.” So this could fall into that “you need to self regulate” camp, show up on time, do all your work and all that. We get there for a different reason and we ask them have you asked the students why they’re showing up late? Like large numbers of you students. And they’re like “no. Clearly-“

Bonni [00:25:17] “The problem is them. Couldn’t be anything else.”

Natasha [00:25:23] No. No. No. Silly people. So we said “well we already have them in focus groups for something else, we will just ask them about it.” And so we asked and lo and behold, the city bus schedule had changes. So this tells me a couple things about this institution. One, the administration is unaware of who their students are. Because if they were, they would have been working collaboratively with the city bus to ensure that scheduling worked if that’s the main mode of transportation for their students. So it’s sort of A) administration, know who your students are. Not the ones that you want, but the ones that you have. And really understand what they go through to show up at your door ready to learn.

Natasha [00:25:59] But two, for the faculty part that tells me that you’ve already thought these students weren’t motivated to learn. The fact that they showed up late proved their point. And that’s a much bigger issue and I think one that gets thrown into this hands up of self-regulation of “it doesn’t matter what I do, if a student doesn’t step forward and do these things then it’s not going to work.” But at the end of the day, our students have a huge amount of barriers and not just one but many. And thinking about our role in lifting them up and sending ongoing sense of belonging to be here, our students know when we think they’re not doing what we want them to do or  “not prepared to be a college student or college ready” or whatever that may be. And I think we get caught up in our practice of research and the day to day and we forget the amount of power we have as individual faculty numbers to send those signals. And it takes one person to say like “I think you deserve to be here and I think you can do it and I can help you with that” or “we’re going to work as a team.” But it’s very easy to default to a self-regulation kind of approach.

Natasha [00:27:10] But that’s not to say there isn’t good important work that’s been done in that space and really understanding how I need to see myself as a learner to be able to engage in making the time in my schedule to study and all of those pieces and understand my learning and really participate. And I think that’s worthwhile. But where I do feel that tension is where it crosses over to brushing off. The really larger deeper conversation of our assumptions that are built in that we need to tackle before we can really help our students learn.

Bonni [00:27:42] Previously on thee show, Jesse Stommel and Kevin Gannon have been just a couple of people who have mentioned starting from the perspective of trusting our students. And what you’re saying is reminding me of it- it doesn’t mean that I think 100 percent of our students will be trustworthy 100 percent of the time. I also by the way don’t think 100 percent of our faculty will be trustworthy 100 percent of the time. But as a starting point for me, because I have taught both ways, boy I’ll tell you it’s a lot easier to look more at the systemic issues if we just start with that point of trusting them. And if we started with that point in the example you gave then we would have been curious of why they were ten minutes late as opposed to I already know why because I’ve got this whole script going in my mind. And that’s not a helpful thing. So I’ve been really inspired by that and it’s it’s definitely transformed my teaching. And I think it’s gotten better results. I think that they can sense that about us and that ties to a sense that we care about them because we tend to care about people we think are trustworthy. And fostering those relationships and then we do know that that ties to learning when a learner feels like that their teacher cares about them. So before we go to the recommendation segment, I’d love to have you share a little bit about Tom Angelo’s learning traps concept and how that comes into play.

Natasha [00:29:03] Yeah I want to tie his concept but I don’t know if you’d agree with this but I can talk to you about it later. But I want to tie it to this idea of that that’s the faculty perception that we have about sort of that trust piece and the role of our students and all that can also impact how we think about assessment in this larger space. If I think you can attain the learning outcome that I have set, if I start from a default setting of everyone in the classroom going to get there this semester or quarter or term or whatever, it is very different from an idea of a norm or some of you won’t and that’s not my responsibility to get you there. So the perspective you bring to being able to make it I think is crucial.

Natasha [00:29:45] But the learning traps fit into that with that same idea of if we start from “everybody can do it”  that it might take some people more time, you might need more or less support, but if we set these learning expectations and you’re walking out with a degree, you’re going to be able to do these things and this is a part of it. Then how do we create learning traps where you can’t escape from it without having learned this material or had these experiences? And so we can actually create transformational educational experiences for folks that stick with them where they keep thinking about it.

Natasha [00:30:14] It’s of the idea like wicked problems that are these ones that just roll around in your mind and you’re thinking about it because it should be where- as faculty we’re inspired by our field and by our disciplines and we think it’s the bee’s knees. We want to talk about it all the time. We love it. We’re constantly thinking about it. But how can you share that love and excitement but instill it in your students. We shouldn’t kill their intellectual curiosity by forcing them to go through rote education. We should be lifting that up and lighting that fire.

Natasha [00:30:44] In the learning trap, we think is on one part be so intentional with our design and innovative in how we’re doing it and involving the students in it that you can’t leave without having learned it. But on the other hand, we’re doing no harm in that process that you’re leaving more excited about education and more confident about your ability to be a part of it and really feeling capable to take your idea and travel down a path and see where it might go instead of saying “I can’t write a book on that. I need to do this because that’s what the class about or what this faculty member thinks or wants.” But really seeing yourself as a part of that scholarly dialogue is in it. That’s a very different perspective. But when I think that if we remind ourselves of what, it’ll approach how we go about this [appraisal of things].

Bonni [00:31:31] I’m trying not to show that I’m laughing while you’re talking because then you would probably wondering why is she laughing while I’m talking but it’s because you’re reminding me of the popularity of escape rooms right now and how and some people who are listening may because we have people from all over the world. Maybe they don’t know what an escape room is. But this is the idea that you pay money to a company to trap you in a room and you have to solve clues and all this. And when I first heard about them, I was just like “that just sounds terrible. Who would pay money to do that?”But part of it was my own thinking of just instantly I went to like “well I wouldn’t be able to get out.” And they’re like “Well no. Everybody gets out Bonni. That’s the thing and they’ll give you hints and stuff.” But I really am intrigued by this because of course learning trap sounds negative. But so does escape room. But people have so much fun with escape rooms because it really is about solving a mystery, working as a team, those desired difficulties. And you’ve really in enveloped people in that learning process. But I will still say that despite the fact that people have now convinced me that they probably are fun, I have not signed up for yet.

Natasha [00:32:41] There’s still time. Actually there’s a cool assignment that a faculty member did at a technical institute in Singapore where he created an escape room and had videos and was watching the students and their final project was actually using physics to solve and get out of the room. And I know of some other faculty who have done one class will design the escape room as their final project, so what would be all the clues or puzzles or things you put in. And then the next class has to solve it. So you end up doing it.

Bonni [00:33:14] Yeah. I love that idea too because you could imagine there being some- because when you were talking about scaffolding and then the whole analogy of the relay race, those kinds of things not only did they help with assessing and learning but they also help with like “Oh that’s going to be me. I see me in them. And I see okay what is this going to look like?” And that just kind of pulling through in terms of motivation, in terms of just self efficacy, I mean there’s so much that I imagine comes in when people do that. What a great idea. I love that.

Natasha [00:33:46] I also love the idea of deciphering just a random tangent but where students are graduating will write letters to incoming students with things that they wish they had known or that if someone had told them this and they didn’t believe it or understand it. So there’s this really nice sort of passing of knowledge. Or even when you get more senior students to evaluate the work of junior students so they can see like “I used to write like that but look at what I do now.” And see that growth and change over time but also giving peer feedback because it’s hard when you’re in a class with peers on the same level to do meaningful peer feedback, you have to really engage in it, be intentional about how you train people to do it and all of that. But if you are further along and in a more advanced class, you’re in a better position to give meaningful peer feedback in a way too that is like “you’re going to get it and it’s going to be OK. And I know because I’ve been through it.” And I think that sense of community and commodity is really nice in the space that is normally built on competition.

Bonni [00:34:45] Absolutely. I’ll post a link in the show notes to add an example of that that I saw recently it was Laura Gibbs who’s been on the show previously and she used a ed tech tool called pad lint and it’s just an ongoing running stream if you will because every new course that comes in adds their advice to it but it’s advice to students who are going to take the class the next time.

[00:35:07] And so I’ll put I’m not positive I can find it but I am positive Laura will send it to me if I can’t find it so I’ll get it in there and I’ll be something to take a look at.

Bonni [00:35:15] This is the time in the show where we each get to give our recommendations and I have two of them today. The first one is related to today’s topic and actually it came up on the podcast but so much of Natasha what you were talking about really relates to this and I think it’s worth just revisiting in terms of the recommendation.

Bonni [00:35:32] So Maria Andersen was on the podcast. She’s actually been on a couple of times but this most recent time was about ESIL, a learning lens for the digital age. And at the risk of oversimplifying what she’s done here is she’s broken it out instead of just we need to know this or we don’t, she says we need to know that this thing exists. So that’s the E an ESIL. Or the S, that it’s supported that can the learner do this with support as in notes, tutorials and peers. Those are two very different things. This thing exists or I could do this if I had a little bit of support. Then there’s the I in ESIL, which would be independent. So can a learner do it independently without assistance and maintain the skill until the next expected refresh? And then the last one is lifetime. Can the learner do it independently without outside help and maintain the skill for lifetime success? And ever since she talked about this I thought we should be having more kinds of conversations like this in our departments. And on the on the podcast episode, her area of- I mean she actually has like about 14 degrees, but one of her areas of expertise is in math. So she was making assertions about this particular thing you have to be able to do unsupportive, but these other things you just need to know they exist, there are 13 ways to solve for this. And so I thought you might not agree if you study math, you might think we’ll know you’d have to know that for your lifetime but at least have the conversation.

Bonni [00:37:06] So my recommendation today is if this is the first time you’re hearing about the soul scale then go to the recommendations and click on the link to go to Maria Andersen’s article about her ESIL lens. And if it’s not the first time you’re hearing it, maybe you just go revisit it and have a conversation with your department, take a few of your learning outcomes and just just see where do you agree in terms of where things might be mapped to? And then where is there healthy debate that might be helpful in terms of your own assessment so that’s the ESIL Scale.

Bonni [00:37:36] And then my second one has nothing to do with the show but has been really happening in our family a lot. So this is going to seem so silly that I didn’t quite get this. I had heard about that libraries would let you now check out books digitally, but I always had this impression that it would be on a clunky app that I mean that’s like- if it’s not on an app where I can easily make highlights and then have those highlights be curated and exported, that’s not as helpful to me because that’s a big part of my own ongoing personal knowledge management system. So I just thought well if it’s a separate app and I can’t even- it’s clunky, some are like PDFs, trying to read via PDF is just not going to work. Well guess what, libraries have changed. So there’s now an app. The Libby app is an app that you can get through many of your libraries that you can download the books, check them out, but then you can send them over to the Kindle app. You can use the Libby app if you want to read within it but then you also can for those of you that use a Kindle app either on a physical Kindle device or on a tablet. And it works great. And so not only now am I regularly checking out books from my public library on the Libby app but now my kids, talk about agency. My husband Dave downloaded the app on theirs and they have library cards too so they’re able to go and find their own books. And how fun is that when your kids are like “can we read?” It’s great. And some of them will read the stories to the kids, that’s an option on some of the books. And they also have audio books so it’s just great fun. I could go on and on but I should probably pass it over to Natasha for her recommendations.

Natasha [00:39:16] No I love it. I was like writing it down, the Libby app.

Bonni [00:39:18] Yes very cool, very cool.

Natasha [00:39:20] Yeah. I have two recommendations that I would like to put up as examples. One is just ridiculous. They’re both slightly ridiculous, I will just say. I’m aware of that but I’m going to do it anyway. Where in assessment at large, we a lot of times default to this it has to be very quantitative and it’s driven by hard science approaches. So I love to put forward if people aren’t familiar, the dance your PhD Annual Awards that are done jointly by a science magazine and AAS and they come together and people submit doctoral students who have completed their dissertation send a three minute YouTube video of an interpretive dance of their dissertation. So they put like the title and you’re like I don’t even understand the chemical stream of crazy words and then it’s a group of researchers that do an interpretive dance of cell interaction or water deflation and with music. And I like it as an example of there are many ways to demonstrate our learning and just because we pick a few that we recognize doesn’t mean that there aren’t many ways to demonstrate our learning that add value and are really wonderful for students and thinking about how to communicate your research in very different ways I think is crucial and great. And so they always come out in February I think is when the winners are announced. So if you’re not familiar, you can get an alert. And you can see there’s a social one, like which dance did you like. But it’s wonderful. All the videos are on YouTube. Once you start watching, you get sucked in checking out all of these crazy videos people have done throughout the year.

Bonni [00:41:00] I’m having a feeling today is a day when I should be grading. And yesterday was a day when I should have been grading and didn’t do it. And this looks like a perfect opportunity because I have to make the show notes… So if you have to make the show notes, I’m sure you have to watch at least 30 or 40 of these. I’m just- I want to get inside the head of the person who comes up with this. This is magnificent.

Natasha [00:41:21] Oh and it’s wonderful. And that feedback that they hear from their research teams that participated or when you bring in your grad student friends to do it. I mean everyone just loves it. And it is so creative. So Dance Your PhD would be one recommendation.

Natasha [00:41:33] And the other one that I have is a children’s book. The Phantom Tollbooth by Norton Juster. Which if you’re not familiar, it is a story of a boy Milo who goes through a tollbooth into this alternate world where there’s the king of word and the king of numbers and they’re brothers and they’re fighting each other because they’re sisters rhyme and reason has been stolen. And it’s a beautiful expose I think of the way that we debate with each other about quantitative versus qualitative data told through this beautiful children’s story of a boy who works his way through and finds it really to understand you need a combination of both. There’s the island of conclusions that you can only get to by jumping. There’s all these little beautiful tidbits of things but I think is a nice helpful way, that’s a fun way to remind us that to really understand our practice, we need a bit of both. And so making sure that we have space for that is always welcome. The Phantom Tollbooth, a great read.

Bonni [00:42:35] Is this from 1961? Am I looking at the right one?

Natasha [00:42:38] Yeah.

Bonni [00:42:39] I love it. Yeah. So it looks like a classic too. I have never I’ve never heard of either one of these. I have something to do after today’s conversation.

Natasha [00:42:49] And they have- there’s a cartoon version that is made. Oh gosh, I don’t remember when but quite a while ago where it’s like live action and then it goes when he crosses into tollbooth it goes into cartoon.

Bonni [00:43:00] That is so fun.

Bonni [00:43:03] Well it has been so nice to be connected with you. I hope this is just the beginning because I know I have so much to learn from you and so many people did today. Thanks so much for coming on the show and just being willing to share your learning with all of us.

Natasha [00:43:15] Thanks so much for having me.

Bonni [00:43:19] This has been episode number 259 of the Teaching in Higher Ed Podcast featuring Dr. Natasha Jankowski. Thanks so much Natasha for coming on the show and sharing all of your expertise with us about assessment and getting us to think about intentionality, transparency, learning traps, escape rooms, relay races, all kinds of stuff.

Bonni [00:43:43] If you would like to not have to remember to come and get the show notes with the links to some of the resources that Natasha and I shared, please subscribe at teachinginhighered.com/subscribe. And by source subscribe what I mean is to the newsletter has once a week sometimes a little less than once a week, I share an email that has the show notes and also has an article about teaching or productivity written by me. So I hope you’ll subscribe and share the word with your friends.

Bonni [00:44:13] Thanks so much, I’ll see you next time.

Teaching in Higher Ed transcripts are created using a combination of an automated transcription service and human beings. This text likely will not represent the precise, word-for-word conversation that was had. The accuracy of the transcripts will vary. The authoritative record of the Teaching in Higher Ed podcasts is contained in the audio file.