Teaching in Higher Ed

grading

More on blind grading

By | | | XFacebookLinkedInEmail

 

blind grading

On episode #068 Dave and I discussed the biases that can be problematic when working to grade exams fairly.

One technique that we shared to promote greater integrity in our exam grading was blind grading. Not everyone agrees with this method, since there are some downsides.

David Gooblar stresses that despite the challenges that knowing which student is being evaluated presents, we should have knowledge as to which student produced the work we are grading.

Gooblar writes:

If I were to grade blind, I wouldn’t be able to chart a student’s progress throughout the term, from one assignment to another, nor would I be able to tailor my grading to the specific skills each student is working on. – See more at: https://chroniclevitae.com/news/1186-should-we-all-be-grading-blind?

While I tend to grow increasingly depressed each time I allow myself to read the comments section of anything on the internet, I was pleasantly surprised by the nuanced reactions to Gooblar's piece.

One of the commenters describes how s/he attains a balance between the two extremes:

I grade essays blind, but then identify each author when I record grades. So I do know how each student is doing through the course, while still reducing bias in grading.

I realize that, like parents with their children, we should love all of them equally. But, like parents, we are human, and we like some better than others.

I didn't emphasize it in episode #068: my blindness in grading is a temporary thing.

Being able to talk with each of our students, individually, about how we see him/her developing is an important aspect of our roles as teachers. However, doing what we can to ensure that we aren't poorly evaluating their work due to conscious or unconscious biases is also paramount.
Create your own user feedback survey

Continuous improvement in teaching

By | | | XFacebookLinkedInEmail

continuous-improvement

My Introduction to Business students just took their first exam of the semester. For some, it was the first college-level test they have ever taken.

I had a couple of goals I was aiming for this semester.

Avoid surprises on the exam structure

Each semester, I have students participate in an online forum, where they reflect on their first exam. They write about what went well, what didn't, what they will do differently next time, and what I could have done differently to support them in their learning.

The most common factor that students have identified in the past as a hinderance to them doing well on the exam was that they didn't know what to expect on the exam. This has frustrated me in the past, because it tends to come from an external locus of control paradigm. However, I now think I was just being stubborn and could have done something much sooner to take away this as a potential challenge for the students.

This semester, I'm using PollEverywhere even more than in the past. In particular, I regularly let them know that the kinds of questions they see (especially the multiple choice and the true/false) are the same types of questions they are going to see on the first part of the exam.

They also take weekly quizzes and I reminded them frequently that those questions were written by the publisher… the same source for the questions that show up on the exam.

Finally, I had them take a non-graded pop-quiz with an essay question that was in the same style / format as the ones that would show up on their exam.

I'm happy to report that not a single one of my 95 students who are in the three sections of the classes I'm teaching this semester indicated that they were unaware of what the exam would be like…

Success.

Prepare students for application-oriented questions

The other goal I had for this first exam was to better prepare students for going beyond memorization, but being able to apply their learning in a given context.

One of the most listened to episodes of Teaching in Higher Ed is the conversation I had with Tine Reimers on critical thinking.

It doesn't surprise me, as I struggle to develop these skills in my students. I had mixed results on the exam in this area.

One of the questions posed to them had to do with defining corporate social responsibility and then giving examples of ways that a particular company might address various stakeholders' needs and wants.

All students in the classes were able to list the stakeholders. The vast majority were able to give generic examples, such as offering benefits to the employees. There were probably 20% who were unable to make their examples specific to the company, however.

When it came to students being able to give examples for the four types of competition in markets, the success rate was much lower. Students were able to regurgitate the definitions from the textbook (which are poor, at best), but the examples they used demonstrated their lack of comprehension of these types of competition.

For monopoly, students were most often able to indicate that this was a market in which there was only one competitor.

However, their examples either had to do with their preference for a particular brand (“Apple is a monopoly, because no other technology company makes products as well as they do and they are one of the wealthiest companies in the world”), or they had answers like that bottled water companies were monopolies (We had discussed many times about how it wouldn't make sense to have multiple pipes going to your house of water and you choosing which one you liked the best). I guess they had water on their mind, but didn't realize how different bottled water is from the water we pay for as a utility.

David Merill always emphasizes that when we are teaching in the phase he calls demonstration, that we should show the learner not just how to do something, but also how not to do something.

The students clearly didn't have enough examples of what wouldn't fit into the category of each of these competitive structures.

The next time I teach this course, I'm either going to decide that this topic just isn't necessary at the level of introductory business and have them learn it in economics, or I'm going to create an experiential exercise in class to hammer home the examples more.

Creating an in-class reinforcement exercise

I'll bring in a bag full of items for each group of 4-5 students. There will be signs hanging around the classroom for each of the type of competition. Students will take each item in the bag and set it on the table under the appropriate competition type sign. Then, I'll have each group go and stand in front of one of the signs and determine whether or not the items that have been set there are correct for their type of competition.

Competition exercise

If I ultimately determine that I don't have enough time in class to go this in depth on this one learning objective, I can always build some kind of online exercise that will accomplish the same thing.

Building an online reinforcement on competition types

I could have a virtual bag in which the items that require sorting could be housed. Then, I could have some kind of a drag-and-drop interaction that has them drag them over to the name of the appropriate type of competition.

Articulate Storyline is a rapid elearning creation tool that has the capability to build drag-and-drop interactions in online courses. We have been big fans of Articulate's eLearning creation applications over the years. However, they are a Windows-only platform and we have now become a Mac-only household.

It looks like Adobe Captivate also now has drag and drop interactions. Captivate is available on the Mac and PC, so makes for a good option.

Both Articulate Storyline and Captivate are high-end applications and have the price tag you would associate with such advanced features.

Continuous improvement in teaching

I am overall pleased with the results of this first exam. Taking the time to reflect on how I can continuously improve this class after each exam is important to me. I'm glad I invested the time today to consider what changes to make, both the next time I teach this class, and as I continue with these students throughout the rest of the semester.

[reminder]Have you had any exams in your classes yet this academic year? If so, what did you discover? [/reminder]

Feedback and grading workflow guest on Mac Power Users

By | | | XFacebookLinkedInEmail

feedback-workflow

A couple of weeks ago, I was the mini workflow guest for the Mac Power Users podcast, episode 240.

Katie and David were in rare form. It was David's birthday and perhaps nearly the last episode he recorded prior to announcing that he was going solo with his law practice…

I spoke about how to use screencasting to give feedback to students in a way that students report seems more authentic, honest, and helpful.

It's the first 15 minutes of their live show, recorded live on February 7, 2015, episode 240.

Listener Follow Up

I heard from a couple of Mac Power Users, after the episode, which was so rewarding.

David M. from Qatar wrote to say that he purposely doesn't include a track changes document for his students, as he wants to be sure they listen carefully and digest more of the feedback that way.

I hadn't clarified on the episode that I don't do track changes for the purpose of making corrections to the students' papers, either. I use the Word (or other) documents to place written comments or to highlight things.

David described more of his process for me:

Amongst other areas of EFL, I teach a lot of writing classes.

I am always concerned about the level of feedback I give students in their drafts. Too much and I am basically writing it for them and too little and I am not helping them.

I began to do feedback by screen cast using Camtasia and MS Word or PDFPen.

I read the essay before casting and then go through it asking for clarification, correcting errors and pointing out poor vocabulary, grammar, etc. I do type into the Word file but importantly I do not send them back the file with corrections, in fact I do not save my changes.

This forces students to listen, understand and fix their own errors.

I post the videos on Vimeo with a password only the student knows. Vimeo is great because I can track how often the students watch the video (up to 8 times). I do have a paid account and uploading can be a little slow but those negatives are not outweighed by the ease and tracking functions.

Tom S. also wrote to say that he uses screencasting in his business. He finds the feedback gets followed through on much more like his original intent when he uses this form of communication.

Tom wrote:

I am a Sales Vice President for a mid-sized electric manufacturing company in Connecticut.

Over the past year I’ve been using Tapes as a mode to provide feedback to not only my direct reports (regional sales managers) as well as my peers on the executive team at the company.

I do this often with financial reports such as excel documents as well as detailed word documents such as contracts and strategic planning documents.

The feedback that my team has about the process is incredible and I’ve found that the understanding and implementation of the feedback I provide is faster and more in-depth.

Reaching Out

I even got to touch base with Betsy Weber, Chief Evangelist for TechSmith (the makers of SnagIt), who I knew way back in my days in the computer training industry.

She thanked me for the mention of SnagIt on the show and also inquired as to whether or not I had any additional feedback for them.

SnagIt is, and has been for years, a fantastic piece of software. The only minor complaint I had was that it would be nice is there was greater parity between the Mac and PC versions.

Even More Tools

As I mentioned on the live MPU show, I reached out to the Mac Power Users Google Plus community for help on another grading-related workflow.

On the thread, Soram K. from UCLA Medical School reached out to share about other annotating tools he discovered from others in the community that meet his needs and augment the power of SnagIt.

He wrote:

I did find iAnnotate in my own research but do not like to convert the Word file my students send me to PDF. Too many steps!

I am looking at the following as a way that I think will be easier with a desktop screen writing app that will record in Snagit.

Here are some links…

Next Steps

As those of you who know me (or are getting to know me through the podcast and the blog) will predict, I am thrilled to discover these new tools.

However, I am going to force myself to hold off until after the semester ends before doing too much experimenting. It is too easy to go down a rabbit trail and lose time on the most important priorities for now.

I just finished reading the book Essentialism and am even more reminded of the importance of saying no to things. I've added the suggestions to my Evernote list of tools I want to try in the future.

Summer will give me opportunities to play a bit with technology, especially going to read some of the older posts on The Digital Researcher blog.

Inviting more ease into your week

By | | | XFacebookLinkedInEmail

ease

I enjoyed the conversation with Natalie Houston immensely, which aired this past week in episode #034. She reminded us all to find ways to have more of a sense of ease in our lives.

She said:

Productivity, to me, is not about doing more things faster. It is about doing the things that are most important to me and creating the kind of life I want to have…

Here are a couple of opportunities for having more ease in my life that I discovered this week:

Set a timer to better enjoy breaks and conversations with students

Since the conversation with Natalie, I've been setting timers like crazy. While I had used this approach previously, she shed new light on the practice and inspired me to take more regular advantage of the focus on the moment.

I found a Tweet where Natalie coached a colleague who wanted motivation and concentration to:

I met with a few students this week and really felt more connected with the conversations in those times when I remembered to set a timer. There was one time when I neglected to do so, and I both wound up completely throwing off my schedule for the latter part of the afternoon, and wasn't as engaged as in the other “timed” conversations.

Natalie also offers these tips about using a timer while grading.

Install a custom style in Zotero, which enables you to create an annotated bibliography

This one is really more for my doctoral students than for me, though I treasure the opportunities to bring more ease into other people's lives. As a part of their program, they're asked a number of times to develop an annotated bibliography.

We also strongly encourage the use of the citations manager, Zotero. The annotated bibliography assignment hasn't aligned well with using the automated features in Zotero, since until now, I thought it could only “spit out” a list of sources, not something that included annotations.

Thanks to Emory Libraries and Information Technology, I found out I was wrong. Turns out, you can create a custom style in Zotero, or download one from their repository and leave the heavy lifting to others.

[reminder]How are you inviting more ease into your life this week?[/reminder]

Assessing and tracking blogs

By | | | XFacebookLinkedInEmail

I made the move a few years ago in my hybrid classes to stay mostly away from discussion forum posts. They tended to generate more of a transactional perception of the assignments.

Blogs seem to inspire people to do their best work, given the public nature of the assignments.

assessing and tracking blogs

Plus, if a person takes their blog seriously, they can be on their way to building a strong personal brand. Their data isn't locked behind an LMS, but is theirs to do with it what they want, after the class has ended.

Audrey Watters has been an advocate for providing students their their own online presence, one that isn't trapped in an LMS. On episode 18, she describes the University of Mary Washington's “A domain of one's own” initiative, in which they provide all incoming students their own website that gets transferred over to their ownership upon graduation.

Anyone who has made the switch from forums to blogs knows that it isn't anywhere near as efficient for the professor. You need to subscribe to all the students' blogs, manually, and commenting is nowhere near as easy as on an LMS.

The benefits far outweigh the challenges, though, so I continue to make use of blogging as an assignment in many of my courses.

Assessing blogs

The vast majority of the assignments in my courses are evaluated using a rubric. I've seen my fair share of atrocious blog rubrics and am continually striving to make mine better.

The best approach to assessment of blogs seems to be that they should be geared toward the learning outcomes for the course (as in that a generic blog rubric could not be used to adequately assess a blog).

This is a blog rubric that I've created for a doctoral course on leadership and technology in which the students develop a personal knowledge management (PKM) and use their blog as a means of sharing their work with others.

A new approach to tracking blogs

I use Feedly to subscribe to the cohort's blogs and then Newsify to actually read them. However, since the rubric calls for different types of posts each week, it is difficult for me to quantify them at the end of the term for grading purposes.

I wind up having to go back and manually count the students' posts to see if they blogged each week. It is also difficult because they don't always categorize their posts, according to the rubric, so I make my best effort to guess which type of post they were intending, a method that no doubt has its weaknesses.

Richard Byrne inspires

rbtweet

Relief came when I saw a Tweet from Richard Byrne about his approach to tracking blogs using a Google form.

I wish I would have implemented his approach over the summer and used it with my undergraduate students in the Fall. However, I didn't invest the time and wound up having both inefficiencies and needless student conflict without a more reliable means for tracking.

blogformsmMy Google form for tracking blog posts

I have modified Richard Byrne‘s approach slightly. Since my students are asked to write three posts per week, they have three places in which to paste their links.

Additionally, there is a place for them to indicate the type of post they are submitting. This should help steer the students toward the types of posts they're required to write each week, as well as making it easier for me to determine their intent.

I'm already excited about the time saving possibilities that this approach will provide.

It also seems like it will help reinforce the expectations for the students.

[reminder]Are you trying out any new approaches in your teaching this year to try to add some efficiencies into the grading process? [/reminder]