We had another intellectually buzzing one hour of online teaching ideas and engaging discussions this month. Our topic this time was assessment and feedback, which is quite a controversial topic nowadays because teachers are struggling to find the best way to test their students and they seem to be losing against Google Translate, spellchecks, copy+paste and group chats. But is this really a lost game? Read on to find out how we can alter our approach to assessment and feedback to catch up with the 21st century!
Jo Szoke - Redesigning a university theoretical exam
My idea was more of a case study of how me and my colleagues at Karoli Gaspar University of the Reformed Church had to redesign an exam of English teaching methodology to suit the different circumstances of this online teaching situation. We decided to accept and embrace the fact that remote exams also mean that everything can be googled and everything can be in front of students, so it's not worth fighting against it.
There have been many attempts from teachers at trying to find the best cheat-proof method or website, but it has to be said out loud that pushing our students into unmanageable situations of having to answer 50 questions in 15 minutes, requiring them to set up at least 2 cameras so that we can monitor them during the entire exam, and asking them to install keyboard and computer activity tracking software to prevent internet searches and texting with group mates make no sense whatsoever and also go against our own beliefs about teaching. We are not prison guards after all! We are teachers who'd like to see if our teaching has been successful or not.
Of course, I have to agree with those who say that sometimes you have to play tough because that's how the system is set up. Well, I'm not going to surprise you by saying that I think the whole system needs changing because it's not healthy for both sides and it also doesn't lead anywhere if all we care about is points and surveillance.
So, what we came up with is that on top of asking a simple question of factual information, we also ask them to prove, explain, justify and exemplify the concept that we would like to know more of. Instead of punishing and penalising our students for looking something up, we give them 1 point for being able to retrieve that piece of information, because efficient data retrieval is actually a 21st century digital skill. But if they cannot make us believe that they actually understand the idea by giving us personal examples and explanations, they're not getting more points.
So, it's still possible to fail this kind of exam (not that this is our goal) if somebody can only search for information. Of course, there are ways to trick this system as well. But we also have to understand that there will always be students whose main source of joy is tricking the system.
Adri Szlapak - Open book assessment
Online schooling changed the context of assessments. Suddenly we weren’t able to resort to exam conditions as easily as before. The new normal meant that the teaching community had to come to terms with the fact that students had help and there was very little anybody could do about it. Help came in different shapes and sizes. But is help a bad thing?
Adri realised she needs to embrace the help they are getting because she can’t fight it. So she started doing open-book assessments, which have had a great washback on her courses and teaching. Open book assessments are exactly what the name suggests. She lets her students use help, be it books, notebooks, Jamboards, Google or dictionaries. These assessments work much better because
they encourage engagement
finding the answers to questions becomes part of the learning process
cheating becomes a non-issue
Here’s how Adri runs open book assessments:
Turns lower-stakes assessments into challenges. By simply renaming a test and calling it a challenge, motivation rises.
Always shares her grading criteria with students before any assessment so that there are no surprises. Once learners know what they need to do and what they will be graded on, they become less anxious.
Sometimes makes the passing grade 100%. If you’re allowed to use help, then you're expected to get everything absolutely correct. At times she even allows students to take these tests as many times as they want to, but expects them to get everything correct eventually.
Allows learners to make their own cheat sheets. She lets her student bring a cheat sheet too because she finds that those who put effort into making them have spent valuable time preparing and it pays off, both for them and for the teacher.
Some guidelines for open book assessments:
By making the tasks as similar to real life as possible we can ensure that we are assessing the application of knowledge. For instance, instead of just asking students to define a word or translate it, ask them to use it in a sentence about themselves.
When setting a writing assessment online, make sure that the task is so specific or personalised that there is very little help students can get from the internet. Give specific instructions as to what you want to see in the writing so they aren't able to just copy-paste from a website.
Online quizzes can also be turned into an open-book assessment. I usually make the students take the test in a live lesson with a time limit. However, I then make the same test available for everyone so that they can take their time going through the same questions with whatever help they want to use.
Read more on the topic on Adri's blog: https://ealdiaries.com/2021/04/open-book-assessments/
Gergo Sipka - Vocabulary quiz revamped
Gergo's idea also stems from the acceptance of the different setup of online teaching and online assessment. He brought us a very specific task for carrying out vocabulary tests.
He creates pairs that work together but adds one outsider (the supervisor) to each group, who's responsible for checking the words the other two managed to use and the time.
The pairs need to write a number of sentences with the words they learned in the previous lesson(s).
After the first round, the roles are switched - the regular players become supervisors and vice versa in order to let everyone experience both sides. However, this time the players need to write situations with the same set of words, which is a higher level task. At the end of the two rounds, all participants give feedback to each other, so there's an opportunity for self-evaluation and for peer evaluation.
The bottomline in this activity is that students see and work with the words in different ways and on multiple occasions, which will lead to better retention and less stressful assessment.
Group discussions
We also had some really interesting discussions on various assessment-related topics, led by my amazingly talented moderators. They will recap the most important points now.
Room 1 - Using projects for evaluation (Dorka)
At first, we talked about what projects are or which activities can be considered as projects. We came to the conclusion that projects are varied, based on the length, number of participating students, topics, but they all require language production and creativity and they all should be linked to the students' interests. It is also a stage in learning in which the student can be responsible for their own learning process.
We agreed that when using a project for evaluation, the criteria (using language / vocab taught, group work, engagement, creative solution, etc.) should be transparent for everybody. There is room for personal evaluation which is based on the progress of an individual student, but - as state schools require in Hungary - there has to be a list of general criteria to evaluate. Projects should also support language production in the sense that students should feel more confident in speaking or writing. Projects can also be linked to "open book" evaluation, as students are encouraged to do research and they do not need to be synchronous.
The final conclusion was that projects are a highly personalised way of grading students for the active knowledge of the grammar/vocabulary that we have taught.
Room 2 - Things not to be submitted (Agi)
We can assess such aspects of learning (e.g. motivation, creativity, improvement compared to a previous stage, persistence, attitude) with alternative assessment that traditional methods cannot. It's important to have many grades (a requirement from the school and the parents) but not to create an increased workload for the teacher. The tasks that belong to this group should be evaluated with positive feedback only, which can compensate for worse grades received in the traditional assessment system. Such an approach can reduce stress and the urge to cheat.
What kind of tasks can we set then?
Give stage to things that students regularly do outside class (watching TV series, a cooking recipe, a makeup tip, a poem, DIY idea etc). Ask them to create a teaser for what they have seen or done, which we evaluate. If somebody likes their idea and watches or does the thing, they can also document that and get some reward for it.
Individual projects that connect to real life. Do something new every day (e.g. walk up the stairs, stand on one leg, meditate) and reflect on how it went.
Pair work. Create an interview with a classmate and present it. This can do good to group dynamics.
Peer evaluation. Ask students to vote on who the most helpful/creative/interesting etc. was.
Study buddy. Students that missed some classes can choose a study buddy who helps them catch up. If it works, they both get some reward.
Room 3 - Rote learning vs applied knowledge (Adri)
In this breakout room we discussed the differences between assessing real application of knowledge versus testing knowledge learned purely for the purpose of a test. We agreed that while a test is like a photo, assessment is more like a mirror that you can keep checking your appearance in. Every time you look into it, you'll get feedback and will be able to make adjustments.
We also discussed that assessment can be made more useful for both the students and the teacher if it allows the teacher to adapt the course to make it more appropriate. This way students will stay motivated to learn.
Last but not least we talked about the problems participants experience with translation tasks in online lessons. Unfortunately, this seems to be rather a headache for some Hungarian state exams as the full translations are available online if students Google them. A possible solution could be to change the format of the tasks so that they don't look like the exam task. This way students are less likely to find the answer keys. In live lessons they can be asked to work in a shared Google doc where the teacher can see everything that's happening too. For teacher-led translation tasks we thought translating collaboratively in speaking could work well to make sure students don't use Google Translate.
Room 4 - What do you know? vs. What don't you know? (Gabi)
Firstly, we discussed why we assess students at all. Our keywords were giving and getting feedback, alternative ways of formative evaluation and motivation.
Maintaining motivation is essential not only for students, but also for us, teachers. By creating funny and challenging edu games (especially escape rooms), we can ensure that students apply the knowledge they've acquired.
If we assign them to create similar games for each other on any platform they wish to use, they’ll revise what they learnt through a creative and entertaining activity. The links of these creations are then shared with the whole class and everyone who’s done a quality job is rewarded with a small 5.
Ági Enyedi's idea works fantastically well: in smaller groups, students should prepare tasks for the others and then share them. 75% of the ‘official’ end-of -unit tests will come from these materials and only 25% will be unknown, written by their teacher.
As for correcting the tasks your students submit, Anna Csíky's brilliant idea will save you a lot of time: try Loom and give detailed feedback to them in a video message.