Blog post
Evaluating my evaluations
Evaluations. Crucial to university student outreach. A minefield? Sometimes. Including one time where I had to ask a young person, ‘what exactly is rizz? Is this good, bad?!’ – it’s good – it’s short for style, charisma, attractiveness – just in case you were clueless like I was. In my field of work, university outreach, we work with young people in a variety of ways to encourage them to explore their future options, whether that’s university or not.
When it comes to this work, we need to know what’s working and what isn’t. Whether you do evaluations through Mentimeter, Google forms or good old-fashioned paper, you know that you are almost 100 per cent likely to receive feedback on a variety of subjects (many out of your control). Such as the food, the phone signal and if the room had windows. And for the most part, you receive fairly average feedback with most students attempting to write something positive, for the sake of your ego.
This doesn’t always help in the way we’d like it to, however. Depending on the type of evaluation you are doing – narrative, empirical enquiry or causality – you may decide to use a variety of evaluation techniques to gain insight from your participants (OfS, 2023). If you are unfamiliar with these three types of evaluation: narrative provides a story and motivation behind its selection of activities; empirical enquiry is about collecting and reporting data to provide proof if an intervention improves outcomes, although it does not establish any direct casual effect; and causality is about providing evidence that there is a casual effect from your intervention. What I’ve struggled with is gaining narrative evaluation that includes comments beyond ‘this is good’ and ‘I liked it’ or ‘more snacks’. And from discussion with peers, many of us struggle to get more than a three-word sentence from our participants, particularly the teenage ones, which doesn’t really help us with our future planning to help us decide what is working or not.
I recently tried a new way of conducting evaluations. I ditched the Google form with scales of good to bad or ‘would you recommend this session’, and I decided to embrace a change. At the end of my session, ‘The role of creativity and play in any subject’, I rejected my usual cracking out of the QR code and Google form. Instead, I handed my participants a piece of paper with a box on it and the instructions, ‘In the box below, draw or mind map your reflection on today’s session’. Under the box were a few lines and the instructions, ‘On the lines, please write a few notes describing your drawing or mind map.’
I won’t lie, I was somewhat petrified. Would anyone draw anything? Would I take home a pile of paper covered in inappropriate drawings?! But I needn’t have worried (this time). The young people rose to the challenge of putting their feelings about the session into drawings. I had a variety of drawings with smiley faces, stars, representations of the session’s activities and flowers. Not only did I enjoy their drawings, I had much more in-depth commentary about how they felt about the session with comments such as ‘peaceful, fun, interesting, helpful’, ‘enjoyable session, all linked in the end’, ‘cool, happiness, exciting, rich’, and ‘happy face, fun, useful, engaging’.
Would I encourage you to try this technique? Absolutely. As children, most of us draw to communicate, and we enjoy it. However, the majority of us just fall out of this habit when we get used to the idea that we should be ‘good’ at drawing realistically or visualising what we imagine drawing. Unfortunately, we are aware that generation alpha is struggling more with anxiety and fears for the future, and drawing has been shown to reduce stress and anxiety, which makes it an excellent task to end a session with (O’Neill, 2024).
‘Generation alpha is struggling more with anxiety and fears for the future, and drawing has been shown to reduce stress and anxiety, which makes it an excellent task to end a session with’
Not only is the actual drawing enjoyable for both you and your participants but it also enables quite easy discussion afterwards, so I urge you to not leave too little time at the end of your day to do it. The more drawing evaluations I have done, the more important I have learned that the discussion post drawing gives your participants the moment to reflect, again, on the activity. Not all participants have engaged warmly with the drawing reflection, but I would say that the majority enjoy having an alternative way to express their feelings other than through a tick box or a sliding scale.
However, do I manage to avoid the dreaded food comments? No, of course not; on one piece of paper, written under their drawing, there was, ‘It was a nice experience. Food, there was food.’ So, I guess I won’t be winning on that one.
References
Office for Students [OfS]. (2023). Standards of evidence and evaluation self-assessment tool. https://www.officeforstudents.org.uk/for-providers/equality-of-opportunity/evaluation/standards-of-evidence-and-evaluation-self-assessment-tool/
O’Neill. (2024, December 6). Gen Alpha shaped by pandemic, cost of living and war, poll finds. BBC News. https://www.bbc.co.uk/news/articles/c6273jnl5xlo
Scott, C. (2021) Arts for health. Emerald Publishing Limited.