Evaluation
Evaluation involves bringing together all the information in a project, or a service, or an experience - and deciding what you think about the nature of this particular thing, or its quality, or value. The sort of evaluations with which I'm involved are usually looking at something that has changed: projects that are new, a change to a service, a different way of doing something.
The information collected is data. The things you can count are generally called quantitative data; the things that are about your senses and feelings and experiences are generally called qualitative data. To evaluate something you need both types of data - and from that you can reach some conclusions (or results, or findings).
The photo above is a very simplistic way of saying that part of evaluation involves investigating the box and its contents and beginning to draw some conclusions. What's in the box? is it dried fruit? is it biscuits? how many? are they the original biscuits? if the biscuits have been eaten, when was the dried fruit added? what sort of dried fruit? have you eaten anything from the box? was it tasty? who gave you the box, and why? how do you use the box now? And so - what are your conclusions about this box and its contents and the changes that have happened?
In the boxes below are the two main types of qualitative methods or approaches that I use in evaluation - mostly this is evaluation of projects, or programmes of work, or services.
The information collected is data. The things you can count are generally called quantitative data; the things that are about your senses and feelings and experiences are generally called qualitative data. To evaluate something you need both types of data - and from that you can reach some conclusions (or results, or findings).
The photo above is a very simplistic way of saying that part of evaluation involves investigating the box and its contents and beginning to draw some conclusions. What's in the box? is it dried fruit? is it biscuits? how many? are they the original biscuits? if the biscuits have been eaten, when was the dried fruit added? what sort of dried fruit? have you eaten anything from the box? was it tasty? who gave you the box, and why? how do you use the box now? And so - what are your conclusions about this box and its contents and the changes that have happened?
In the boxes below are the two main types of qualitative methods or approaches that I use in evaluation - mostly this is evaluation of projects, or programmes of work, or services.
Most Significant Change (MSC)
Most Significant Change (MSC) is a qualitative form of research and evaluation, where what's gathered to begin with are stories of change. You're asking someone - what was it like before? what's it like now? which change stands out the most for you?
'Before' just means before something started - like coming to a course, or going to an event, or moving to a new place. 'Now' is - just as it says! - how things are for you now, after this change. Sometimes lots of things have changed: you might know more, or feel more confident, or enjoy something better, or ... and one of those changes might be top of your list. 'Most significant' doesn't mean giving something marks out of 10. It just means saying which change matters most to you - and why. Once you've got a few (anonymised) stories of change, the next stage is to have a discussion with a few other people who aren't the storytellers. You might collect stories of change from your Advisory Board, and give them to a group of older volunteers to discuss. Or you might take stories from people who receive a service and ask members of staff to discuss these. The aim is to find 1 or 2 stories that everyone agrees stand out the most, and why. The reasons why that story is chosen are as important as the story itself. MSC is participative, collaborative, and democratic - and a great way especially to evaluate people-orientated services or where complex changes might make it difficult to pinpoint outcomes. |
Realistic Evaluation
Realistic evaluation is a way of looking across three linked aspects of change, to find out:
Sometimes realistic evaluation is called realist evaluation. What I've put here is a very simplistic explanation. Many researchers have explored this approach and written about it in detail, starting with the two people who first developed Realistic Evaluation in 1997, Ray Pawson and Nick Tilley. |
|
SERA - Story-based Evaluation and Research Alliance
SERA was launched in January 2021. It was developed by a group of like-minded researchers, writers and evaluators who work across the arts, health, and social sectors. We have a shared passion for Most Significant Change - and we're interested in other ways of using storytelling (or narrative) in research and evaluation work. We decided to form SERA so we could share our skills and knowledge about, and enthusiasm for, MCS. We hope to create a 'Community of Learning', and help others implement the MSC approach. SERA - https://www.seralliance.org/ |
|
|
|
