Most Significant Change (MSC): How to measure behavioural change?
'The literacy rate has increased with 20 % in five years!' 'Fifty percent of the young people, who participated in reading and writing classes have acquired a job within a year after their studies!' 'The number of parents, who take on volunteer responsibilities in the Youth Centre have increased with 30 parents in two years!'
These are just three examples of project indicators, which can be formulated according to the Logical Framework (LF) principles for a youth development project in a big slum area in Kampala City, Uganda. However, most statistical indicators are difficult to measure. And what do statistics say about structural change?
Mostly change can not be liniarly and logically explained......
Most structural change starts with behavioural change. Many times I hear people express after an inspirational workshop about social media; 'Tomorrow I am going to open an account at Twitter!', 'I will update my profile at Linkedin!' 'I will more actively participate in discussion groups! '
However, one month later people who expressed these statements, have not changed their habits. They are still not on Facebook, not on Twitter and hardly participate in Linkedin discussions.
Then suddenly, members of the group of people who participated in the social media training, loose their job. The jobless situation creates a new perspective. What do I want? How does my network look like? This is the moments habits and behaviour start to change. Linked in profiles are updated, a Twitter account is activated and regularly used and stories are shared at Facebook.
Was it the social media training that changed people? Was it the jobless situation that confronted people with a new situation? Was it the network around these people that made them change? And what was the attribution of the social media training event to this change? And what would have happened if people had not lost their jobs?
As you can read, the reality of change and development can not be explained logically and liniarly. When people attend a training, it does not automatically mean that they will start to change their habits and attitude. Change mostly comes gradually and is fostered by many other influences than training or projects. External influences (economic or a political crises, technological changes) or drastic personal changes (loss of a job, dramatic event in the family, health related problems) can be an accelerator of change.
Most Significant Change (MSC)
The Most Significant Change (MSC) provides an answer to the aforementioned dilemma. The MSC-technique is a form of participatory monitoring and evaluation. It involves many stakeholders. The approach provides data on impact and outcomes that can be used to help assess the performance of a programma as a whole. The technique is an excellent complementary tool to 'Outcome Mapping (OM)'. Outcome Mapping (OM) is a planning tool as part of the Project Cycle Management Approach, where planning and monitoring of behavioural change is one of the basic principles. The MSC-technique can also be complementary to the Logframe Approach, especially when there are no pre-defined indicators and measurement of behavioural change is part of the process. Read more - The OM versus the LF approach.
How does MSC work? ***
MSC has a step based approach where personal stories from key stakeholders are collected, analyzed and verified. One of the key steps is the phase of defining the 'Domain of Change' and collecting the stories. This is the phase where the MSC-question is asked for a specific change in a domain.
For example:
(1) Looking back over the last month, (2) what do you think was, (3) the most significant (4) change (5) in the quality of people's lives (6) in this community?
Three to five domains are a manageable number for conducting a MSC-evaluation. Mostly stories are collected on a written half A-4 page of paper. Whereafter the stories are collected, analyzed, and verified. An important part is the process of drawing conclusions, lessons and recommendations. Involvement of the right stakeholders determines the quality of the outcome of the evaluation.
Video as a tool for Most Significant Change (MSC)
Video is also an excellent tool to collect 'Most Significant Stories'. One-minute video messages or Testimonials (see an example of Simon Koolwijk sharing an inspirational story about an IAF-Benelux Conference) are ways on how 'stories of change' can be collected.
Training in Project Cycle Management
The MSC-technique was one of the topics discussed during the training Project Cycle Management (PCM) which was held at the Hendrik Kraemer Institute (HKI) on the 4th July, 2011 and conducted by Simon Koolwijk. A Development Professional from ICCOenKerkinActie was prepared using and getting acquainted with the principles of PCM and the Logical Framework Approach. During the discussions reference was made to the Outcome Mapping (OM) and Most Significant Change (MSC) technique.
*** Reference is made to the handbook: The 'Most Significant Change' Technique - authors: Rick Davies and Jess Dart
I found this a really helpful blog post. I think the problem might be that in these lean times, proving bang for buck is increasingly important.
ReplyDeleteIm finding that I am increasingly hired to develop results orientated, measurable strategies with a clear link between inputs and outcomes. It is what the donors want, so that is the direction that development agencies are going. Unless there is an easy way to link outcome mapping and most significant change into an evaluation framework which can indicate some level of attribution of causality - I fear Im going to continue to struggle to get my clients to accept it.
Thomas Scalway
Thomas, thanks for your contribution. Yes, it is very recognizable the wish and pressure from donors to report back short based results. It is very helpful when you can work with a donor that is open minded and appreciates 'small behavioural changes' of people which eventually leads to sustainable change. For example, when somebody is already applying more participatory approaches in their community work, it is not a guarantee that the community will change in one month. But at least the NGO or social worker has made a big step and behavioural change in getting the community more involved in their own decision making processes. It is just a small example. Thanks, Simon
ReplyDelete