Most significant change technique

The Most Significant Change Technique (MSC) is a monitoring and evaluation (M&E) method used for the monitoring and evaluating of complex development interventions. It was developed by Rick Davies as part of his PhD field work with the Christian Commission for Development in Bangladesh (CCDB) in 1994. CCDB, a Bangladeshi NGO, subsequently continued and expanded the use of MSC to monitor the impact of its participatory rural development projects for the rest of the decade.

Following publication of the CCDB experience on the internet in 1996, MSC was progressively adopted for use by other NGOs in Africa, Asia, Latin America and Australasia. These experiences were then documented in the 2005 MSC Guide, co-authored by Davies and Dart, which remains the most widely cited reference on how to use MSC. Jess Dart, the co-author of the Guide, carried out the first use of MSC in Australia as part of her PhD research. Her company Clear Horizon has since been the main provider of MSC training in Australia.

MSC represents a shift away from more conventional quantitative and expert driven evaluation methods toward a more qualitative and participant driven approach, focusing on the human impact of interventions. In summary: the MSC process typically involves the collection of qualitative information from the intended beneficiaries of an intervention, in the form of a description of a change each considers as the most significant within a given period of time; and then an explanation of why they see that change as most significant. This collection process is followed by the use of one or more selection panels, where those participants (or other stakeholders) review the set of collected MSC stories and identify the one which they agree (and explain) is most significant of all, as seen from their perspective.

Steps involved in the Most Significant Change process

There are 10 steps involved in the Most Significant Change process

1. Starting and raising interest
2. Defining the domains of change
3. Defining the reporting period
4. Collecting significant change stories
5. Selecting the most significant of these stories
6. Feeding back the results of the selection process
7. Verification of stories
8. Quantification
9. Secondary analysis and meta-monitoring
10. Revising the system

Steps 4 to 6 are the essential kernel of the process. Steps 1 to 4 are necessary preparatory steps. Steps 7 to 10 are optional follow up steps

Benefits and limitations of the MSC technique

Benefits

MSC enables the identification of the kinds of changes people value, why they value those changes and the extent to which they share those views with others. MSC stories, and discussions of these, can also shed light on the causal processes that generated the changes, and implications for actions that could be taken in response to those changes.

The focus is on learning rather than accountability. Managers, field staff and evaluation team members can be asked to reflect on and openly question the intervention programme and their interactions with the community in which the intervention takes place. Its use can also support other M&E processes, for example by identifying the aspects of the intervention which could benefit from more quantitative investigations. In addition, the process can give the evaluators a greater sensitivity to the perspectives of beneficiaries, which it could be argued, may be conducive to more successful outcomes.

Limitations

1. MSC is a participatory process that emphasizes dialogue. This can be time consuming. Both participants and facilitators of an MSC exercise may have other pressing demands on their time
2. The elicitation of individual MSC stories and the facilitation of discussions about multiple MSC stories both require some skill, in interviewing and small group facilitation
3. Participants will often have concerns and reservations about the meaning of "significance", the acceptability of subjectivity and the appearance of competition, which need to be addressed
4. The process as described by Davies and Dart in 2005 may be incompletely implemented. Changes may be adequately described but explanations of their significance may be cursory or absent (Step 4). MSC stories may be collected then not followed by a selection process (Step 5). Feedback of the choices made between a set of MSC stories, to the original providers of those stories, may be forgotten (Step 6)

These and other challenges are discussed in Chapter 3: Troubleshooting in the 2005 MSC Guide

Usage

MSC is now widely used by development aid agencies, especially NGOs. The original MSC Guide has since been translated into 13 languages (Arabic, Bangla, French, Hindi, Bahasa Indonesian, Japanese, Malayalam, Russian, Sinhala, Tamil, Spanish and Urdu), typically by organisations working within those language groups.

Since 2000 there has been an active and global "community of practice" that shares experiences with the use of MSC in different settings. The email list used by this group is currently hosted by Google Groups. [7]

More recently, an online bibliography has accumulated information on the global use of MSC [8]. References are now available on more than 290 papers and reports that have been published on the use of MSC. These cover both developed and developing countries and relate to interventions in a wide range of sectors – including health, education, agriculture, infrastructure, governance, and community development.