• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!


Pilot Test Oxfam Novib

Page history last edited by Alexandra Pittman 13 years, 1 month ago

Piloting Most Significant Change with Partners in Oxfam Novib

Summary prepared by: Alexandra Pittman, Jeanette Kloosterman, and Esther Benning


The following summary of the pilot exchange process of the Most Significant Change (MSC) technique in Oxfam Novib came from discussions at the Measuring Gender Equality Initiative at the Dunford House in West Sussex, UK from May 19-20, 2011. The meeting, hosted by Gender at Work (G@W) and the Institute for Development Studies (IDS) was supported by Global Fund for Women (GFW), NORAD, and UNWOMEN. It brought together a dynamic set of practitioners working on gender equality and organizational learning in women’s organizations, university settings, and donor agencies. Staff members from diverse organizations were present in this specific discussion, including: Esther Benning (ON), Rex Fyles (G@W), Jeanette Kloosterman (ON), Hannah Sheppard (G@W), Michelle Odayan (Indiba-Africa Alliance), Ratna Sudarshan (Institute of Social Studies Trust-India), Alex Pittman (AWID), Erica Zwaan (CORDAID), Jenny Bell (Justice and Women), Geeta Misra (CREA), Katy Oswald (IDS), and Michelle Higelin (World YWCA).


In 2010, Oxfam Novib (ON) began a process to pilot test a model for exploring changes in the deep structure[1] of grantee organizations supported by their Aim 1 and 4 (sustainable livelihoods and political participation) work. The Most Significant Change (MSC) pilot was done in the context of the Gender Mainstreaming and Leadership trajectory (GMLT), an Oxfam Novib program to promote gender mainstreaming, especially with partners in Aim 1 and 4, because these partners appeared to be weak on applying a gender justice approach in their work. So MSC stories were collected, not so much in women’s rights or gender programs, but in agricultural, environmental or human rights (political participation) work, from partners that were working on improving their gender mainstreaming efforts. The objective was to support grantees’ greater reflection on different pathways to change, including unexpected outcomes and their exploration of internal organizational norms, which underlined program implementation.


This process was unique as it represented a cutting-edge effort on the part of Oxfam Novib to pilot test a model of assessment that focuses less on the traditional linear logical framework-type models (responding to critiques from women’s organizations) and more on testing dynamic approaches for exploring program outcomes and deepening organizational reflection and learning in the field of gender equality.


The Most Significant Change Implementation Process


In order to implement the MSC method, ON brought in representatives from Gender at Work (G@W) to help train a group of 20 partner organizations and 12 consultants from 12 different countries (read more about MSC here). The pilot started with a training workshop in the MSC technique. In that workshop both partners and the consultants who had been involved in the GMLT activities participated. The idea was to use consultants that had extensive gender analysis and leadership training to support partners with the implementation of MSC. In other cases, partners indicated that they had sufficient gender expertise in the organization, and so they did not need the support of the consultants. The participants were trained to implement the technique, which involved collecting stories of significant change from participants in programs. They then guided the organization through a story selection and learning process.


Take for example a program held in a village aimed at transforming power relations between men and women through a political leadership program. A story collector would ask participants, “In the past year, what was the most significant change in relations between men and women that you saw in this village?” The storyteller talks from her/his own experience. The story collector must write down the story, capturing the nuance by asking clarifying questions about why the participant felt his/her story is significant and how the change came about.


During the training workshop, participants started practicing the story collecting and selecting process. The selection and decision process fostered deep discussions amongst the participants, surfacing important questions about values and the types of changes being created, as well as illuminating different assumptions and pathways for how changes came about. Through the discussion process, the participants gained greater clarity about some of their underlying working assumptions, their connections to strategies and outcomes, and highlighted areas for necessary improvements or re-strategizing. The selection process also allowed for the selection of stories that best represented significant changes that had taken place.


In the implementation of the MSC technique, once the partner organization comes to agreement on the MSC stories they feel best represent the shifts the program made, it then sends them up to the national office and/or to the international level, where similar discussions and rankings take place. Not all stories are sent up to higher levels, only the most compelling. They are then used to complement the results of other M&E tools, as narratives for newsletters, funding reports, media campaigns, etc.


Challenges and Opportunities Faced in the Pilot


Oxfam Novib ran into a few obstacles with MSC implementation and buy-in at the institutional and partner levels. First, the significant budget cuts to Dutch civil society organizations (CSOs) under MFS-II in November 2011 affected Oxfam as well as other Dutch CSOs. In Oxfam Novib’s situation, this meant there were substantial reductions, and the gender equality work was hard hit, with gender equality budgets being reduced by 40%. This translated into Oxfam Novib having to make hard and difficult decisions on ending funding for a number of long-time partners that had made substantive shifts to bringing about positive changes in gender equality across the world. Given that the pilot testing of MSC started before partners were cut, a number of them did not continue to pilot test the method; only 12 partner organizations ended up implementing MSC.


Internally, economic declines have brought about a constraining of dedicated support for gender equality work and an increasing potential for gender and diversity work to be mainstreamed. There is a need to carefully address this trend, particularly given that mainstreaming gender has not necessarily been effective in bringing about the deeper transformations of power relations necessary for full realization of gender equality and rights—changes which have historically formed the core of Oxfam Novib’s commitment to gender justice and equality.


Early Learnings from the Pilot Process


In terms of the initial stages of the MSC pilot process with Oxfam Novib grantees, a number of strengths and challenges were identified by policy and evaluation advisors within the organization as well as other participants in the meeting.


Staff from Oxfam Novib reflected that the story collector (often consultants in this case) must have good interviewing skills. They also found that too much detail in stories could be confusing. This finding was somewhat in contrast to the assertion that MSC requires little to no training.


Initial informal reports suggest that stories being relayed back from grantees have deepened program staff’s knowledge about the nuanced and rich changes that were occurring in a way that was previously unknown to them. It also provided partner staff in different contexts the opportunity to think more critically about change processes and trajectories and discuss implicit pathways of change.


However, staff involved in the pilot also identified some unexpected challenges. First, not all of the participants had enough training to effectively apply the technique—so some stories of change are more descriptive than others, raising questions about the extent to which the organization could trust the validity and comprehensiveness of some of the stories submitted. Using a story collector as an intermediary to take notes on stories may create greater room for reporting errors or omissions of critical information (even though the story collector’s role is meant to be clarifying).

This is because the technique creates an additional interpretive barrier between the actual voices of change. While the MSC technique has mechanisms in place for the story teller to review their stories with story collectors as one means to work against bias, the very act of using story collectors may introduce a “safe story” bias as participants may not feel comfortable sharing stories that are deeply personal or that touch on taboo subjects with people they are not familiar with.


Moreover, obtaining stories that touch on sensitive subjects requires a deep level of trust that cannot be fostered overnight. Some potential suggestions for strategizing around some of these challenges included removing some of the intermediary barriers to storytelling and recording stories by tape recording and then getting direct transcripts or by videotaping the stories. 


Another limitation revolved around the potential conflicting effect of the competitive ranking of stories on the aim to stimulate reflection on deep culture. In some ways the ranking process might even intensify implicit norms related to decision-making, further entrenching informal power relations within an organization.


A variety of possible adaptations to MSC were also suggested in the context of the group discussion in Sussex, including:

  • ensuring a more holistic analysis of stories by doing a systematic analysis of the thematic trends in all of the stories for reporting and evaluation processes, 
  • aiming to understand cycles of change and feedback loops involved in transforming power relations within an organization and/or programmatically, and
  • learning about how change is effectively catalyzed through donor-grantee relationships (value added).


Oxfam Novib will be holding a follow-up meeting with partners in September 2011 to gather deeper insights about the pilot testing phase and to discuss how to best adapt the model for potentially scale up across the organization. The next round of feedback will be directly from partners who have been using the tool and will be targeted towards reflecting on its applicability for the exploration of deep culture and unexpected program outcomes. Updates from the outcomes of that meeting will be shared on the AWID wiki.


In the meantime, we would like to start a broader discussion with interested women’s rights organizations, interested international NGOs, international development evaluators, etc by posing the following questions:


  •  How has MSC been used in your setting, and what have been the benefits and limitations?
  •  How can we best address the interpretive barrier between story collectors and the actual voices of change?
  •  How can we address power relations between story collectors and the actual voices of change?
  •  How can we address the power relations (informal and formal) during the ranking and selection of stories?


Please share your feedback and personal experiences with the MSC tool in the comments section below or add your own experience to the wiki if it is a larger case study. Also feel free to contact Jeannette Kloosterman and Esther Benning at Oxfam Novib for more information or Alex Pittman for additions to the wiki.






[1]Using Rao and Kelleher’s (2000) framework, Batliwala (2008:22) describes deep structure as: “…the hidden sites of power and influence, the implicit culture, the informal values and systems of reward and recognition, all of which have enormous impact on how people and the organization actually function, and who has real power in the system.”


Comments (0)

You don't have permission to comment on this page.