Friday, June 26, 2015

Outcome harvesting – an excellent tool for evaluating projects in complexity!

‘Outcome harvesting is an excellent tool for evaluating projects, that operate in a complex environment and that do not have a pre-defined strategy!’   This was one of the comments from one of the participants, who participated in a workshop conducted by Ricardo Wilson Grau, one of the founding fathers of the Outcome Harvesting method.  The workshop was hosted on the 23rd June, 2015 in The Hague by the Nedworc Association and OXFAM Novib,   participants were Nedworc consultants and MEL-officers of OXFAM Novib.

Outcome harvesting is an evaluation approach, involving all stakeholders in a project,  in a program context where there are no clear relations between cause and effect.  Especially projects that involve lobby and advocacy,  fast changing contexts and unpredictable situations can benefit from the Outcome harvesting approach. In 2013, UNDP selected Outcome Harvesting as one of the 11 major Monitoring and Evaluation innovations. There is a toolkit guide and several case studies on how Outcome Harvesting is applied. Read more at:  http://betterevaluation.org/plan/approach/outcome_harvesting
Ricardo Wilson - Grau

The method has been developed by Ricardo Wilson Grau and his colleagues. They were inspired by the Outcome Mapping and the Utilization-Focused Evaluation.   Outcomes are defined as  change in behavior, relationships, actions & activities and  policies & practices of a  social actor (individual, groups & communities, institutions, organizations). The Outcome Harvesting method collects evidence on what has changed (who, what, when, where and how). then, working backwards, determines whether and how an intervention has contributed to these changes.  The evaluation focuses on effectiveness, rather than on efficiency, which is done in projects with casual relationships with activities, results and goals. The approach is composed of 6 steps;

Step 1, Design the harvest;  determines the set-up of the evaluation, the data collection methods and the evaluation questions that will be asked;
Step 2,  Focuses on review of the documentation and evidence collected.  In this phase the most important Outcome is defined and the most crucial activity or event that contributed to this outcome is determined.
In step 3, Engage with informants more detailed questions are asked from the stakeholders in the project (see in the diagram above)  in knowing more details about the outcome and the contribution.  This can be done in a  face-2-face workshop setting with  the help of a facilitator having all the stakeholders involved. However, it can also be done by e-mail posing the questions to each of the stakeholders. This is also the phase where significance will be given to the outcome by the informants.
Step 4; Substantiate.  This is the phase were external stakeholders, who have not been involved in the project but who are authorities in domains of the project, are consulted. They comment and give input on the data and evidence, that has been collected in step 2 and step 3.
Step 5, Analyse, interpret and step 6, support use of findings complete the Outcome Harvesting Cycle.  This is phase where lessons, conclusions and recommendations are drawn. This is where the learning amongst the stakeholders take place and where recommendations for improvement can be formulated.

During the workshop conducted by Ricardo Wilson Grau, the group worked on a case study of a funding agency, promoting the advancement of women’s rights.  We noticed that often specific information (What,  when, where and how?) is lacking and you need to ‘dig deeper’ to complete the information.
Outcome Harvesting is characterized by the following strengths:
§ Corrects the common failure to search for unintended results.
§ Has verifiable harvested outcomes.
§ Uses a logical, accessible approach that makes it easy to engage informants.
§ Employs various means to collect data: face-to-face interviews or workshops, communication across distances (surveys, telephone, or email), and written documentation.
§ Ties the level of detail provided in the descriptions directly to the questions defined at the outset of the process; these descriptions may be as brief as a single sentence or as detailed as page or more of text, and may or may not include explanations of other variables.
Because of its nature, Outcome Harvesting also has certain limitations and challenges:
§ Skill and time are required to identify and formulate high-quality outcome descriptions.
From: Outcome Harvesting, Ricardo Wilson-Grau and Heather Britt (2012, revised 2013) link http://www.outcomemapping.ca/resource/outcome-harvesting

Outcome Harvesting in practice – the GPPAC case
In the second half of the workshop, Paul Kosterink working with The Global Partnership for the Prevention of Armed Conflict (GPPAC), explained on how Outcome Harvesting is applied for projects in their organisation.  They got assistance and advice from Ricardo Wilson Grau in building their Outcome Harvesting design.  GPPAC  is a member-led network of civil society organisations (CSOs) active in the field of conflict prevention and peacebuilding across the world, http://www.gppac.net/nl .  GPPAC is active in a number of coalitions and networks in preventing armed conflict. Most of their work is focused on lobbying and advocacy through the networks.  ‘The context in which we work is very complex’ explains Paul Kosterink.
Paul Kosterink

When we ask our partners to report, they have difficulty in specifying and writing down their results. Therefore, we ask our partners concrete questions on what changed and on what they achieved,  explains Paul Kosterink.    In  the environment we work, it is difficult to reach big results within a short time. Therefore,  we ask our partners to come up with simple and low-key outcomes, which can signify that things are changing.  For example a letter from a Ministry of Defense giving compliments for our work specifying on how we prevent conflict, can already be a significant Outcome, says Paul Kosterink.
Lessons learned
Participants expressed that the workshop was useful and helpful.  Some of the participants quoted;

·      If we would have linked Outcome Harvesting to our reporting system, it would have produced much better reports;
·         The method is appealing. I am motivated to do more study on how the method works, before I start applying it;
·         I will propose the Outcome Harvesting (OH) method as a tool for a developmental  evaluation for a proposal, that I am developing;
·         The Outcome Harvesting is an excellent method for projects operating in a complex environment, that do not have a pre-defined strategy;
·         Outcome Harvesting will save you a lot of resources in time, people and money. The evaluator does not need to collect all the outcomes, but can start working from the evidence that is collected in the reports;
·         Outcome Harvesting enables to ask more detailed and interesting questions for doing an evaluation.
·         Outcome Harvesting is not easy, but it is very interesting!


Simon Koolwijk and Erica Wortel
June, 2015

Monday, May 11, 2015

Participatory video stories vizualizing the progress on education, Ethiopia




This video shows on how stories are visualized through participatory video stories in one week. From story board development to filming, to editing and then finally presenting and discussing the videos at a community dialogue meeting participants from social accountability partners learn to apply participatory video for dialogue with the community in discussing the progress on educational service delivery in Shashemene, Ethiopia. The training took place from 23 – 27 March, 2015 in Hawassa and Shashemene, Ethiopia. The participants inteviewed and involved 6 stakeholder groups in the discussion on the progress of educational services in Shashemene, Ethiopia. Both user groups, students, and service providers, teachers were involved in this participatory process. As experiment, also control groups from both students and teachers were involved in the PV process. The control groups came from schools, who did not participate in the social accountability process. The training was implemented by a team of participatory video trainers coming from Ethiopia and The Netherlands. Between April and July, 2015 the participants will apply the participatory video in their own workplace and social accountability projects involving their communties. The participatory video training and program is part of the Ethiopian Social Accountability Program Phase 2.

Advanced Training Participatory Video, Ethiopia, March 2015

From 18th – 20th March, 2015 twenty participatory video facilitators and practitioners attended a three day advanced training in participatory video. The twenty pv facilitators are representing 10 social accountability implementing partners (SAIPs), that are involved in implementing social accountability processes in the sectors of health, education, agriculture, water & sanitation and rural roads in the whole country of Ethiopia. The pv facilitators conducted pv interventions between October 2014 and February, 2015. This training gave them the opportunity to share their experiences and update their knowledge on participatory video, to learn to work with advanced software in video editing and to learn how to use the participatory videos for monitoring and evaluation and to discuss the opportunities and challenges on how to sustain participatory video in their organisation. Between May and July 2015, these SAIPs will continue to implement new participatory video interventions and are planning to participate in a new PV Oscar Reward Competition for the Ethiopian Social Accountability Program Phase 2. Watch video; 

Friday, March 13, 2015

Seven critical factors for a successful Participatory Strategic Planning (PSP)

In early 2015 I conducted a Participatory Strategic Planning for a consultancy business. For this I applied the Participatory Strategic Planning (PSP)  approach, which has been developed by the Institute of Cultural Affairs
The Strategic Planning consists of six key steps,
0. Intake formulation of the focus question
1.  Formulation / Review  Mission, values/ Trend analysis and environmental scan;
2.  Practical Vision;
3.  Underlying contradictions;
4.  Strategic Directions;
5.  Operational planning.


The method

Step 0: Intake and formulating the focus question
Before step 1 takes place in the strategic planning process, an essential step is necessary, to move forward.  This is the intake and the formulation of the focus question.
The ownership of the organizational change starts at the intake. Asking the right questions and getting the right people at the table are crucial conditions for enabling a successful strategic planning.  The focus question is the starting point of the  participatory strategic planning, through which the progress of the planning is monitored. This question is a perfect guide in monitoring the development of the client’s and group’s ownership during the planning process.


Step 1: Values, Identity, Mission, Trendanalysis and Environmental scan
Shared values ​​and identity build the foundation, why people commit themselves to a group or organization. Increasingly, identity and values are the pillars for an organizational strategic plan.  They are part of the mission statement, that explicity answers the question ‘Why are we as organisation on earth?’
The trendanalysis is a tool that analyzes the development  taking place in the context, which are influencing the organization. During the environmental scan opportunities and threats are mapped in a systematic manner. Both tools will help to answer the question if the existing organizational mission is still relevant or need to be modified.

Step 2: Practical Vision
This phase builds on the dreams and the positive energy of the group. In an associative way, the participants create a new perspective of the future of the organization or group. At the end, participants have defined a vision for a defined period of time.

Step 3: Underlying obstacles
Before participants reflect on the  strategic priorities for the future, they look at the underlying obstacles that hinder them in achieving their future goals. During this session, a self-reflective process is facilitated in researching the current situation why the organization is not realizing their future dreams. This phase helps to take a step back and observe from a distance and have a fresh watch at the current reality.  This is a crucial phase were the transformation is taking place.  Participants can not blame others, but are guided in a thorough process of self reflection.

Step 4: Strategic Directions
Once the blocks have been mapped, the group starts with the formulation of the strategic priorities. This phase helps the group to formulate a new strategic focus that will deal with the opportunities and the threats.  The challenge is to provide a new perspective on the current situation. In this session participants will be challenged to develop new products & services, identify new markets and new modes of working. 


Step 5: Operational planning
Based on the selected strategies, the group defines a one-year plan including milestones and a three months operational plan. During this phase tasks and responsibilities will be divided. Commitments will be made on the implementation and monitoring the progress of this plan.  It is crucial in this stage that the conditions and available resources in time and money are communicated to enable the implementation of the operational plan.

Critical factors for Success
There are 7 critical factors, which determine if a Participatory Strategic Planning will become a success.

1. The Intake, ensuring commitment from the leadership
The intake is a crucial phase in the PSP. From the beginning to the end the senior management has to take ownership of the process. During the whole process of planning up to implementation the senior management has to lead and to monitor the development of the focus question and the implementation of the plan. Will the key question be answered?  If the leadership does not take their full responsibility the process is doomed to fail.

2. Quality of participants
The quality of participants will determine the outcome and successful implementation of the plan. Not only senior management should be involved, but also staff from all levels in the organization. Diversity ,  knowledge and views from different perspectives to bring in as much viewpoints as possible. An adequate selection of participants is needed for a successful outcome of the participatory strategic planning.

3. Quality of facilitation
The quality and experience of the facilitator will be of crucial importance in helping the group to explore new territory. The facilitator not only needs to know about attractive and innovative facilitation methods, but also is required to know how to deal with group dynamics,  emotions of fear and anger and moments of unexpected resistance. Resistance and conflict are valuable moments for facilitating change.

4. Conditions for implementing the strategic plan should be clear
The conditions for implementing the change and the strategic plan should be clear from the beginning. Staff from the organization should know from the beginning how much time, resources and support from the senior management is available for implementing the strategic plan. The commitment and seriousness from the management should be clear and therefore, they need to communicate these conditions clearly during the participatory strategic planning event.  Especially time for doing for example training, skills development or innovation should be allocated by the senior management. If staff do not get the extra time and the resources to implement changes, a successful implementation is doomed to fail.

5. Follow-up
Plans get adjusted all the time.  Most strategic plans get adjusted during the implementation due to unforeseen changes in the contextual environment, changes in the organizaiton or change of staff.  Mostly a plan is implemented successful in case an organization is able to adjust the path during the journey. A good plan is more than half of the work, but a successful strategy can only be implemented if the organization is able to deal with the challenges it meets on the road. Therefore,  regular follow-up     (at least (bi) monthly) is needed ensure a successful implementation. 

6. Documentation is done by the organization
It is not the facilitator who is doing the documentation of the participatory strategic plan, but it is required that the organization is documenting and writing the strategic plan themselves. If real commitment comes from the senior management, it is up to them to write, document and disseminate the strategic plan. In most cases when the strategic plan is written by the facilitator, it becomes a paper tiger and ends in a desk drawer. So, it is important during the intake and preparation of the participatory strategic planning to make the client responsible for the documentation and dissemination of the strategic plan.

7. Keep the momentum and ensure that implementation continues straight after the participatory strategic planning event
Due to the participatory process many participatory strategic planning events gain a lot of enthousiasm, joy and energy with the group. Therefore, it is important that the momentum is continued after the strategic planning event.  The participants should become the owners of the implementation process and the senior management must have the courage to trust, to facilitate and delegate parts of the implementation to the staff. Every moment of delay in the process, will take away the energy and the momentum with the staff in the organization. So the energy and the momentum should be continued. Therefore it is important that the working groups, that are created at the end of the participatory strategic planning have formulated challenging milestones for the quarters to come.  If they have the opportunity to celebrate a victory within considerable time, the energy can be maintained and continued.




A book I can really recommend and is worthwhile reading, is the book ‘TransformationalStrategy’ written by Bill Staples. It describes the process of participatory strategic planning in detail, it puts the method in today’s context and it gives some helpful and useful tips and tools on how to facilitate it effectively. The book contains some valuable case studies from the profit and non-profit sector and it provides some checklist on how you prepare a successful participatory strategic planning.   See>>>  TransformationalStrategy’ written by Bill Staples.  The Participatory Strategic is part of the Technology of Participation methods (TOP). 

Thursday, February 19, 2015

Introduction training to Knowledge Management (KM) in the Agricultural and Rural Development Sector

From 3 - 7 November, 2014 CTA (Technical Centre for Agricultural and Rural Cooperation ACP-EU) organised a five day pilot training on Knowledge Management for development practitioners working in the agricultural and rural development sector. This pilot training was attended by knowledge management practitioners and development practitioners working on agricultural and rural development projects in the ACP (Africa, Carribean and Pacific) countries. The training was conducted by Lucie Lamoureux, trainer/ facilitator and knowledge management expert working with KM4D Associates. This five day training was held in Ede and was closed with a knowledge fair at the CTA Office in Wageningen, The Netherlands. Watch the video: 

The training provides an introduction into Knowledge Management.  Key questions which are answered during the training are;  What is Knowledge Management? What do I already do with KM in my organization and how do I create the proper conditions for having the appropriate cultural environment to do KM?  In addition participants learn a number of tools and methods on how they can initiate and encourage knowledge management in the organisation.  The course is very practical and provides a lot of useful tips and hints.

Tuesday, February 3, 2015

Knowledge Management for Senior Management in the Agricultural and Rural Development Sector

From 10 - 12 November, 2014 CTA (Technical Centre for Agricultural and Rural Cooperation ACP-EU) organised a pilot training in Knowledge Management for Senior Management in the Agricultural and Rural Development Sector. The training was attended by Knowledge Management and project managers and practitioners and experts in knowledge management. The 2 day training was composed of 6 units aiming to expose the participants with the basic concepts and tools of knowledge management. The training was conducted by Jaap Pels and Simon Koolwijk. Watch the video:



The video gives a logical explanation about the theory, the principles and the definition of knowledge management and it explains the process of the two - day training for senior managers. It shows how the senior management gets acquainted with some of the key models in knowledge management and how to relate this to the work in the own organization. During the video the participants comment on how they experienced the exercises and the training as a whole. 

Sunday, November 9, 2014

Advanced Training Participatory Video August 2014, Ethiopia

From 27th - 29th August, 2014 thirty participatory video facilitators and practitioners attended a three day advanced training in participatory video. The thirty pv facilitators are representing 15 social accountability implementing partners (SAIPs), that are involved in implementing social accountability processes in the sectors of health, education, agriculture, water & sanitation and rural roads in the whole country of Ethiopia. The pv facilitators conducted pv interventions between march and july, 2014. This training gave them the opportunity to share their experiences and update their knowledge on pv, to learn to work with advanced software in video editing and to present the M & E results and to discuss the opportunities and challenges on how to sustain participatory video in their organisation. On the third day of the training, the participatory video oscar reward 2014 was given to the Social Accountability Partner who best applied Pv in helping their community to support the process of change in improvement of service delivery in their community. Watch: 

Thursday, October 16, 2014

Results PV Oscar Reward Competition 2014 - Ethiopia

On the 29th August, 2014 the Ethiopian Social Accountability Program Phase 2 (ESAP2) organized the Participatory Video Oscar Reward ceremony in Addis Ababa. Fifteen organisations had submitted each 5 videos for participatory video interventions, they had conducted between March and July, 2014. Five organizations, who were nominated, had to compete prior to the PV Oscar Reward ceremony. Finally the winner was selected and rewarded in the Washington Hotel, Addis Ababa, Ethiopia. In this video some of the participatory video facilitators and a community member comment on how PV impacted and influenced their work. The PV Oscar competition brought some surprising result for them.Watch: