Many policy research organizations are grappling with how to assess their policy influence in a sustainable, viable and meaningful manner. Some of them have tried to develop monitoring and evaluation methods with the aid of experts and/or donors but many of them failed to really implement and use these systems for many reasons: they become too time consuming, the type of indicators are too rigid for the different programs, they end up with reports that no one uses in terms of decision-making, etc.
Several others count with some concrete tools to monitor and evaluate one or two sorts of results: visibility (by tracking media appearances) or reach (paper downloads, web site traffic, or re-tweets), which provide of course valuable data but not enough to make any important decisions in terms of strategies, focus, etc.
In the course “Monitoring, evaluation and learning on policy influence at developing countries” which P&I is currently facilitating with the support of the Think Tank Fund and the Think Tank Initiative, participants have recently developed a SWOT analysis or a self-assessment capacity focused on monitoring, evaluation and learning (MEL). This is for us a key step in terms of setting the basis to then decide their MEL strategies or even baby steps (some think tanks may opt to track 1-2 new indicators or to deploy one new tool rather than designing a whole system, as humble as this may be, due to their current situation). Organizations need to start where they are: one way is by acknowledging their strengths, weaknesses, opportunities and threats. Another option is to assess their capacity today by analyzing who is doing some MEL (is it the communications officer with his/her reports, is it the Board by requesting some key evidence by the end of the year, etc.?), with which tools, using what resources, and how that is used (among other things).
Why is knowing this so important? It allows people to raise their awareness both on their potential (they usually discover that there are some strengths or capacities underused such as data collection skills of researchers) and their short comes (reports are produced solely for donors and not used as a unique and real opportunity to learn on an ongoing basis). Some may get very confident on their large potential and this might help them start to innovate in terms of MEL. Others might realize that they should begin by addressing other major challenges before investing energy in MEL. Either way, such an exercise allows them to make more strategic and evidence-informed decision (evidence might include issues such as interests and motivations of staff members in terms of MEL).
By Dietmut Teijgeman-Hansen under CC license
Furthermore, they can start thinking about how to link strengths with opportunities, or opportunities to overcome the threats, etc. A more systemic thinking is enabled when participants are requested to re-build the big picture. This is a needed “zooming out” effort: the entire landscape is seen from above so that then organizations can decide how to then “zoom in”: select where to start and focus on.
Linked to this more holistic view is our suggestion (an insistent one!) that participants conduct this type of exercise with other team members so as to foster discussion, consensus-building and collective brainstorming. Group work triggers effects at the organizational level (such as becoming more aware of how each one plays or could play a role in MEL) as well as paves the way for future changes that think tanks might want to promote after the course has ended. The exercise serves as a thermometer to detect hot areas, i.e. issues that need to be attended first or opportunities that should be seized really soon, as well as where the conflicts and opposing views/priorities emerge. In that sense, one can begin to imagine what sort of MEL innovation could work better for almost everyone. Or even which risk can be taken in that direction that would not entail a significant loss.
Due to all these reasons, we insist on the importance of starting from where you are to evaluate, monitor and learn from policy influence. Is it that simple? Yes, the principle is simple. What is not that easy is to find the time and way to make sure your organization has a deep and complete understanding of where you really are (potentials and risks, included). A course provides a good stop-over to do so. However, if others are interested in trying this out, we offer our help to share tools first and then provide feedback with what you develop. Just let us know!
Comments