Governments across the world regularly pursue reforms that achieve less than was originally expected or is needed to make the state function better. The limits to reform success are often obvious in even the early days of reform, where gaps and weaknesses manifest. Many governments have no mechanisms built into their reform processes to see these gaps and weaknesses, however, and persist with predefined reform plans instead of adapting designs to close the gaps and address weaknesses. One antidote to this challenge is to create reflection points where reformers scrutinize their progress to identify weaknesses, reflect on these weaknesses, and adapt their next steps to address the weaknesses. In the spirit of John Kingdon’s work on ‘policy windows’, we call these reflection points ‘adaptation windows’—moments where reformers acknowledge problems in their reforms, adapt reforms to address such, and mobilize support for this adaptation. This paper discusses an effort to open an adaptation window for reformers to ‘see’ and then respond to public financial management (PFM) reform gaps and weaknesses in Mozambique. The paper details why and how this work was pursued, and also reflects on results of the government’s reflection at the adaptation window.
There is an inherent tension between implementing organizations—which have specific objectives and narrow missions and mandates—and executive organizations—which provide resources to multiple implementing organizations. Ministries of finance/planning/budgeting allocate across ministries and projects/programmes within ministries, development organizations allocate across sectors (and countries), foundations or philanthropies allocate across programmes/grantees. Implementing organizations typically try to do the best they can with the funds they have and attract more resources, while executive organizations have to decide what and who to fund. Monitoring and Evaluation (M&E) has always been an element of the accountability of implementing organizations to their funders. There has been a recent trend towards much greater rigor in evaluations to isolate causal impacts of projects and programmes and more ‘evidence base’ approaches to accountability and budget allocations. Here we extend the basic idea of rigorous impact evaluation—the use of a valid counter-factual to make judgments about causality—to emphasize that the techniques of impact evaluation can be directly useful to implementing organizations (as opposed to impact evaluation being seen by implementing organizations as only an external threat to their funding). We introduce structured experiential learning (which we add to M&E to get MeE) which allows implementing agencies to actively and rigorously search across alternative project designs using the monitoring data that provides real time performance information with direct feedback into the decision loops of project design and implementation. Our argument is that within-project variations in design can serve as their own counter-factual and this dramatically reduces the incremental cost of evaluation and increases the direct usefulness of evaluation to implementing agencies. The right combination of M, e, and E provides the right space for innovation and organizational capability building while at the same time providing accountability and an evidence base for funding agencies.