Contributing to a Problem Driven Project in Mozambique

PDIA in Practice Series: No. 2 | Download PDF | Listen to Podcast

Background: The PDIA in Practice Series

Many government policies and reforms fail in developing countries. Research at the Building State Capability (BSC) program ties such failure to the tendency of governments to adopt external ‘solutions’ that do not fit their contexts and overwhelm their capabilities. We believe that governments should build their capabilities by employing processes that allow their own people to find and fit their way to solving their country’s real problems. We propose a process for doing this, called problem driven iterative adaptation (PDIA) and have been working since 2009 to explore ‘how to do’ PDIA practically, in the real world. This note summarizes one of the engagements in this journey, and what we learned from this engagement. It does so by answering an adapted set of questions we always ask of PDIA in practice: What did we do? What results emerged? What did we learn? What did we struggle with? What was next?

What Did We Do?

This note covers our experience working with a broad group of officials across Mozambique’s public financial management (PFM) sector, and donors (particularly the World Bank) between March 2010 and December 2013. This was an early engagement in our learning on how to do PDIA, where we focused on helping government officials (and donors) prepare for and design a problem-driven donor-funded project on PFM for service delivery.

The project aimed to address gaps in the implementation of the country’s PFM system (the product of over a decade of reform), which manifested in financial, personnel, procurement and supply chain deficiencies at the local level and impeded the provision of key services to citizens.

The PDIA work emerged from a series of what we call adaptation window workshops with the government and the World Bank (in 2009). These workshops revealed the existence of compliance gaps and weaknesses in the PFM system, and showed that these gaps were most severe at the provincial and local level—undermining service delivery at the front lines. Officials from various sector ministries (education, health, agriculture, and justice) approached the World Bank after these workshops and asked if a project could be designed to close PFM system gaps in their sectors. These officials were responsible for managing high level resource allocation processes in their sectors (like developing annual budgets) and were interested in ensuring that PFM system weaknesses did not undermine service delivery.

The World Bank project team asked Matt Andrews (Faculty Director of BSC) to help facilitate discussions with government officials in these sectors—and in the Ministry of Economy and Finance—about the specific PFM problems undermining service delivery. Matt engaged with the World Bank team—visiting three to four times a year for a period of three years to hold discussions with different groups, centered on asking three key questions in the PDIA process: What is the problem? Why is it a problem? Where can you start in trying to solve the problem, and what can you do? 

The process of helping government teams to identify their problems was iterative and recursive in all sectors, and tended to follow the following structure:

  • The first step involved asking high level authorizers (ministers and senior bureaucrats) to identify the headline service delivery problem they were most concerned about, and to then authorize a working team to add detail to this general problem. The minister of health, for instance, identified the high maternal mortality rate in key provinces as his most worrying ‘problem’, and authorized a multidepartment team (of civil servants mostly from his ministry and central health sector agencies) to further engage in working with this problem.
     
  • A second step was to convene meetings with the newly authorized team and construct the problem; building a narrative about the problem by using data and stories to show the size of the problem and to explain why it mattered. The health sector team, for example, collected data on maternal mortality rates for Mozambique’s different provinces, and created a narrative about where this problem was most severe.
     
  • A third step was to return to the authorizer with the newly constructed problem, where the working team presented its data and stories on the size of the problem. This allowed an opportunity to adapt the problem definition so that it properly reflected available evidence and the authorizer's concerns and to ask for continued authorization for the process (which in some cases also involved authorizing more officials to join the working teams).
     
  • A fourth step involved gathering the working team to deconstruct the problem, by identifying reasons for the problem (and especially those related to PFM and resource processes). Matt Andrews employed various mechanisms to do this, including supporting a research-based approach in one team (where team members were asked to scour existing studies and identify, with evidence, the factors causing the problem). Donors who had worked on the problem were convened in meetings to offer findings from their studies as well, with the goal of drawing as much expert knowledge into the process as possible. Matt also experimented with a more dynamic and opinion-based exercise focused on tapping the contextual, experiential tacit knowledge of government officials (instead of analytical studies). Based on the ‘5 why’ method, team members were asked to offer their views (evidence-based or not) on ‘why the problem was being caused’, with multiple follow-up 'why' questions posed until the team felt it had identified the root cause (not just a manifestation of such). These processes yielded long lists of causes and sub-causes, which were then organized using Ishikawa (or fishbone) diagrams (see a basic structure of the fishbone diagram in Figure 1). These ‘fishbones’ allowed teams to showcase their educated opinions of the problem’s clustered causes and sub-causes.

Figure 1. The basic fishbone (or Ishikawa) diagram

The basic fishbone (or Ishikawa) diagram

Source: Ishikawa, Kaoru (1968). Guide to Quality Control. Tokyo: JUSE.

  • A fifth step was to have the team return to the authorizer and present the deconstructed problem, showing all the causes and sub-causes. This gave the teams opportunities to ensure the authorizer was on board with their version of the deconstructed narrative, and to use this to start identifying (and getting authorization for) tangible and realistic entry points for action, and to obtain authorization to expand team membership or consultation (as this was always required by such point, with the deconstruction exercise routinely illustrating the importance of agents outside of the team in the problem and, by extension, its solution). In the health sector, this step had multiple iterations (with different versions of the sixth step described below and the steps described above coming in-between the different authorizer check-ins). In one of the meetings, the team showed the Minister what it saw as the causes of maternal mortality in key provinces, and obtained authority to present its fishbone in provinces. The team then presented its fishbone at provincial level and was surprised by some feedback from provincial experts, which emphasized one key causal strand (related to the medicines supply chain) as THE PROBLEM itself. The team then re-constructed the problem to identify (with evidence) what the problem was in the medicines supply chain, and deconstructed this problem in more detail (in a new fishbone with four main causes) and returned to the Minster with a new fishbone—showing just the problems in the medicines supply chain—to see if he would authorize the change in problem definition and allow them to work on just this part of the fishbone. This decision also necessitated an adaptation in the team composition, where more members from the medicinessupply chain were engaged and the head of the Central Medicals Store was appointed team leader.
     
  • A sixth step (which sometimes ran in parallel with steps 4 and 5) involved the team reaching out to other agents to validate their problem definition and fishbone, and/or to engage in a practical exercise to obtain their own first-hand evidence of the problem's size or causes. These kinds of steps were intended to increase views on the problem and to enlist more participation in the process (from bureaucrats at different levels of government, for instance, or from donors involved in the sector). In the health sector, for instance, the team initially took its fishbone diagram on maternal mortality to various provinces and asked for various implementers’ views. The team members were surprised at the high level of engagement by these implementers, around the problem, and had to make many changes to the fishbone diagram. In particular, the province-level contacts all stressed the defining importance of one causal strand—related to the slow or non-delivery of medicines to health posts—which led the working team to consider focusing on medicines delivery as THE PROBLEM itself. Once they had the approval of their minister to do this, the team decided to build its new narrative of the problem and its causal analysis in an active manner, by visiting central storage facilities (warehouses) and examining first-hand what the problem involved and what they ‘saw’ was causing it. This practical exercise gave the team a common, tangible feel for the problem, and helped to galvanize their team spirit as they prepared to work towards a solution.

Only three teams—education, health and justice—made it (at various points in the first half of 2011) as far as identifying the problems and to address the causes of those problems. But the work was not done. An operational vehicle needed to be created to address the problems. 

The World Bank team had identified the ‘Program for Results’ (PforR) financing instrument as the most appropriate to use in this process to support the government teams to tackle the problems they had identified. The PforR approach allowed for a high level of flexibility in implementation, which is vital if a genuinely PDIA-style iterative and adaptive process is going to be employed, focused on finding locally workable solutions to the problems (as opposed to a more planned-out process that focuses on adopting a predefined solution). 

The PforR approach required that the government identify ‘results’ to aspire towards, and then identify milestones for these results (where money would be disbursed in support of reaching the intermediate and final results, with no predefined solution about 'how’ the results would be obtained). This requirement was difficult to reconcile with PDIA: the idea of shifting conversations about ‘problems’ to now talk about ‘results’ seemed antithetical to the PDIA process that was now so advanced. The shift was particularly worrying because many results indicators in development projects focus on adopting parts of predefined solutions (which locks the solution in and undermines the process of searching for local solutions). After some discussion, an approach to identify ‘results’ was identified that did not clash with the problem-driven nature of the work: the results would be identified by asking ‘what would the problem look like solved?’

This question was optimistic, and caused the teams to think in an aspirational way about what was possible if they were given support to tackle their problems in new ways. It was a question that could be asked with different time horizons, and led to the identification of aspirational results in six-month periods from the date of starting the project to four years after that date. These results indicators did not contradict the problem driven focus of the work, given that they were derived from the discussion about problems. In fact, the identification of ‘problem solved’ targets enriched the process and motivated the teams. 

The project was not yet ready to proceed, however. The proposed implementation process still needed to embed PDIA process ideas on how the government could move from the problem to action and implementation. Questions were numerous: What would this process look like? Who would need to be involved? What role would the World Bank play in this process? and more. Much of 2012 was spent reflecting on these kinds of questions, and identifying an iterative process that government teams could follow in tackling the problems (and causes).

The process design stressed mobilizing teams of actual implementers located as close to the problems as possible (rather than external experts) to: 

(i) identify ideas to try (rather than predefined solutions) and then

(ii) experiment with these ideas and stop at specified points to assess results and harvest lessons (with the help of a coach), and then

(iii) adapt the ideas and try again.

The PforR approach supplied coaches to teams, provided financial support to teams in the experimentation process, and also offered financial incentives for teams reaching targeted results (intended to motivate participation in the process). The full details of this approach were extremely detailed and different to most other World Bank operations, and were subject to many rounds of discussion and deliberation at the Headquarters in Washington (where multiple mid-level and high level meetings were required to ensure full understanding of the approach and to bring the project to a decision point).

The project was only approved and activated in 2014 as a result of this lengthy process, and in the end only the health and education sectors were included (to work alongside the Ministry of Economy and Finance). Matt Andrews and BSC were not involved in the engagements after December 2012, a point at which the project was patiently shepherded into being by government and World Bank counterparts who had been pushing the agenda since 2009 (some of whom then led or contributed to implementation all the way until 2018).

What Results Emerged?

The process delivered notable results even in the first two years of project preparation. Sector agencies were engaged in reflecting on their problems more than was previously the case, and many lessons were learned about these problems (through the problem construction and deconstruction processes). ‘Improved interaction and learning’—often through an adapted conversation—is seldom recognized as a ‘result’ of this kind of intervention, but is actually vital when the aim is to build capability and foster change in the face of complex problems. These problems can only be managed (or sometimes even solved) by improving engagement (because they typically involve many distributed agents) and expanding know-how. 

Beyond these key capability-enhancing results, however, one might be critical of how long the project preparation process was—and how long it took to even start the action and implementation processes that would ultimately deliver results. While we at BSC believe this kind of process could run faster (as discussed in the section on what we struggled with), the government and World Bank teams were doing something new and this always takes time—especially when it is not only novel but challenges entrenched norms and practices. Regardless of these rational reasons for the time taken, there were voices in the Maputo donor community and in the World Bank who spoke out against the project and called for a more traditional design and approach (that they believed could be delivered more rapidly).

These critiques continued into the project itself (after 2014); especially in the first 18 months. Implementation status reports (ISRs) show that the project started slowly. It took time to find and appoint coaches to work with teams, and to mobilize the teams to develop ideas for action and to activate these. Critics in donor agencies (including some in the World Bank) portrayed these difficulties as the product of design weaknesses, and some international experts called in to help on the project were alarmed by the messy processes they saw. Some of these experts engaged Matt Andrews to ask if he thought the experience showed that the PforR and PDIA approach was poorly suited to a low capability, complex environment like Mozambique. Whereas these experts believed they were seeing a messy and potentially failing project, we at BSC believe they were just seeing the PDIA process in action. This process was intentionally focused on helping local actors engage with their uncertain and complex settings to find workable domestic solutions to their problems. At BSC, we have found that the PDIA process is generally novel and is also typically messy—and often goes through ups-and-downs rather than following a linear progression to completion. Literature shows that this messiness is common (and needed) in the face of complex problems and when the goal is to build local capability, especially in low capability contexts. The challenge in Mozambique was that this messy process was playing out through a vehicle (an international donor project) that was not typically open to messiness.

However, the project moved through its opening storms quite quickly, and by 2015 was receiving 'satisfactory' ratings on implementation status reports about progress on meeting objectives. The transition towards performance was largely due to the patient and resilient project management by key members of the World Bank team and government counterparts, and because of the success in building a broad coalition of reform implementers across the health and education sectors. The World Bank team (with supportive authorizers in Washington and with committed government counterparts in Mozambique) routinely scored the project risk as 'substantial', revealing a realistic view on how difficult it was, but their commitment to making it work was exemplary. By working steadfastly to ensure the project was moving ahead (even in the face of criticism, and when roadblocks manifest) they helped to ensure that the project provided an effective holding environment for local experimentation and learning.1

As of April 2018, the project was in its final year having benefited from a one-year extension (quite common to World Bank projects). It had already produced impressive results, on a number of dimensions:

  • The number and diversity of agents involved in the projects was much larger than in any similar project, with engagement extending to the point of service delivery (where PFM systems gaps were most pronounced). The project managed to engage commitment and participation of the agents in the ministries of economy and finance, health and education, and in the central medicines storage unit, and in provincial education and health and medicines storage entities, and in district level health and education facilities, and all the way down to local health centers and over 1,000 schools (where principals, parents and local officials were all involved in experimenting with new ideas to solve old problems). 
     
  • Most of the results indicators had been achieved (with significant progress in getting to 'problem solved' in the education and health sectors).2 For instance, a variety of new solutions implemented across the country ensured that over 1,000 schools complied better with policies governing financial flows (where persistently late or interrupted fund flows had been identified as a cause of service delivery failure in schools, and a sub-cause of this was that schools did not meet the criteria required for these finances to flow). These 'results' meant that budgetary funds were flowing more reliably to over 1,000 schools (a fundamental ‘problem solved’ goal of the project). In other examples, most provinces had introduced measures to ensure improved compliance with standards of stock management, warehousing and the distribution of medicines (needed before medicines are released to those locations), and over 80% of health centers were properly filling-out their requisitions for tracer medicines (needed to ensure supply), and the proportion of treatment sites with stock-outs of key medicines had declined from 27% to 5%. 
     
  • In short, many small interventions produced large results in a short period, closing systems gaps that persisted before the project began—and improving system effectiveness overall. 
     
  • Government capabilities were improved through the process, with 2017 implementation status reports referring to changed behaviors and changed work processes that can be directly attributed to this project (including a new and diffused approach to identifying and tackling problems in health and education, the introduction of indicators as management tools, and a consolidation of new coaching capabilities).3

There were a number of dimensions of the PforR project that are not explicitly part of the PDIA approach (especially the use of financial incentives to motivate performance) and status reports suggest that these other dimensions were important in fostering the results achieved. We at BSC are confident that the explicitly PDIA dimensions have also been vital in fostering these results, however (where these include problem construction and deconstruction, working with broad groups, iterating and learning):

  • It is clear that the impetus for this kind of project dated to the adaptation window workshops in 2009, for instance (and would not have happened without these workshops).
     
  • Furthermore, the influence of other PDIA dimensions are reflected directly in a comment made in an April 2018 report on the project’s success:4"The project adopted an innovative problem-driven and iterative approach towards implementation," said Humberto Cossa, World Bank senior health specialist supporting the implementation of the health component. "Frontline implementers focused their attention on identifying bottlenecks to the achievement of results in their sectors, bringing people out of their silos to craft integrated solutions." The project engaged a team of coaches and facilitators who were assigned to each of the participating ministries and in each province to support the coordination, behavior change and implementation discipline needed to reach the targets."

Some might ask how these results compare with a 'counterfactual' in which government followed a more traditional solution-driven, pre-planned, top down PFM project approach instead of one that focused on addressing specific problems and gaps through an iterative process. Interestingly, Mozambique provides a good domain to investigate such a question. With the help of a variety of donors, the government pursued a more conventional PFM reform program alongside the PforR project. This other program was conceived, managed and implemented by a narrow set of Ministry of Economic and Finance (MEF) departments (and related agencies) and focused on introducing new formal PFM mechanisms and systems (particularly IT systems) that built on reforms between 1998 and 2011. These prior reforms had generated compliance gaps prior to 2009, particularly at the local level (some of which were targeted for solution in the PforR). The more traditional MEF strategy did not directly address these gaps, opting rather to promote even more formal systems development. Unfortunately, history shows that the gaps in these systems festered; and the system's continued weaknesses contributed to a corruption crisis in 2016 (where evidence emerged of money being spent outside of the system, and of compliance failures undermining the credibility of spending mechanisms). 5

Further research into this situation is required, but it seems that the 'problem driven' project approach yielded more relevant and effective results (and contributed to ensuring PFM and management systems actually worked to facilitate service delivery)than the more conventional project (which did not close system compliance gaps and may even have exacerbated these). Additionally, the PforR project results were not all that came from this work. The long process of World Bank project preparation allowed time and opportunity for the emergence of another externally financed operation, USAID's Results-Based project in Mozambique’s Central Medical Store (CMAM). This initiative focused explicitly on the problems identified through the PDIA process (poor information accuracy and flow between the central, provincial, and district levels, ad hoc distribution of medicines from provinces to districts and health facilities, fragmented management responsibility, and inflexible financing), and employed an iterative, adaptive approach to facilitating reform. The project has been evaluated as a success, with a 2016 study showing evidence of improved 'results' and capability: 6

"CMAM's performance continually improved over baseline and … achieved many of its performance targets, for example, timely submission of quarterly supply and distribution planning reports. Warehouse indicators, such as inventory management and order fulfillment, proved more challenging but were nonetheless positive. By linking payments to periodic verified results, and giving CMAM discretion over how to spend the funds, the RBF agreement motivated the workforce; focused attention on results; strengthened data collection; encouraged teamwork and innovation; and ultimately strengthened the central supply chain."

As with the PforR project, this USAID-funded intervention employed mechanisms that are not typical in the BSC PDIA model (especially results-based financing and financial incentives). However, similar to the PforR project, the evaluations of this intervention speak directly to PDIA elements when explaining success. In particular, mention is made of the up-front interactions around ‘the problem’, and the learning that emerged through the iterative and adaptive process. The 2016 study notes, in particular, that “the flexibility to evolve as lessons were learned was an important element of the positive results of the FARA in Mozambique."

1 The 'holding environment' idea is drawn from the literature on Adaptive Leadership (a la Ron Heifetz and Marty Linsky). It is a space where agents are forced to address the discomfort associated with a need for change (like the awareness of problems and gaps in the Mozambican PFM system) but are given a safe space to address this change (to experiment with new ideas, fail if necessary, and learn in a constructive way).

2 For more detail on these results, see World Bank (2018). Placing Results Front and Center in Health and Education in Mozambique.

3 The World Bank's December 2017 Implementation Status Report notes, “[T]he counterpart has begun to internalize result-based management processes, with new performance-based allocation (PBA) mechanisms and DLI-based operations in the three ministries. The approved extension of the Program closing date by one year will help consolidate these gains while enabling key PBA expenditures to target service users. Coaches and Provincial facilitators and focal points have demonstrated increasing capacity and engagement for change management. The Ministry of Economy and Finance, in consultation with sectors, is also seeking to internally streamline the transfer of funds to accelerate the use of the capacity development window in the remaining year."

4 See World Bank (2018). Placing Results Front and Center in Health and Education in Mozambique.

5 See Andrews, M., McNaught, T., and Samji, S. 2018. "https://bsc.cid.harvard.edu/publications/pfm-reform-gaps-mozambique”. CID Working Paper 341.

See Spisak et al. (2016). Results-Based Financing in Mozambique’s Central Medical Store: A Review After 1 Year. Global Health: Science and Practice March 2016, 4(1):165-177.

What Did We Learn?

Lessons about the potential and value of PDIA
We learned a lot from this experience; between 2010 and 2012 when we were working on the initiative directly, and in the 2014-2018 period where we were observing its implementation.

A fundamental lesson is that a focus on problems can change the reform conversation—from one dominated by high-level external experts and centered on ‘what-should-be solutions’ to one that involves many different local actors and emphasizes the realities of 'what is' and how this can be improved through active, on-the-ground experimentation by locals and for locals.  

This lesson emerged early on, through new connections between government actors working from the bottom up and from the top down, who found common ground in the conversations about problems. The lesson was consolidated over time, as the process diffused to the frontlines of service delivery and saw low and high level officials working together to propose new ideas and experiment with these ideas to solve shared problems. These engagements were novel in Mozambique, which has a hierarchical public sector and segmented social structure. The experience showed that problem-driven conversations and iterative experimentation can bring actors together even across established organizational and governmental barriers.

A second lesson is that PDIA can be used by aid organizations like the World Bank, in both fostering conversations that feed project development and in structuring and implementing the projects. The problem-driven conversations were key to establishing a need for the PforR financing instrument, and brought agents together as the clients and implementers of the donor-funded project. The PDIA-like emphasis on broad-based engagement was also key to ensuring that the project had high-level authorization, mid-level policy engagement, and low-level (implementation level) traction. Further, the PDIA-like emphasis on iterative experimentation and learning during project implementation allowed for the emergence of localized solutions to the problems identified, and to the empowerment of local officials in this emergence process. Given the PDIA-like process applied in this project, results were not just achieved as ‘problem solved’; there was also significant building of state capability at multiple levels of the Mozambican system. 

This learning on the potential and value of the PDIA-like process helped us develop a rough view on how one might think of (and even evaluate) the impact of such engagement, shown in the following figure and described thereafter. 

Figure 2. A simple approach to thinking about the impact of PDIA-like interventions 

A simple approach to thinking about the impact of PDIA-like interventions

The figure shows, first, that the PDIA process yields new relationships within government, and across different parts of government (both vertical and horizontal). These relationships open new opportunities for sense-making (especially about problems and ideas to resolve problems) and for experimentation (especially when such required the engagement of actors located in different parts of the government structure). The new relationships also facilitate modest shifts in power across the systems (with lower level officials playing more of a role in reforms than they had before, for instance, and line ministry officials having a say about public financial management issues that are usually reserved for ministry of finance experts). These modest power realignments foster even more engagement and interaction in the government system.

Second, the process facilitates significant new experiential learning across actors in government, which–when combined with the new relationships—generate emergent improvements in their capabilities. The lessons center on ‘soft’ management mechanisms necessary to identify and address problems, motivate personnel, mobilize support for new ideas (and more). Lessons also include the emergence of new ‘hard’ policy and implementation ideas that help resolve problems (mechanisms to make parent committees actually work in schools, for instance, or that improve the rate of fund release to schools). These lessons are crucial to establishing new capabilities in government, which were already having knock-on effects in Mozambique in 2018 (with government officials employing the new management mechanisms in other areas).

Third, the PDIA-like process produces indirect and emergent new ideas and opportunities and vehicles for intervention, like the many novel solutions emerging in the PforR project, the newly productive reform coalitions established, and the USAID project. These ideas, opportunities and vehicles emerged as new conversations about problems exposed change needs and opportunities that could be exploited by agents involved in new relationships.

Fourth, the PDIA-like process facilitates the achievement of real progress in solving entrenched problems—in a step-by-step process—as reflected in improved performance on key ‘problem solved’ indicators.  One could say that this progress took the form of new or improved outputs (like systems to monitor medicine storage and processes to validate school funding requests) and outcomes (the decreased incidence of stock outs at medical stores, for instance, or the higher rate of on-time financing to schools) and impacts (like more citizens having timely access to medicines and more schools having uninterrupted ability to teach students).  

Interestingly, most of the results evidenced in this work were not the direct product of BSC (or Matt Andrews’) engagement in the 2009-2012 period. Most of the results came about after this period because local actors (including World Bank officials located in Mozambique) diffused the process and/or took advantage of the creativity fostered through this process. As with our prior work in Mozambique, therefore, “we learned that the PDIA process really has its impact via indirect or second-order influences that emerge through local agents—where the externally led intervention nudges internal agents to think and act and interact differently, empowering new engagement and learning-by-doing, building internal capabilities to act, and fostering emergent and potentially surprising responses.” 

Lessons about doing PDIA
We built upon our ideas about problem construction and problem deconstruction from this experience. We learned that our simple processes of getting agents to think about ‘what the problem is’ and ‘why it matters’ and ‘what is causing it’ were useful and effective, and translated easily across many domains and actor-groups. We also learned that the process of constructing and deconstructing problems is itself iterative, especially when the problem is complex and involves multiple agents in different positions (who engage with the problem in different ways).  An important lesson in this regard is that problem driven convening needs to be staged, to harvest many views and to mobilize attention and support and ownership of enough agents to make real progress. A simplified rendering of how this convening was done follows:

  • The convening here began with engaging political and bureaucratic authorizers to identify the high-level problem(s) they were most concerned about, to get authority for future convening.

  • It progressed to work with small teams of central government technicians who were charged with developing the problem narrative and deconstructing the problem into its causes.
     
  • The small teams then engaged with other distributed teams to get views on the problem and its causes, and made adaptations based on the interactions (to their teams and problems).

  • The adapted teams and problems were presented back to the authorizers for approval.

While simplified, this process is demonstrably different to the convening methods one might find in work on participatory governance (where large convenings are often advised at the outset of a change process to ensure everyone has a voice from the start). The experience here was that convening needs to achieve results quickly, and this is best done with small engagements. ‘Where one starts is not where one ends up’, however, and the convening process allows for greater participation and reach and engagement over time.  

This approach to problem driven convening was the first time that we at BSC started employing Marshall Ganz’s snowflake model of social organizing when thinking of how to convene for change. Our application of this approach suggests that one starts with a small central team of agents and allows that team to build outwards (through their contacts) to other teams (as in Figure 3, where the central team is the darkest set of dots and the lighter dots show agent teams further and further diffused from the center). This means one has many small and context-specific convening points (not one huge convening point) that emerge organically through the local agents involved.  

As agents are engaged—asked about problems, or beyond this about ideas for solving such—their voices are empowered and they are in fact introduced into leadership positions. 

Figure 3. Convening in a snowflake model of agent engagement

Convening in a snowflake model of agent engagement

Source: Adapted from our inspiration, Ganz, Marshall (undated). Organizing: People, Power, Change.

We also learned a great deal about structuring the iteration and learning process following problem construction and deconstruction. The iteration design was simple: teams identify ideas, try these ideas out in action in tight time-bound periods, stop and reflect on their experience, adapt their ideas based on their learning, and try again. This thinking was crucial in our design of PDIA Action Pushes in later work (where each action push is an iteration). 

We also learned a lot about the way PDIA can interact with more traditional management approaches, which focus on solution-based results and not problems and which employ linear planning and implementation tools like log frames and GANTT charts. We found that one can identify aspirational ‘results’ associated with PDIA processes without departing from the focus on problems, by having teams identify what ‘problem solved’ would look like at different points in the future. These aspirational goals could provide milestone-like points for teams to focus on, much as one would have milestones in a linear strategy (like a log frame). This thinking informed our Searchframe concept adopted in later initiatives.  

We learned that organizations like the World Bank require this kind of reconciliation (of PDIA to traditional management methods) in order to apply PDIA thinking in their projects. This is because traditional management methods dominate the way projects are developed; totally new approaches like PDIA are difficult to introduce 'as-is'.

What We Struggled With

Introducing PDIA-like approaches into established donor vehicles
This work was explicitly focused on preparing and ultimately implementing a World Bank project using PDIA-like tools. This was a difficult objective, given that traditional donor approaches to project identification and implementation contrast a lot with PDIA approaches. There were many challenges, therefore, as we tried to stay faithful to the PDIA methods but also package these methods in ways acceptable to the World Bank. It was vital to have a World Bank team that was committed to PDIA and also knew the detailed ins-and-outs of the organization’s procedures. This team identified a leading edge project type in the World Bank, called the Program for Results (PforR), which allowed a lot of flexibility in the identification and implementation of solutions to problems. This provided a vehicle through which PDIA could be applied, even though quite a lot of ‘reconciliation’ was still needed to frame aspects of the PDIA process to make sense (and be acceptable) in a World Bank operation. The experience showed that PDIA could indeed be incorporated into donor operations, albeit with a great deal of commitment and shoehorning.

Dealing with ‘the messiness’ of problem driven conversations
As in most of the PDIA work, the work was messy, and required multiple iterations and broad engagement that was not typical to donor projects. This messiness was not well received by many in the donor community—and even by some in the World Bank and government. People in these entities pressurized the process, questioning both the PDIA method and the fit of such in a PforR project. This was a struggle which we weathered because demand for the PDIA approach was established early on amongst key authorizers in the government and on the World Bank team. These agents had been part of Adaptation Window workshops in 2009, where ‘gaps’ in Mozambique’s reforms were clearly shown, and it was made apparent that incumbent reform approaches would not close these gaps. 

Time and momentum difficulties
This work took a long time; it was over three years from the first Adaptation Window workshop to the finalization of the World Bank project. This long delay fed into critiques that PDIA would take too long to be practical. The delays also made it very difficult to build momentum, which meant that project progress was always being interrupted.

The PDIA process was not the reason for these delays, and the long preparation time, however. These issues were rather a product of the World Bank project preparation protocol, which emphasizes the role of the World Bank staff members in project preparation. Given the dependency on World Bank staffers, the project took shape largely during visits by World Bank team members (and Matt Andrews) that occurred only every three to four months. The dependence on irregular visits by outsiders meant that the project only received attention a few days each month, and ultimately took years to prepare. 

We at the Building State Capability (BSC) program learned a lot from this, and developed a mantra about time and PDIA interventions; ‘you need to work in days and weeks instead of months and years’. This mantra has carried into all of the BSC work, which now emphasizes a fused project preparation and implementation process where teams of insiders lead the process and work consistently ‘in days and weeks’ to make the most of time, build momentum, and achieve rapid and consistent progress.  

This approach is vital to building and growing authorization for action, and for ensuring that teams experience progress and success needed to foster intrinsic motivation. 

Reflecting on our role, as external agents
As with all of our work, we at the Building State Capability (BSC) program (and Matt Andrews in this case) struggled with determining 'our' role in the change process—and establishing new expectations for this role in the eyes of those with whom we were working. We were seen as outside experts called in to advise on ‘what and how’ the government should proceed with reforms. Our most direct engagement was with the World Bank team; who were also seen as experts, and were also financiers to the government.

This situation was difficult to manage, and fed many entrenched perceptions of what our roles should be and how we would engage with government. For instance, the project preparation process proceeded in inefficient fits and starts that coincided with World Bank team visits partly because ‘this is how it always was’—the World Bank team members would call most of the shots, do much of the expert work, and prepare the project, so government counterparts tended to wait for their visits instead of moving ahead with the work in their absence.  

Matt Andrews ultimately dropped out of the team—and the World Bank relationship—to try and establish a new kind of facilitator role with the government. Through this experience, and others, Matt and the BSC team have learned to always ‘give the work back’ to government officials in local contexts—and limit our own role to that of process facilitator. This is not to say that outsiders should never be playing more expansive roles—as experts or financiers in the case of entities like the World Bank (whose staff members played vital roles in the PforR experience). The lesson is, rather, that PDIA processes are designed to bring local officials back into their own policy and reform processes, and outsiders working on PDIA processes need to let this happen. 

What Was Next?

Following this work, we continued engaging in Mozambique—taking the next step of working as independent facilitators with teams of government officials who were not involved in the World Bank project. These teams were in the justice and health sectors. This work allowed us to experiment with problem construction, problem deconstruction, action push periods, and coaching. It was also our first independent piece of work with government officials, and led to significant lessons on taking PDIA process from problem analysis through experiments and to results.