The enthusiasm for the potential of RCTs in development rests in part on the assumption that the use of the rigorous evidence that emerges from an RCT (or from a small set of studies identified as rigorous in a “systematic” review) leads to the adoption of more effective policies, programs or projects. However, the supposed benefits of using rigorous evidence for “evidence based” policy making depend critically on the extent to which there is external validity. If estimates of causal impact or treatment effects that have internal validity (are unbiased) in one context (where the relevant “context” could be country, region, implementing organization, complementary policies, initial conditions, etc.) cannot be applied to another context then applying evidence that is rigorous in one context may actually reduce predictive accuracy in other contexts relative to simple evidence from that context—even if that evidence is biased (Pritchett and Sandefur 2015). Using empirical estimates from a large number of developing countries of the difference in student learning in public and private schools (just as one potential policy application) I show that commonly made assumptions about external validity are, in the face of the actual observed heterogeneity across contexts, both logically incoherent and empirically unhelpful. Logically incoherent, in that it is impossible to reconcile general claims about external validity of rigorous estimates of causal impact and the heterogeneity of the raw facts about differentials. Empirically unhelpful in that using a single (or small set) of rigorous estimates to apply to all other actually leads to a larger root mean square error of prediction of the “true” causal impact across contexts than just using the estimates from non-experimental data from each country. In the data about private and public schools, under plausible assumptions, an exclusive reliance on the rigorous evidence has RMSE three times worse than using the biased OLS result from each context. In making policy decisions one needs to rely on an understanding of the relevant phenomena that encompasses all of the available evidence.
Core dual ideas of early development economics and practice were that (a) national development was a four-fold transformation of countries towards: (i) a more productive economy, (ii) a more responsive state, (iii) more capable administration, and (iv) a shared identity and equal treatment of citizens and (b) this four-fold transformation of national development would lead to higher levels of human wellbeing. The second idea is strikingly correct: development delivers. National development is empirically necessary for high wellbeing (no country with low levels of national development has high human wellbeing) and also empirically sufficient (no country with high national development has low levels of human wellbeing). Three measures of national development: productive economy, capable administration, and responsive state, explain (essentially) all of the observed variation in an omnibus indicator of wellbeing, the Social Progress Index, which is based on 58 distinct non-economic indicators. How national development delivers on wellbeing varies, in three ways. One, economic growth is much more important for achieving wellbeing at low versus high levels of income. Two, economic growth matters more for “basic needs” than for other dimensions of wellbeing (like social inclusiveness or environmental quality). Three, state capability matters more for wellbeing outcomes that depend on public production than on private goods (and for some wellbeing indicators, like physical safety, for which growth doesn’t matter at all). While these findings may seem too common sense to be worth a paper, national development--and particularly economic growth—is, strangely, under severe challenge as an important and legitimate objective of action within the development industry.
This paper re-examines why global collective action problems persist, and how to overcome them. Drawing on 140 interviews with campaigners, politicians, and businesses in 10 European countries, it suggests that many activists are stuck in a despondency trap. Never seeing radical reform, they lower their ambitions, and invest in more feasible but sub-optimal alternatives. This creates a negative feedback loop, in which the dearth of radical reform becomes self-fulfilling. But if reformists see advances at home and abroad, they may become more optimistic about collective mobilisation and break out of their despondency trap. This is shown by tracing the drivers of ground-breaking legislation. From 2018, large French firms must mitigate risks of environmental and human rights abuses in their global supply chains, or else be liable. This bill – the world’s first of its kind – was vociferously contested by businesses. But French campaigners and politicians persisted for four years, because they saw reasons for optimism. These include growing international support; public outcry; the French political culture (state intervention, and distrust of multinationals); together with a Centre-Left Government. Optimism galvanised relentless mobilisation. Legislative success in France then delivered a positive shock to activists across Europe, who were emboldened to launch similar campaigns and escape their despondency trap.
This is one of a series of working papers from “RISE"—the large-scale education systems research programme supported by the UK’s Department for International Development (DFID), Australia’s Department of Foreign Affairs and Trade (DFAT), and the Bill and Melinda Gates Foundation.
Support for gender equality has risen, globally. Analyses of this trend focus on individual and/or country-level characteristics. But this overlooks sub-national variation. Citydwellers are more likely to support gender equality in education, employment, leadership, and leisure. Why is this? This paper investigates the causes of rural-urban differences through comparative, qualitative research. It centres on Cambodia, where the growth of rural garment factories enables us to test theories that female employment fosters support for gender equality: potentially closing rural-urban differences; or whether other important aspects of city-living accelerate support for gender equality. Drawing on this rural and urban fieldwork, the paper suggests why social change is faster in Cambodian
cities. First, cities raise the opportunity costs of gender divisions of labour – given higher living costs and more economic opportunities for women. Second, cities increase exposure to alternatives. People living in more interconnected, heterogeneous, densely populated areas are more exposed to women demonstrating their equal competence in socially valued, masculine domains. Third, they have more avenues to collectively contest
established practices. Association and exposure reinforce growing flexibility in gender divisions of labour. By investigating the causes of subnational variation, this paper advances a new theory of growing support for gender equality.
Many development agencies and governments now seek to engage directly with local communities, whether as a means to the realization of more familiar goals (infrastructure, healthcare, education) or as an end in itself (promoting greater inclusion, participation, well-being). These same agencies and governments, however, are also under increasing pressure to formally demonstrate that their actions ‘work’ and achieve their goals within relatively short timeframes – expectations which are, for the most part, necessary and desirable. But adequately assessing ‘community-driven’ approaches to development requires the deployment of theory and methods that accommodate their distinctive characteristics: building bridges is a qualitatively different task to building the rule of law and empowering minorities. Moreover, the ‘lessons’ inferred from average treatment effects derived from even the most rigorous assessments of community-driven interventions are likely to translate poorly to different contexts and scales of operation. Some guidance for anticipating and managing these conundrums are provided.
This paper shows that countries may reduce labour repression if they perceive this as conducive to export growth. This paper traces what happened before, in the presence of, and then following the withdrawal of international economic incentives for pro-labour reforms in Vietnam and Bangladesh. The Government of Vietnam announced it would allow independent trade unions, in order to join the Trans-Pacific Partnership (TPP) and increase market access. Similarly, the Government of Bangladesh rescinded restrictions on unions, following global condemnation of Rana Plaza and fear of buyers leaving en masse. Both governments reduced labour repression to promote export growth. With high-level authorisation, Vietnamese and Bangladeshi activists and reformists became less fearful, and mobilised for substantive change. However, these economic incentives were short-lived: after Trump’s election, the USA withdrew from TPP; buyers continued to source from Bangladesh, and squeezed prices (without requiring labour reforms). Both governments then amped up labour repression - notwithstanding private regulation, economic upgrading, industry growth, and mass strikes.
Evaluations of development projects are conducted to assess their net effectiveness and, by extension, to guide decisions regarding the merits of scaling-up successful projects and/or replicating them elsewhere. The key characteristics of ‘complex’ interventions – numerous face- to-face interactions, high discretion, imposed obligations, pervasive unknowns – rarely fit neatly into standard evaluation protocols, requiring the deployment of a wider array of research methods, tools and theory. The careful use of such ‘mixed methods’ approaches is especially important for discerning the conditions under which ‘successful’ projects of all kinds might be expanded or adopted elsewhere. These claims, and the practical implications to which they give rise, draw on an array of recent evaluations in different sectors in development.
A defining task of development is enhancing a state’s capability for policy implementation. In most low- income countries, alas, such capabilities seem to be stagnant or declining, in no small part because dominant reform strategies are ill-suited to addressing complex non-technical aspects. This has been recognized for at least six decades – indeed, it was a centerpiece of Albert Hirschman’s understanding of the development process – yet this critique, and the significance of its implications, remain on the margins of scholarship and policy. Why? I consider three options, concluding that, paradoxically, followers of Hirschman’s approach inadequately appreciated that gaining more operational traction for their approach was itself a type of problem requiring their ideas to embark on ‘a long voyage of discovery’, a task best accomplished, in this instance, by building – and tapping into the distinctive insights of – a diverse community of development practitioners.
Globalization has fed significant economic gains across the world. The gains lead some policymakers in developing countries to believe in the potential of ‘catch up’—where they leverage the gains of an open world economy to foster rapid progress and compete with more developed nations. This belief is particularly evident in countries like Rwanda, where policymakers aspire to turn the country into ‘Africa’s Singapore’. This paper asks if such aspiration is realistic: Do developing countries really gain enough from globalization to catch up to more developed countries? The paper examines the world economy as a league in which countries compete for winnings (manifest in higher income and production). Wealthier countries are in the top tiers of this league and poorer countries are in the lower tiers. The paper asks if gains from the last generation of growth have been distributed in such a way to foster ‘catch up’ by lower tier countries, and if we see these countries ‘catching up’ by moving into higher tiers. This analysis of the world economy is compared with a study of English football, where over 90 clubs play in an multi-tier league system. Prominent examples of ‘catch up’ in this system include Leicester City’s rise from the third tier in 2008 to become first tier champion in 2015. The paper asks if such ‘catch up’ is common in English football, given the way winnings are distributed, and if ‘catch up’ is more common in this context than in the world economy more generally.
Observers claim that public policies fail ‘often’. This paper asks, ‘how often’? It is an important question, because public policies absorb resources to address major social issues. We should know if policies are proving bad social investments; routinely failing to solve focal problems at high costs. Unfortunately, it is not easy to assess this. Many public policy organizations—governments in particular—do not provide accessible views onto overall success or failure. The World Bank does, however, provide such view—and it supports policy interventions one finds in governments across the world. The paper thus examines World Bank failure rates. It finds that there are different answers to the ‘how often’ question, depending on responses to a second question, ‘what is failure anyway?’ In studying both questions, the paper identifies a bias in the World Bank—and probably all organizations adopting rational ‘plan and control’ policy processes—to measuring ‘project and product success’ rather than a broader view of success as ‘problems are solved with development impact’. This means that policy organizations like the Bank judge success based on whether planned products are delivered through an efficient process; not whether policies solve the problems that warranted intervention in the first place, or whether the policies promoted development outcomes. Is this how citizens would want their public policy organizations to conceptualize success?
Governments across the world regularly pursue reforms that achieve less than was originally expected or is needed to make the state function better. The limits to reform success are often obvious in even the early days of reform, where gaps and weaknesses manifest. Many governments have no mechanisms built into their reform processes to see these gaps and weaknesses, however, and persist with predefined reform plans instead of adapting designs to close the gaps and address weaknesses. One antidote to this challenge is to create reflection points where reformers scrutinize their progress to identify weaknesses, reflect on these weaknesses, and adapt their next steps to address the weaknesses. In the spirit of John Kingdon’s work on ‘policy windows’, we call these reflection points ‘adaptation windows’—moments where reformers acknowledge problems in their reforms, adapt reforms to address such, and mobilize support for this adaptation. This paper discusses an effort to open an adaptation window for reformers to ‘see’ and then respond to public financial management (PFM) reform gaps and weaknesses in Mozambique. The paper details why and how this work was pursued, and also reflects on results of the government’s reflection at the adaptation window.
In rich and poor countries alike, a core challenge is building the state’s capability for policy implementation. Delivering high-quality public health and health care – affordably, reliably, at scale, for all – exemplifies this challenge, since doing so requires deftly integrating refined technical skills (surgery), broad logistics management (supply chains, facilities maintenance), adaptive problem solving (curative care) and resolving ideological differences (who pays? who provides?), even as the prevailing health problems themselves only become more diverse, complex and expensive as countries become more prosperous.
The current state of state capability in developing countries, however, is demonstrably alarming, with the strains and demands only likely to intensify in the coming decades. Prevailing ‘best practice’ strategies for building implementation capability – copying and scaling putative successes from abroad – are too often part of the problem, while individual training (‘capacity building’) and technological upgrades (e.g., new management information systems) remain necessary but deeply insufficient. An alternative approach is outlined, one centered on building implementation capability by working iteratively to solve problems nominated and prioritized by local actors.
A long-standing literature in the sociology of organizations (e.g., DiMaggio and Powell 1983) suggests that, as change agents face uncertainty about actions and outcomes, they often seek legitimacy through isomorphism: adopting structures, policies and reforms similar (at least in appearance) to those deemed successful elsewhere. We examine history’s most rapid reduction of fertility—from 8.4 in 1985 to 2.4 in 2002, in rural Iran—as an example of successful autonomous reform. The Iranian state, which was self-consciously cut off from nearly all of the traditional vectors of global isomorphism, initiated a successful behavioral change in a domain (family planning) perhaps unexpected for an Islamic state. We describe and explain the Iranian approach, in particular the rural program, contrasting it with the global strategy of adopting universal "best practices."
Many countries, like Sri Lanka, are trying to diversify their economies but often lack the
capabilities to lead diversification programs. One of these capabilities relates to preparing the investment climate in the country. Many governments tackle this issue by trying to improve their scores on ‘Doing Business Indicators’ which measure performance on general factors affecting business globally (like how long it takes to open a business or pay taxes). Beyond these common indicators, however, investors face context specific challenges when working in countries like Sri Lanka that are not addressed in global indicators. Governments often lack the capabilities to identify and resolve such issues. This paper narrates a recent initiative to establish these capabilities in Sri Lanka. The initiative adopted a Problem Driven Iterative Adaptation (PDIA) process, where a team of Sri Lankan officials worked with Harvard Center for International Development (CID) facilitators to build capabilities over a six-month period. The paper tells the story of this process, providing documented evidence of the progress over time (and describing thinking behind the PDIA process as well). The paper will be of interest to those thinking about the challenges associated with creating a climate that is investor or business friendly and to those interested in processes (like PDIA) focused on building state capability and fostering policy implementation.
Many countries, like Sri Lanka, are trying to diversify their economies but often lack the
capabilities to lead diversification programs. One of these capabilities relates to engaging new investors—in new sectors—to bring their FDI and know-how to a new country and kick-start new sources of activity. This paper narrates a recent (and ongoing) initiative to establish this kind of capability in Sri Lanka. The initiative adopted a Problem Driven Iterative Adaptation (PDIA) process, where a team of Sri Lankan officials worked with Harvard Center for International Development (CID) facilitators to build capabilities over a six-month period. The paper tells the story of this process, providing documented evidence of the progress over time (and describing thinking behind the PDIA process as well). It shows how an investment engagement approach can emerge in a reasonably limited period, when a committed team of public officials are effectively authorized and engaged. The paper will be of particular interest to those thinking about investor engagement challenges and to those interested in processes (like PDIA) focused on building state capability and fostering policy implementation in public contexts.
Many countries, like Sri Lanka, are trying to diversify their economies but often lack the capabilities to lead diversification programs. One of these capabilities relates to targeting new sectors to promote and pursue through a diversification policy: countries know they are ‘doomed to choose’ sectors to target,1 but lack effective capabilities to do the targeting. This paper narrates a recent (and ongoing) initiative to establish this kind of capability in Sri Lanka. The initiative adopted a Problem Driven Iterative Adaptation (PDIA) process, where a team of Sri Lankan officials worked with Harvard Center for International Development (CID) facilitators to build capabilities. The paper tells the story of this process, providing documented evidence of the progress over time and describing the thinking behind the PDIA process. It shows how a reliable targeting mechanism can emerge in a reasonably limited period, when a committed team of public officials are effectively authorized and engaged. The paper will be of particular interest to those thinking about targeting for diversification and to those interested in processes (like PDIA) which are focused on building state capability and fostering policy implementation in public contexts.
We divide the 102 historically developing countries (HDCs) into those with ‘very weak’, ‘weak’, ‘middle’, and ‘strong’ state capability. Analyzing the levels and recent growth rates of the HDCs’ capability for policy implementation reveals how pervasively “stuck” most of them are.
Only eight HDCs have attained strong capability, and since most of these are small (e.g., Singapore, UAE), less than 100 million (or 1.7%) of the roughly 5.8 billion people in HDCs currently live in high capability states.
Almost half (49) of these countries have very weak or weak capability, and thus their long-run pace of acquiring capability is also very slow.
Alarmingly, three quarters of these countries (36 of 49) have experienced negative growth in state capability in recent decades, while more than a third of all countries (36 of 102) have low and (in the medium run at least) deteriorating state capability.
At current rates, the ‘time to high capability’ of the 49 currently weak capability states and the 36 with negative growth is obviously “forever”. But even for the 13 with positive growth, only three would reach strong capability by the end of the 21st century at their current medium run growth.
Many development challenges are complex, involving a lot of different agents and with unknown dimensions. Solutions to these challenges are often unknown, and contextually dependent. At the same time, there are political imperatives at play in many contexts which create pressure to ‘find the solution now…and then scale it up.’ Such pressure raises a question: how does a policy entrepreneur or reformer find a new solution and scale it up when dealing with complexity? This is the subject we address in the current paper, which is the fifth in a series on ‘how to’ do problem driven iterative adaptation (PDIA) (Andrews et al. 2015, 2016a, 2016b, 2016c).
The paper focuses on building broad agency solutions in the process of identifying problems and finding and fitting contextually appropriate solutions. The broad agency is, in our opinion, a most effective mechanism to ensure scaling and dynamic sustainability in the change process. As with other working papers on this topic, the contents here do not offer all answers to those asking questions about how to do development effectively. It closes by reflecting on the importance of ‘you’ (the reader, and ostensibly part of a policy change or reform team somewhere) using this and the other ideas as heuristics to rethink and reorient how you work—but with your own signature on each idea.
Many of the challenges in international development are complex in nature. They involve many actors in uncertain contexts and with unclear solutions. Our work has proposed an approach to addressing such challenges, called Problem Driven Iterative Adaptation (PDIA). This paper is the most recent in a series intended to show how one can do PDIA, building on the first paper, "Doing Problem Driven Work.” The current paper addresses a key part of the approach one moves to once a problem has been identified, performing real-time experimental iterations. This is intended as a practical paper that builds on experience and embeds exercises for readers who are actually involved in this kind of work.
Muchos de los desafíos que enfrentamos en el desarrollo internacional son complejos por naturaleza. Involucran a muchos actores en contextos de incertidumbre y sin soluciones claras. Nuestro trabajo ha propuesto una aproximación a esos retos que se llama Adaptación Iterativa Orientada al problema (en inglés Problem Driven Iterative Adaptation, PDIA). Este artículo es el más reciente dentro de una serie que pretende mostrar cómo uno puede hacer PDIA, y que parte del primer artículo, “Trabajando con orientación al problema”. Este artículo se ocupa de una parte clave del enfoque que asumimos, que ocurre una vez hemos identificado el problema y empezamos a hacer iteraciones experimentales en tiempo real. Esperamos estar ofreciendo un artículo práctico que se apoya en la experiencia y que contiene ejercicios para lectores que ya están involucrados en este tipo de labor.