The enthusiasm for the potential of RCTs in development rests in part on the assumption that the use of the rigorous evidence that emerges from an RCT (or from a small set of studies identified as rigorous in a “systematic” review) leads to the adoption of more effective policies, programs or projects. However, the supposed benefits of using rigorous evidence for “evidence based” policy making depend critically on the extent to which there is external validity. If estimates of causal impact or treatment effects that have internal validity (are unbiased) in one context (where the relevant “context” could be country, region, implementing organization, complementary policies, initial conditions, etc.) cannot be applied to another context then applying evidence that is rigorous in one context may actually reduce predictive accuracy in other contexts relative to simple evidence from that context—even if that evidence is biased (Pritchett and Sandefur 2015). Using empirical estimates from a large number of developing countries of the difference in student learning in public and private schools (just as one potential policy application) I show that commonly made assumptions about external validity are, in the face of the actual observed heterogeneity across contexts, both logically incoherent and empirically unhelpful. Logically incoherent, in that it is impossible to reconcile general claims about external validity of rigorous estimates of causal impact and the heterogeneity of the raw facts about differentials. Empirically unhelpful in that using a single (or small set) of rigorous estimates to apply to all other actually leads to a larger root mean square error of prediction of the “true” causal impact across contexts than just using the estimates from non-experimental data from each country. In the data about private and public schools, under plausible assumptions, an exclusive reliance on the rigorous evidence has RMSE three times worse than using the biased OLS result from each context. In making policy decisions one needs to rely on an understanding of the relevant phenomena that encompasses all of the available evidence.
Core dual ideas of early development economics and practice were that (a) national development was a four-fold transformation of countries towards: (i) a more productive economy, (ii) a more responsive state, (iii) more capable administration, and (iv) a shared identity and equal treatment of citizens and (b) this four-fold transformation of national development would lead to higher levels of human wellbeing. The second idea is strikingly correct: development delivers. National development is empirically necessary for high wellbeing (no country with low levels of national development has high human wellbeing) and also empirically sufficient (no country with high national development has low levels of human wellbeing). Three measures of national development: productive economy, capable administration, and responsive state, explain (essentially) all of the observed variation in an omnibus indicator of wellbeing, the Social Progress Index, which is based on 58 distinct non-economic indicators. How national development delivers on wellbeing varies, in three ways. One, economic growth is much more important for achieving wellbeing at low versus high levels of income. Two, economic growth matters more for “basic needs” than for other dimensions of wellbeing (like social inclusiveness or environmental quality). Three, state capability matters more for wellbeing outcomes that depend on public production than on private goods (and for some wellbeing indicators, like physical safety, for which growth doesn’t matter at all). While these findings may seem too common sense to be worth a paper, national development--and particularly economic growth—is, strangely, under severe challenge as an important and legitimate objective of action within the development industry.
The ability to exercise leadership effectively requires skills and capacities that must be developed; they are not innate. Leadership is rarely making one decision and sticking to it, or making a grand development happen with a touch. Mostly, leadership is non-heroic and involves painstaking work, paying attention, and being able to learn quickly and in real time.
This paper re-examines why global collective action problems persist, and how to overcome them. Drawing on 140 interviews with campaigners, politicians, and businesses in 10 European countries, it suggests that many activists are stuck in a despondency trap. Never seeing radical reform, they lower their ambitions, and invest in more feasible but sub-optimal alternatives. This creates a negative feedback loop, in which the dearth of radical reform becomes self-fulfilling. But if reformists see advances at home and abroad, they may become more optimistic about collective mobilisation and break out of their despondency trap. This is shown by tracing the drivers of ground-breaking legislation. From 2018, large French firms must mitigate risks of environmental and human rights abuses in their global supply chains, or else be liable. This bill – the world’s first of its kind – was vociferously contested by businesses. But French campaigners and politicians persisted for four years, because they saw reasons for optimism. These include growing international support; public outcry; the French political culture (state intervention, and distrust of multinationals); together with a Centre-Left Government. Optimism galvanised relentless mobilisation. Legislative success in France then delivered a positive shock to activists across Europe, who were emboldened to launch similar campaigns and escape their despondency trap.
This is one of a series of working papers from “RISE"—the large-scale education systems research programme supported by the UK’s Department for International Development (DFID), Australia’s Department of Foreign Affairs and Trade (DFAT), and the Bill and Melinda Gates Foundation.
Support for gender equality has risen, globally. Analyses of this trend focus on individual and/or country-level characteristics. But this overlooks sub-national variation. Citydwellers are more likely to support gender equality in education, employment, leadership, and leisure. Why is this? This paper investigates the causes of rural-urban differences through comparative, qualitative research. It centres on Cambodia, where the growth of rural garment factories enables us to test theories that female employment fosters support for gender equality: potentially closing rural-urban differences; or whether other important aspects of city-living accelerate support for gender equality. Drawing on this rural and urban fieldwork, the paper suggests why social change is faster in Cambodian
cities. First, cities raise the opportunity costs of gender divisions of labour – given higher living costs and more economic opportunities for women. Second, cities increase exposure to alternatives. People living in more interconnected, heterogeneous, densely populated areas are more exposed to women demonstrating their equal competence in socially valued, masculine domains. Third, they have more avenues to collectively contest
established practices. Association and exposure reinforce growing flexibility in gender divisions of labour. By investigating the causes of subnational variation, this paper advances a new theory of growing support for gender equality.
Many development agencies and governments now seek to engage directly with local communities, whether as a means to the realization of more familiar goals (infrastructure, healthcare, education) or as an end in itself (promoting greater inclusion, participation, well-being). These same agencies and governments, however, are also under increasing pressure to formally demonstrate that their actions ‘work’ and achieve their goals within relatively short timeframes – expectations which are, for the most part, necessary and desirable. But adequately assessing ‘community-driven’ approaches to development requires the deployment of theory and methods that accommodate their distinctive characteristics: building bridges is a qualitatively different task to building the rule of law and empowering minorities. Moreover, the ‘lessons’ inferred from average treatment effects derived from even the most rigorous assessments of community-driven interventions are likely to translate poorly to different contexts and scales of operation. Some guidance for anticipating and managing these conundrums are provided.
This paper shows that countries may reduce labour repression if they perceive this as conducive to export growth. This paper traces what happened before, in the presence of, and then following the withdrawal of international economic incentives for pro-labour reforms in Vietnam and Bangladesh. The Government of Vietnam announced it would allow independent trade unions, in order to join the Trans-Pacific Partnership (TPP) and increase market access. Similarly, the Government of Bangladesh rescinded restrictions on unions, following global condemnation of Rana Plaza and fear of buyers leaving en masse. Both governments reduced labour repression to promote export growth. With high-level authorisation, Vietnamese and Bangladeshi activists and reformists became less fearful, and mobilised for substantive change. However, these economic incentives were short-lived: after Trump’s election, the USA withdrew from TPP; buyers continued to source from Bangladesh, and squeezed prices (without requiring labour reforms). Both governments then amped up labour repression - notwithstanding private regulation, economic upgrading, industry growth, and mass strikes.
Evaluations of development projects are conducted to assess their net effectiveness and, by extension, to guide decisions regarding the merits of scaling-up successful projects and/or replicating them elsewhere. The key characteristics of ‘complex’ interventions – numerous face- to-face interactions, high discretion, imposed obligations, pervasive unknowns – rarely fit neatly into standard evaluation protocols, requiring the deployment of a wider array of research methods, tools and theory. The careful use of such ‘mixed methods’ approaches is especially important for discerning the conditions under which ‘successful’ projects of all kinds might be expanded or adopted elsewhere. These claims, and the practical implications to which they give rise, draw on an array of recent evaluations in different sectors in development.
A defining task of development is enhancing a state’s capability for policy implementation. In most low- income countries, alas, such capabilities seem to be stagnant or declining, in no small part because dominant reform strategies are ill-suited to addressing complex non-technical aspects. This has been recognized for at least six decades – indeed, it was a centerpiece of Albert Hirschman’s understanding of the development process – yet this critique, and the significance of its implications, remain on the margins of scholarship and policy. Why? I consider three options, concluding that, paradoxically, followers of Hirschman’s approach inadequately appreciated that gaining more operational traction for their approach was itself a type of problem requiring their ideas to embark on ‘a long voyage of discovery’, a task best accomplished, in this instance, by building – and tapping into the distinctive insights of – a diverse community of development practitioners.
Globalization has fed significant economic gains across the world. The gains lead some policymakers in developing countries to believe in the potential of ‘catch up’—where they leverage the gains of an open world economy to foster rapid progress and compete with more developed nations. This belief is particularly evident in countries like Rwanda, where policymakers aspire to turn the country into ‘Africa’s Singapore’. This paper asks if such aspiration is realistic: Do developing countries really gain enough from globalization to catch up to more developed countries? The paper examines the world economy as a league in which countries compete for winnings (manifest in higher income and production). Wealthier countries are in the top tiers of this league and poorer countries are in the lower tiers. The paper asks if gains from the last generation of growth have been distributed in such a way to foster ‘catch up’ by lower tier countries, and if we see these countries ‘catching up’ by moving into higher tiers. This analysis of the world economy is compared with a study of English football, where over 90 clubs play in an multi-tier league system. Prominent examples of ‘catch up’ in this system include Leicester City’s rise from the third tier in 2008 to become first tier champion in 2015. The paper asks if such ‘catch up’ is common in English football, given the way winnings are distributed, and if ‘catch up’ is more common in this context than in the world economy more generally.
Observers claim that public policies fail ‘often’. This paper asks, ‘how often’? It is an important question, because public policies absorb resources to address major social issues. We should know if policies are proving bad social investments; routinely failing to solve focal problems at high costs. Unfortunately, it is not easy to assess this. Many public policy organizations—governments in particular—do not provide accessible views onto overall success or failure. The World Bank does, however, provide such view—and it supports policy interventions one finds in governments across the world. The paper thus examines World Bank failure rates. It finds that there are different answers to the ‘how often’ question, depending on responses to a second question, ‘what is failure anyway?’ In studying both questions, the paper identifies a bias in the World Bank—and probably all organizations adopting rational ‘plan and control’ policy processes—to measuring ‘project and product success’ rather than a broader view of success as ‘problems are solved with development impact’. This means that policy organizations like the Bank judge success based on whether planned products are delivered through an efficient process; not whether policies solve the problems that warranted intervention in the first place, or whether the policies promoted development outcomes. Is this how citizens would want their public policy organizations to conceptualize success?
Governments across the world regularly pursue reforms that achieve less than was originally expected or is needed to make the state function better. The limits to reform success are often obvious in even the early days of reform, where gaps and weaknesses manifest. Many governments have no mechanisms built into their reform processes to see these gaps and weaknesses, however, and persist with predefined reform plans instead of adapting designs to close the gaps and address weaknesses. One antidote to this challenge is to create reflection points where reformers scrutinize their progress to identify weaknesses, reflect on these weaknesses, and adapt their next steps to address the weaknesses. In the spirit of John Kingdon’s work on ‘policy windows’, we call these reflection points ‘adaptation windows’—moments where reformers acknowledge problems in their reforms, adapt reforms to address such, and mobilize support for this adaptation. This paper discusses an effort to open an adaptation window for reformers to ‘see’ and then respond to public financial management (PFM) reform gaps and weaknesses in Mozambique. The paper details why and how this work was pursued, and also reflects on results of the government’s reflection at the adaptation window.
In rich and poor countries alike, a core challenge is building the state’s capability for policy implementation. Delivering high-quality public health and health care – affordably, reliably, at scale, for all – exemplifies this challenge, since doing so requires deftly integrating refined technical skills (surgery), broad logistics management (supply chains, facilities maintenance), adaptive problem solving (curative care) and resolving ideological differences (who pays? who provides?), even as the prevailing health problems themselves only become more diverse, complex and expensive as countries become more prosperous.
The current state of state capability in developing countries, however, is demonstrably alarming, with the strains and demands only likely to intensify in the coming decades. Prevailing ‘best practice’ strategies for building implementation capability – copying and scaling putative successes from abroad – are too often part of the problem, while individual training (‘capacity building’) and technological upgrades (e.g., new management information systems) remain necessary but deeply insufficient. An alternative approach is outlined, one centered on building implementation capability by working iteratively to solve problems nominated and prioritized by local actors.
A long-standing literature in the sociology of organizations (e.g., DiMaggio and Powell 1983) suggests that, as change agents face uncertainty about actions and outcomes, they often seek legitimacy through isomorphism: adopting structures, policies and reforms similar (at least in appearance) to those deemed successful elsewhere. We examine history’s most rapid reduction of fertility—from 8.4 in 1985 to 2.4 in 2002, in rural Iran—as an example of successful autonomous reform. The Iranian state, which was self-consciously cut off from nearly all of the traditional vectors of global isomorphism, initiated a successful behavioral change in a domain (family planning) perhaps unexpected for an Islamic state. We describe and explain the Iranian approach, in particular the rural program, contrasting it with the global strategy of adopting universal "best practices."
Many countries, like Sri Lanka, are trying to diversify their economies but often lack the
capabilities to lead diversification programs. One of these capabilities relates to preparing the investment climate in the country. Many governments tackle this issue by trying to improve their scores on ‘Doing Business Indicators’ which measure performance on general factors affecting business globally (like how long it takes to open a business or pay taxes). Beyond these common indicators, however, investors face context specific challenges when working in countries like Sri Lanka that are not addressed in global indicators. Governments often lack the capabilities to identify and resolve such issues. This paper narrates a recent initiative to establish these capabilities in Sri Lanka. The initiative adopted a Problem Driven Iterative Adaptation (PDIA) process, where a team of Sri Lankan officials worked with Harvard Center for International Development (CID) facilitators to build capabilities over a six-month period. The paper tells the story of this process, providing documented evidence of the progress over time (and describing thinking behind the PDIA process as well). The paper will be of interest to those thinking about the challenges associated with creating a climate that is investor or business friendly and to those interested in processes (like PDIA) focused on building state capability and fostering policy implementation.
Many countries, like Sri Lanka, are trying to diversify their economies but often lack the
capabilities to lead diversification programs. One of these capabilities relates to engaging new investors—in new sectors—to bring their FDI and know-how to a new country and kick-start new sources of activity. This paper narrates a recent (and ongoing) initiative to establish this kind of capability in Sri Lanka. The initiative adopted a Problem Driven Iterative Adaptation (PDIA) process, where a team of Sri Lankan officials worked with Harvard Center for International Development (CID) facilitators to build capabilities over a six-month period. The paper tells the story of this process, providing documented evidence of the progress over time (and describing thinking behind the PDIA process as well). It shows how an investment engagement approach can emerge in a reasonably limited period, when a committed team of public officials are effectively authorized and engaged. The paper will be of particular interest to those thinking about investor engagement challenges and to those interested in processes (like PDIA) focused on building state capability and fostering policy implementation in public contexts.
Many countries, like Sri Lanka, are trying to diversify their economies but often lack the capabilities to lead diversification programs. One of these capabilities relates to targeting new sectors to promote and pursue through a diversification policy: countries know they are ‘doomed to choose’ sectors to target,1 but lack effective capabilities to do the targeting. This paper narrates a recent (and ongoing) initiative to establish this kind of capability in Sri Lanka. The initiative adopted a Problem Driven Iterative Adaptation (PDIA) process, where a team of Sri Lankan officials worked with Harvard Center for International Development (CID) facilitators to build capabilities. The paper tells the story of this process, providing documented evidence of the progress over time and describing the thinking behind the PDIA process. It shows how a reliable targeting mechanism can emerge in a reasonably limited period, when a committed team of public officials are effectively authorized and engaged. The paper will be of particular interest to those thinking about targeting for diversification and to those interested in processes (like PDIA) which are focused on building state capability and fostering policy implementation in public contexts.
Previous papers such as Russell, Barrios & Andrews (2016), Guerra (2016), and Russell, Tokman, Barrios & Andrews (2016) have aimed to provide an empirical view into the sports economy. This proves to be a difficult task, given the many definitions of ‘sports’ and data deficiencies and differences in the sports domain (between contexts and over time). The emerging view in these previous papers provides interesting information about the sports sector, however: it shows, for instance, that different contexts have differently intensive sports sectors, and that sports activities overlap with other parts of the economy. This kind of information is useful for policymakers in governments trying to promote sports activities and use sports to advance the cause of broad-based social and economic development.
This paper is written with these policymakers in mind. It intends to offer a guide such agents can use in constructing sports policies focused on achieving development goals (what we call development through sports), and discusses ways in which these policymakers can employ empirical evidence to inform such policies.
The paper draws on the concept of ‘governance’ to structure its discussion. Taking a principal-agent approach to the topic, governance is used here to refer to the exercise of authority, by one set of agents, on behalf of another set of agents, to achieve specific objectives. Building on such a definition, the paper looks at the way governmental bodies engage in sports when acting to further the interests of citizens, most notably using political and executive authority to promote social and economic development. This focus on governance for development through sports (asking why and how governments use authority to promote sports for broader social and economic development objectives) is different from governance of sports (which focuses on how governments and other bodies exercise authority to control and manage sports activities themselves), which others explore in detail but we will not discuss.
The paper has five main sections. A first section defines what we mean by ‘governance’ in the context of this study. It describes an ends-means approach to the topic—where we emphasize understanding the goals of governance policy (or governance ends) and then thinking about the ways governments try to achieve such goals (the governance means). The discussion concludes by asking what the governance ends and means are in a development through sports agenda. The question is expanded to ask whether one can use empirical evidence to reflect on such ends and means. One sees this, for instance, in the use of ‘governance indicators’ and ‘governance dashboards’ in the international development domain. A second section details the research method we used to address these questions. This mixed method approach started by building case studies of sports policy interventions in various national and sub-national governments to obtain a perspective on what these policies tend to involve (across space and time). It then expanded into an analysis of sports policies in a broad set of national and sub-national governments to identify common development through sport ends and means. Finally, it involved experimentation with selected data sources to show how the ends and means might be presented in indicators and dashboards—to offer evidence-based windows into development through sports policy regimes.
Based on this research, sections three and four discuss the governance ends and means commonly pursued and employed by governments in this kind of policy process. The sections identify three common ends (or goals)—inclusion, economic growth, and health—and a host of common means—like the provision of sports facilities, organized activities, training support, financial incentives, and more—used in fostering a development through sports agenda. Data are used from local authorities in England to show the difficulties of building indicators reflecting such policy agendas, but also to illustrate the potential value of evidence-based dashboards of these policy regimes. It needs to be stated that this work is more descriptive than analytical, showing how data can be used to provide an evidence-based perspective on this domain rather than formally testing hypotheses about the relationship between specific policy means and ends. In this regard, the work is more indicative of potential applications rather than prescriptive. A conclusion summarizes the discussion and presents a model for a potential dashboard of governance in a development through sports policy agenda.
 This terminology comes from Houlihan and White, who identify the “tension between development through sport (with the emphasis on social objectives and sport as a tool for human development) and development of sport (where sport was valued for its own sake)” (Houlihan & White 2002, 4).
 The paper relates to a vibrant literature on this topic, which investigates the reasons and ways governments support the sports sector (classic and recent studies in this literature include Adams and Harris (2014), Gerretsenand Rosentraub (2015), Grix and Carmichael (2012), Grix (2015), Hallman and Petry (2013), Houlihan (2002, 2005, 2016), Houlihan and White (2002), Hylton (2013), Koski and Lämsä (2015), Schulenkorf and Adair (2013), and Vuori et al. (1995).
 Work on the governance of sports assesses the way international entities like FIFA and the IOC work with national and local governmental bodies to oversee, regulate, and otherwise manage sports like football and the Olympic movement, using authority to create and implement rules on behalf of those involved in the sport itself. See, for instance Forster (2006), Geeraert (2013), and Misener (2014).
Men’s professional football is the biggest sport in the world, producing (by our estimate) US $33 billion a year. All is not well in the sector, however, with regular scandals raising questions about the role of money in the sport. The 2015 turmoil around FIFA is obviously the most well known example, creating a crisis in confidence in the sector. This study examines these questions, and the financial integrity weaknesses they reveal; it also offers ideas to strengthen the weaknesses.
The study argues that football’s financial integrity weaknesses extend far beyond FIFA. These weaknesses have emerged largely because the sector is dominated by a small elite of clubs, players and owners centered in Europe’s top leagues. The thousands of clubs beyond this elite have very little resources, constituting a vast base of ‘have-nots’ in football’s financial pyramid. This pyramid developed in recent decades, fuelled by concentrated growth in new revenue sources (like sponsorships, and broadcasting). The growth has also led to increasingly complex transactions—in player transfers, club ownership and financing (and more)—and an expansion in opportunities for illicit practices like match-fixing, money laundering and human trafficking. We argue that football’s governing bodies – including FIFA – helped establish this pyramid.
We explore the structural weaknesses of this pyramid by looking at five pillars of financial integrity (using data drawn from UEFA, FIFA, clubs, primary research, and interviews).
In the first pillar of Financial Transparency and Literacy we find that a vast majority of the world’s clubs and governing bodies publish no financial data, leaving a vast dark space with no transparency.
In the second pillar of Financial Sustainability we estimate that a majority of global clubs and governing bodies are at ‘medium to high risk’ of financial failure.
We find, additionally, that European tax debts have grown despite Financial Fair Play, and confederations and FIFA contribute to a pattern of weak Fiscal Responsibility.
We then create a new metric of Financial Concentration and find that the football sector is at ‘high risk’ of over-concentration, which poses existential questions for many clubs and even leagues.
In the final pillar of Social Responsibility and Moral Reputation, we find that football’s governing bodies face a crisis of legitimacy stemming from a failure to tackle moral turpitude, set standards and regulate effectively. We suggest a set of reforms to re-structure FIFA in particular, separating its functions and stressing its regulatory role.