The Limits of Institutional Reform in Development

Focusing governance indicators on capability gaps, continued

In my post a couple of days ago I gave some ideas of governance indicators that capture critical gaps in state capability in developing countries. These included road deaths, birth and civil registration. I like these indicators. If governments in low income countries can improve their performance on these indicators then I am certain we will see improved governance, which I define as 'the exercise of authority by governments on behalf of citizens.'

I want to offer some comments on the last post and this post before continuing, in answer to those who have asked me why I am engaging in this conversation at all:

• First, I did not initially want to engage in this conversation: I am not an avid supporter of including governance indicators in the post-2015 goals. I still believe the idea has not been treated seriously enough to warrant the attention it is getting (see my posts that question how much we know about what governance is, etc. http://matthewandrews.typepad.com/mattandrews/2011/09/making-good-good-governance-indicators-the-construct-validity-question-1.html; http://matthewandrews.typepad.com/mattandrews/2013/01/uncomfortable-thoughts-on-governance-can-we-ever-say-it-is-good-or-bad.html). Let me go further and say I also question the value of MDGs as a whole. I have yet to see a thorough analysis of their influence and I think there are many unanswered questions about this (see http://matthewandrews.typepad.com/mattandrews/2013/05/what-is-the-story-with-mdg-performance.html; http://matthewandrews.typepad.com/mattandrews/2013/05/fousing-on-growth-versus-creating-post-2015-mdgs.html). And I would describe myself as a critic of the post 2015 'goal defining' process. It seems to be incredibly expensive, centered on ensuring the importance of international organizations, and full of consultations that do not/will not ask tough questions about the effectiveness of past MDGs and about how much we really know about the world, issues like governance, etc.(see posts http://matthewandrews.typepad.com/mattandrews/2013/02/post-2015-mdgs-should-be-shaped-around-country-specific-problems-not-global-agendas.html; http://matthewandrews.typepad.com/mattandrews/2013/11/the-cost-of-post-2015-indicator-consultations-money-well-spentor-not.html; http://matthewandrews.typepad.com/mattandrews/2013/11/the-delusion-or-deception-of-mdgs-and-measurement.html).
• But I am convinced that there will be post 2015 indicators, and that there will be a post 2015 governance indicator. The list of people who have convinced me of this is extremely long. I believe it.

  Given this, I am worried that the post 2015 governance indicator will be a bad one. Many 'indicators of governance' reflect highly normative prescriptions of what good governance 'should be' and lack theoretical or empirical foundations about what governance is or how it matters, across highly different contexts (http://www.tandfonline.com/doi/pdf/10.1080/13600810802455120).I see lists of ideas referring to 'transparency' and 'rule of law' and 'openness' and other concepts that are often not well considered and seem to me to reflect characteristics experts in advanced Anglo or Nordic countries might wish on governments in developing countries. I question whether many of the concepts have any relevance in developing countries and believe that they are actually detrimental (in my book I argue that these kinds of indicators foster 'signaling' where countries do reforms that make them look better but don't make them any better).

• So, I am now trying to be constructive, and suggesting a course of action to identify indicators of governance that I think could actually make the world better. This does not mean that they cannot be gamed. All indicators can be gamed and, as Philipp Krause notes, the political pressure to game governance indicators might be higher than for other indicators. I expect this to be the case (my entire argument about signaling and the application of isomorphic mimicry makes this point). I also expect unintended consequences with all governance indicators. We are talking about governance here, not pick-up basketball. Governance is about authority, power, and politics. It I about real, big-time winners and losers. So governance reforms (which indicators are intended to inform and motivate) are about the redistribution of authority and power. All indicators and reforms in this area will have dark sides to them, whether one is talking about road deaths or population registers or improved transparency or whatever. It is crucial that those discussing indicators accept this, and agree to the risks of what they are doing. If we were doctors and had to make the oath they do, would we be constructing these indicators?
• These are IDEAS, not final and formal proposals… I hope that they promote discussion, but they are not intended as hard and fast recommendations for any of the high-level, low-level, mid-level or any-level committees working around the world. This is not my first post on the topic so I do hope the others will also get some attention (http://matthewandrews.typepad.com/mattandrews/2012/10/a-systematic-approach-to-governance-indicators.html). As always, however, I am aware that some ideas are better than others...and all ideas need to be plugged in to generate any real energy.

  

Now, let me get back to current ideas I have for indicators.

Those I suggested on Monday are what I would call 'state capability indicators' that reflect on basic capabilities governments need to even pretend that they are 'exercising authority' in meaningful ways.

As my silly drawing shows, these capabilities are like the bridge governments need to exercise authority in effective and responsible ways. As readers of my work would know (especially if you have read the 'capability trap stuff' coauthored with Lant Pritchett and Michael Woolcock), I believe that we often find developing country governments facing these capability limits. The problem is often exacerbated by many governance reform agendas (and governance indicators informing such) prescribing solutions that do not focus on building missing capabilities but actually put existing capabilities under extreme pressure.

We refer to this as 'premature load bearing'; where governance agendas introduce new 'solutions' that are way beyond the capabilities of developing country contexts…and ensure that governments get stuck in a trap of continual reform and reform failure. With this in mind, I would not suggest that any indicators prescribe ways of dealing with the missing capabilities in road safety, child registration or other areas (these are the ones I discussed on Monday. There are more that could be identified). The goal of the indicator is to get governments building the capabilities that allow the passage of heavier loads (problems, challenges, etc) so that they can more effectively and responsibly exercise the authority they enjoy.

Today I would like to suggest other indicators of critical 'administrative gaps' that I feel compromise governance capabilities. I believe that closing these gaps will lead to better governance in countries, and I believe that developing countries have bigger gaps than the gaps in developed countries.

• The first gap relates to public finance. All governments have the authority to raise and spend money. This authority is given to them so that they can do the business of the people, financing policies that produce public goods etc. Now these policy choices are really different in different countries and the processes of determining these choices also differs. I have my own personal views on which kinds of policy choices are better and which kinds of policy processes are better, but I think we move into an area of contestable norms when discussing such things. However, I think there are some ideas about governance in this discussion that evoke pretty global norms and reflect practical and pragmatic realities of public administration:
  • Reliable budgets are 'good'. I believe it is pretty uncontroversial to say it is better if public organizations behave in ways they commit to behave. In a public finance setting this means that it is generally good if governments spend money in accordance with the budgets they set. Now this does not mean that actual spending should not be allowed to depart in some way from proposed budgetary targets, and it should not be read to say that there are never times where even drastic reallocations of funds make sense and are actually good practice. But in many developing countries the gap between proposed and executed budgets is large and persistent and undermines governance. If the gap were closed it would mean that governments are planning better, budgeting better, and executing in accordance with budgets better. All good things. This kind of indicator would be interesting to construct: I would look at it at three levels; 1. The aggregate gap of spending across government (is government aggregate expenditure more closely aligned with budgeted amounts than it was before?); 2. The gap between allocations to functional or organizational recipients in the budget (is government doing a better job ensuring that it actually allocates money across government in accordance with the budgetary promises, such that education/health/agriculture gets what it asked for and we don't see money routinely moved away to defense/presidential funds, etc.). 3. The gap between when money should be provided to spending entities and when it actually gets released (are finance agencies in government doing a better job of actually getting money to the service delivery agencies when they need the money to do the jobs and provide the services committed to people in the budget and associated plans?). PEFA indicators (PI1 and PI2) could provide interesting starting points for the first two aspects, but we would need to think carefully about the third one in most cases. We would also need to think of the time horizons for different parts of this indicator. Given how hard it might be to construct, I would suggest considering country specific indicators (where countries decide this is a major gap they face, the countries work with partners to define exactly how they will construct indicators, set baselines and define targets…the goal is not international comparison but rather domestic improvement).
  • Living up to commitments is 'good'. I believe it is pretty uncontroversial to say it is better if public organizations live up to their commitments. In public finance this means paying bills, to external creditors and internal staff. In many developing countries this is a major gap. External debt is not always a distinct problem but there are common concerns about payments of contracts and salaries and wages. These are often not paid on time, leading to penalties or work stoppages (in contracting) or strikes and morale problems (with staffing). It seems to me that governments are authorized to act on behalf of citizens when making financial commitments, and governments will be exercising this authority well when actually making good on these commitments. This kind of indicator would be interesting and difficult to construct: I would focus on debt, contracting and salary/wage payments of the state, as well as data on arrears. There are few countries in which one could find data on most of these commitments or on the payment of bills. As with the indicator on budget reliability, therefore, one would need to construct this kind of indicator from the ground up in many cases. I would argue that governance would be improved if this kind of indicator were constructed, and rudimentary systems were introduced to measure 'Bill paying'. A tough task technically and politically (but this is what post-2015 indicators are about if I am to believe the MDG folks when they say that their indicators help push tough agendas forward and that they want real ideas on how to improve the way governments exercise their authority—or govern).
  • I could think of other such indicators centered on other public finance issues (for instance, the 'procurement value' (are goods and services procured at the right times, with appropriate quality, and within budgets), and 'debt efficiency' (is debt accessed in a strategic and cost efficient manner?), and 'macroeconomic stability' (are public finances being managed in a manner that best fosters stability). These kinds of indicators reflect on whether governments are closing administrative gaps needed to effectively exercise their authority. I would focus on outputs and outcomes (as above) and not processes – understanding that the process solutions at the disposal of different governments are different. But the gaps can be closed by all.
• The second gap relates to what I might call 'policy legitimacy'. All governments need to have some policy legitimacy to govern, conferred by citizens or elites like the private sector. Note I am not talking here about democratic legitimacy (which is also important in my normative view but is not the topic here). If government policies are not reliable they will not enjoy legitimacy and one can expect that they will struggle to use the authority they have to do things only they can do. For instance, governments that introduce new laws and then do not implement them will struggle to exercise their authority in the legal domain. Similarly, governments that deregulate certain sectors in a formal sense but then impose red tape through other administrative barriers will not be considered legitimate. In considering such issues, I think it can be generally said that legitimacy results when governments do what they say they will do. It is better, then, to have a weaker law that you can actually implement—or a higher regulatory burden that you actually adhere to—than great laws or minimalist regulations that you don't adhere to. Here are some ideas for indicators in this area:
  • Being equally transparent about spending and spending promises is 'good'. The government budget is one of the major policy tools it has. It is the major tool that governments use on a repeated basis (every year) to communicate priorities, goals, and strategies. The transparency agenda has helped promote the idea that governments should make these 'policy statements' openly accessible to citizens, elites, etc. Indicators like the Open Budget Index now reflect on how open governments are about this. This is a good thing. But in many developing countries there is a gap between openness in the formulation of budgets (where they make their promises) and in the execution of budgets (what they actually do). (see my prior blogs on this, including the most recent where you will see some back and forth with Paolo de Renzio at IBP http://matthewandrews.typepad.com/mattandrews/2013/05/good-or-good-looking-governance-that-is-the-question.html;http://matthewandrews.typepad.com/the_limits_of_institution/2013/10/how-transparent-are-open-budgets.html). This is not so good. It suggests that governments don't realize the legitimacy they lose by making promises in a transparent way and then limiting information on how well they performed on these promises. Closing the gap could be a critical governance goal for many countries, and an indicator of such gap could be constructed using data from organizations like OBI. In some countries the strategy for closing the gap could center on being more transparent with execution; In other countries there may be a strategy that involves (at least in the short run) becoming a little less transparent in budget formulation while execution transparency is caught up. I imagine observers will not be terribly happy with the latter cases, because there is normative view that transparency is always good. I don't think this is true and also don't think it is realistic. Most countries find it easier to open up on their promises than their execution and the transparency gaps are high and growing and suggest a false level of transparency. Let's get real by closing these gaps.
  • Actually implementing the anticorruption laws a country has is 'good'. A similar gap exists in most developing countries when one looks at the quality of laws addressing corruption and the actual implementation of these laws. (See http://matthewandrews.typepad.com/mattandrews/2013/05/good-or-good-looking-governance-that-is-the-question.html).Many governments in developing countries have spent many years passing fabulous laws in this area and then do not implement the laws. This is surely not a good thing for what some might call 'the effective rule of law' (which surely means little more than 'governments implement the laws that exist' or 'citizens and other entities expect that existing laws will be executed'). I often like to point out that Uganda has the best anticorruption laws in the world but one of the biggest gaps (measured at about 50 by Global Integrity, with laws at about 100 out of 100 and implementation of these laws at about 50). Italy, by contrast, has a gap of something like 10 (laws are about 80 and implementation of laws are about 70). From a governance perspective we must surely agree that Italy is 'better governed'—not because it has better laws (which it does not) but because it actually implements its laws. Uganda's efforts to create fantastic laws (often as signals to donors, who readily accepted them, in my opinion) have actually undermined governance because they have undermined policy legitimacy: The government has introduced policies it cannot implement, which has exacerbated a gap that surely weakens its execution of authority. Closing the gap could be a critical governance goal for many countries, and an indicator of such gap could be constructed using data from organizations like Global Integrity. As above: In some countries the strategy for closing the gap could center on just implementing laws that are on the books; In other countries there may be a strategy that involves (at least in the short run) actually pulling back on some of the laws. I imagine observers will not be terribly happy with the latter cases, because there is normative view that better laws are always better (and much of the anticorruption agenda centers on introducing laws). I don't think this is true and also don't think it is realistic. Most countries find it easier to pass new laws than they do to implement the laws the implementation gaps are high and growing and suggest a false anticorruption agenda. Let's get real by closing these gaps.
  • Implementing regulations as they are written is 'good'. Let me propose one last policy legitimacy gap indicator here today. It reflects on the authority governments have to regulate economic behavior. The Doing Business indicators assess such regulations, and impose the normative view that 'less regulation is good'. This is a view that has led many developing countries to decrease the formal regulatory burdens facing business. These countries have changed regulations and streamlined formal procedures related to getting permits, paying taxes, and a range of other things. In most places the measure of success in this kind of reform is whether 'according to regulations' you can get through government processes in fewer days than before. So, countries declare success when (according to regulations) firms can get a construction permit in fewer days than they could before (as an example). But there is a problem. The Doing Business indicators only ask about written regulations, not actual practice. And in many countries Business Enterprise surveys tell us that there are huge gaps between what formal regulations say and what actual firms go through. Where regulations decrease the number of day it should take to get a construction permit from 30 to 5, for instance, one often finds that business enterprise surveys find many companies taking over 100 days to get the permits. This gap again undermines the legitimacy of the state and its policy goals: private organizations learn to discount what the government promises in such situations and all sorts of gaming ensues that limit the exercise of authority (governance). Closing the gap could be a critical governance goal for many countries, and an indicator of such gap could be constructed using data from surveys like Doing Business and Business Enterprise. The idea of such an indicator would be to calculate the gap between formal regulatory promise and actual administrative behavior. As above: In some countries the strategy for closing the gap could center on just implementing regulations that are on the books; In other countries there may be a strategy that involves (at least in the short run) actually re-regulating some sectors (or passing laws that create realistic expectations about how long it will take to do certain things, what kinds of loopholes exist, etc.). I imagine observers will not be terribly happy with the latter cases, because there is normative view that lower regulations and promises of lower regulations are always better (and much of the private sector development agenda is focused on this). As above: I don't think this is true and also don't think it is realistic. Most countries find it easier to change regulations to commit to 'more efficient administration' than they do to actually become more efficient and end up with serious gaps between what they promise and what they do. Let's get real by closing these gaps.
  • I'm sure I can think of many other indicators that relate to gaps like these—where governance will be enhanced if states do what they say in a more effective manner. It should be noted that the idea of focusing on these gaps is a dramatic shift away from current governance indicator proposals. These proposals often emphasize the one side of the gap—I often hear advocates speaking of 'making budgets more transparent' for instance or 'introducing anticorruption mechanisms like whistleblower laws' or 'deregulating for small business creation'. In my opinion many of these recommendations focus purely on the form of governance and promise to exacerbate gaps, reduce policy legitimacy and undermine governance (as in the silly drawing alongside). The proposal I have is to get away from this form based agenda and focus on the problem of becoming more functional—which involves discussing what forms are relevant and how to implement and execute on these forms. In my book I call this getting away from the 'what you see is not what you get problem in development'.

I will continue with more ideas in the weeks to come, but let me take a break now and wait to see what you all think of these ideas. In the meantime, for those who have asked: My book is now available in paperback at Amazon and Cambridge University Press.

Explaining POSITIVE reform experiences

This week we held an interesting and inspiring workshop on various aspects of governance. At least it's inspiring if you're a governance nerd, I guess. Bob Klitgaard had some interesting ideas and admonition a when talking of new ways to do reform, and about how evaluation could be useful for reformers. One comment that hit home was simply "give them positive examples... Where you can show your ideas at work." His challenge to me was "I am a fan of PDIA but give the evidence and the stories..."
So this week we (at the Building State Capability Program at Harvard) released a paper showing that elements of PDIA show up in most stories of successful reform told by the Innovations for Successful Society folks (at Princeton). See http://www.hks.harvard.edu/var/ezp_site/storage/fckeditor/file/pdfs/centers-programs/centers/cid/publications/faculty/wp/267_Andrews_Explaining%20positive%20deviance.pdf
Next...I will be drawing on these cases to show that real success is possible and to illustrate how pdia worked in these cases. I'm excited.

PDIA and the epidemic of cheating

The following is an article that suggests PDIA offers an appropriate, roll up your sleeves and get to work approach to solving complex problems like cheating... In the Indian Premier League cricket context and beyond. It first appeared in Live Mint.com

http://www.livemint.com/Opinion/3ADaM7VyCYwzHwLCRcBuaM/An-epidemic-of-cheating.html?facet=print)

Sun, Jun 02 2013. 08 28 PM IST

An epidemic of cheating

In a winner-take-almost-everything society, there is a powerful incentive to cross a few lines to get ahead

A file photo of an IPL match. The most infamous of the recent cases of cheating and dishonesty is the one related to the Indian Premier League (IPL). Photo: Virendra Singh Gosain/Hindustan Times

A pandemic of cheating appears to be upon us.

The last few months have been depressing even for the optimists among us. Cases of cheating and dishonesty are everywhere you turn. I define cheating as acts by otherwise normal individuals or institutions that cut corners or break rules to get ahead. The most infamous of these is the one related to the Indian Premier League (IPL). The multifaceted nature of the allegations boggles the mind—betting, spot fixing, match fixing, insider trading—juvenile Srees and greedy Appans are popping up everywhere. Not to be outdone, occupants of Main Street are contributing their own stories. A whistle-blower at Ranbaxy revealed a long-term fraud that resulted in a $500 million fine the company paid to the US Food and Drug Administration (FDA). That sordid story is laid out in a Fortune article, appropriately titled “Dirty Medicine”. I am willing to bet (figuratively, of course) that your disgust for the cheating will be accompanied by a grudging admiration for the audacity and ingenuity of the methods employed. In one instance, the article talks about how company officials aged an operations manual to cheat inspectors by keeping it in a steam room overnight.

To be fair, the virus appears to have afflicted many other countries as well. Previously revered companies such as Apple in the US have been charged with shifting revenues to tax havens. Senator Carl Levin, a Michigan democrat, summed it up thus: “Apple sought the holy grail of tax avoidance. It has created offshore entities holding tens of billions of dollars, while claiming to be tax resident nowhere.” Apple has not been accused of breaking any laws. European authorities have similarly charged other companies such Google, Starbucks and Amazon. Apple Operations International, its primary offshore holding company registered in Ireland, has paid no taxes on $30 billion of net income over the last three years. That’s a dream device called i-no tax.

The problems at IPL began with a birth defect. The constitution of the Board of Control for Cricket in India (BCCI) was amended to permit an honorary member to bid for commercial interest. So-called independent members sat quietly while changes to the constitution consolidated power in the president of BCCI. Like the nowhere tax jurisdiction of Apple International, BCCI is governed by no one—not the government, not by shareholders, and not by the public. The only authority it accepts, its own constitution, was hijacked by its president because of the passivity of its members. For any president of BCCI to be removed, a two-third majority in a secret ballot is required. The late Hugo Chavez of Venezuela must be doffing a hat in his grave at the masterful use of a constitutional mechanism to protect authoritarian rule.

Is there really a problem? If so, what can be done? Is the cheating pandemic merely coming to light because we are seeing more cases being reported? Or, is there more cheating than ever before? Is it because of the large magnitudes that we are taking note now?

While it is difficult to know precisely, there is probably some truth in each of these observations. In the IPL case, terrorism-related wiretaps, security cameras, and mobile phone records played a role in the original investigation and continue to play a role in the prosecution strategy. There is qualitative evidence to believe that cheating has indeed increased.

The gains from getting to the top of your profession, of boosting your net income with tax evasion and of betting with loaded dice are enormous relative to the dangers of getting caught. In a winner-take-almost-everything society, there is a powerful incentive to cross a few lines to get ahead.

Beyond the talking and the writing and the screaming on TV, can anything at all be done? In a paper entitled Escaping Capability Traps Through Problem-Driven Iterative Adaptation (PDIA), professors Matt Andrews, Lant Pritchett and Michael Woolcock of Harvard University talk about a systematic, local, iterative and adaptive method to address complex problems and bring about sustainable change. If the problem is endemic, then, according to Andrews et al, the solution has to come from local discussion, engagement and iterative improvements. No force fitting of external solutions is likely to solve it, they say. The problem itself has to be decomposed into its constituent parts—too much power in one office, non-independence of members, for instance—and then adaptive solutions tried for each part. Easier said than done, for sure, but no substitute for the hard work and the trial and error nature of it all.

Let me end on an encouraging note. Sociologists who work on the behaviour of aggregate (free) society believe that issues such as cheating are cyclical and ebb and flow with time. Societies that go through periods of material excess, often correct through a backlash. Like other historical pandemics, the hyper-materialism and uber-greed of our time will pass.

PS: “Lying is done with words, and also with silence,” said Adrienne Rich.

Narayan Ramachandran is chairman, InKlude Labs. Comments are welcome at [email protected] To read Narayan Ramachandran’s previous columns, go to www.livemint.com/avisiblehand-

Posted with Blogsy

From political economy analysis to political economy engagement

Building on the post I just drafted, I would summarize PDIA as a process of political economy engagement, where one is constantly pushing on, learning about, and responding to, political contexts. This is different to political economy analysis where one looks at an writes about political contexts, assuming what one sees is both accurate and real. I think we need to move beyond PEA to PEE.

Posted with Blogsy

Politics and PDIA

I am not a political scientist. As a result, some readers have not been able to find enough 'politics' in the language I use to discuss the limits of institutional reform or in the idea of PDIA. This is interesting, because I see politics all over the book and PDIA. Indeed, I see PDIA as a strategy to effect change and development in contexts where one expect problematic politics. Let me explain.

In the book I argue that political analysis has an important place in development, but is limited. First, it is static and is typically done during project design. But change is dynamic and requires adaptation throughout the muddling through process. Second, it typically focuses on what can be seen and even more narrowly on what outsiders think are the power structures and relationships in a given context. But contextual realities are often hidden, below the sea level in multiple institutional arrangements (ice bergs) in the context being examined, so much goes unseen in these analyses. Third, political analysis often describes either a context that is open to change or one that is not, but often (not always) fails to tell us what to do in the context. But we need to know how to manage politics in real time, not just observe what is observable.

In the book I discuss contexts in which reforms occurred with lots of political analysis but failed to achieve objectives because of the above limits. I found that projects with more impact did not necessarily have lots of political economy analysis up front but instead focused groups in the context on problems they cared about (where enough political support could be located to allow for an entry point) and allowed step by step progression or iteration (to allow political obstacles to emerge, and to respond to such, and to build political support as the change emerged) through broad groups (to ensure that political support was not overly concentrated and to promote inter-agent accountability). This is the essence of PDIA.

• Use a problem as an entry point where there is enough political support to initiate change.
• Go step by step with the change so that you don't demand too much political support at a time, and can learn about the political realities as they go along, and build political support on the basis of each step. Experiments with results help to spread the risk that political authorities may take in supporting change as well.
• Allow flexibility and learning in the process so that you can adapt to the political realities that you see when you push into the context.
• Work with groups and not individuals, to expand the political support you have for change, and to bolster support bases for those who give you support.

Just because my language is not typical to the political economy literature does not mean I ignore politics in development, and just because I don't actively argue in favor of more external political economy analysis in projects does not mean I don't think about managing politics in change processes. I think politics matters a lot and we need to have change processes that take politics seriously. In fact, PDIA is all about working dynamically in complex political settings where I assume we don't see the realities upfront but need to learn and adapt as we go along.

Posted with Blogsy

Why development and institutional reform must be step-by-step

A key argument in my book is that reforms that actually improve functionality and yield institutions that work in complex contexts typically happen step-by-step. In Mozambique I say poco-a-poco.

The book explains the rationale for this by referring to the 'muddling through' work of Charles Lindblom. I argue that we need to create processes of 'purposive muddling' to overcome the limits of institutional reform in development.

In a recent executive education course (on Public Financial Management) a student helped me think about a visual representation of my argument. I thought I would share it here.

The first picture here shows the simple challenge that many governments face in development. They have low functionality (an education system that does not educate or a roads department with poorly maintained roads, etc.). They also have low levels of legitimacy, meaning that the citizens they are meant to serve are fed up with them or that outside entities on whom they depend (donors, etc.) don't think they are up to the job they are doing. These internal and external agents hold back support (often in the form of finances) because of the low legitimacy.

The second picture shows the common response to this challenge. Governments are unsure how to increase their functionality, partly because the problems they face seem absolutely impossible to understand and deal with (and are hence not given the attention they require), incumbent rules of the game are too difficult to overcome, capacity is low, etc. Instead of working against these factors and putting political or bureaucratic capital behind some or other untested and high risk potential solution (that could enhance functionality) many government officials look for 'solutions' that are being proposed by those whose support they need. These are often 'best practices' that outsiders see as routine solutions and 'the right thing to do.' The best practices are attractive because they are pre-packaged and garner important short-term support. In development this support typically comes from donors but is also often forthcoming in the local context as well (doing the 'best practice' reform mean more money in the national budget, better performance on global indicators of management and governance, etc. which can all help to build political support at home). [This is isomorphism…the word you will see throughout the book and beyond]

Now some may do reforms like this just to gain legitimacy, as in the arrow moving up the left hand side of the graph. But some may also assume that these 'best practices' will actually—in time—yield better functionality. In my book I argue that this assumption is frequently proved to be incorrect, however. Best practice reforms do not fit into the many contexts where they are adopted, demanding more content than local capacities can provide, and involving too few agents to actually become implemented. The result is reform with limits: laws that are not implemented, central agencies that have no authority, etc. In the book I call this the problem of 'What you see is not what you get' (wysinwyg). When reforms are shown to be limited, reformers commonly blame the design and initiate a 'better' version. I call it repeat signaling and I see it as a major disease in development. In work I have done with Lant Pritchett and Michael Woolcock we call this the 'successful strategy for persistent failure in development'.

The book—and work with Lant and Michael—suggests a different strategy for doing institutional reform and development. We call it problem driven iterative adaptation (PDIA). The approach embraces the importance of step-by-step interventions that are required when problems are complex and reformers have to i) learn in order to 'find and fit' functional reforms, ii) build and maintain legitimacy and support to get reform done, and iii) build and adjust capacity to ensure implementation is achieved and emerging solutions can be diffused. The approach is shown in the figure below. The basic idea is that reformers in these situations draw attention to the problem they are facing, which they then use to build basic support and legitimacy for some step-by-step experimentation. They take some initial steps, trying various potential reform options at a time. They ensure there is lots of feedback for them to learn what works and why, and they manage the message after these early steps to ensure they can tell some stories about 'quick wins'. They use these messages to build their legitimacy a little more, convincing political leaders, outside funders and citizens that they are on track to solve problems and gaining support for a next step.

This iterative process continues over time until a new solution has been found, fitted and diffused and enjoys legitimacy because it improves functionality.

In the book I argue that successful reforms always emerge through this kind of process. The problem that I see is that this is prohibitively different to the formal way international development organizations work. For these organizations , muddling through is a synonym for 'being muddled' and learning and adaptation are not encouraged or rewarded. Projects must be pre-designed around best practice solutions in order to get support in these organizations. There is little space to support the step-by-step realities of finding and fitting solutions, building political support and capacity, and diffusing new institutions.

The result, unfortunately, is that many institutional reforms in development are ultimately limited….

Regular reviews and feedback help reformers manage contextual complexities and adapt reform designs

The following post was received as a comment from Martin Johnson. I have worked with Martin in various countries and always find his stories of working on long term reform in places like Albania, Uganda and Pakistan fascinating. He I also one of the founders of the PFM Board (with Mauro Napodano), a blog space where you can go for all things Public Financial Management (http://pfmboard.com/)

By Martin Johnson, Principal Consultant REPIM

Matthew Andrew’s 'The Limits to Public Financial Management Reform in Development' draws attention, among other things, to the widespread use of governance and related reforms that are ‘best-practice’ in design but are short-term and seem to ignore context and the myriad layers of minor and major context-specific issues that must be negotiated before technical solutions have even a chance of being adapted properly, never mind implemented at the front line by people that matter (however insignificant or distant those people may appear to project designers or project implementers).

He is not alone in being frustrated by observing the widespread use of this approach and it is no surprise that he presents so much evidence of success in terms of ‘signals’ and so (relatively) little impact from that apparent success.

The time is ripe for a weighty tilt against this paradigm. The evidence that Matt has assembled is certainly weighty. It is also persuasive and certainly has the potential to begin shifting the paradigm.

A fair amount of my own practical experience chimes with much of the analysis and evidence in Matt’s book. This includes coming into contact with projects that aim to support governance reforms but have unrealistic or inappropriate log-frame design, both in terms of nonsensical periods of time over which fully-formed reforms are expected to be designed and delivered and in terms of the contextual “constraints” that are simply overlooked or assumed away.

In one case I had an unusual opportunity to periodically review a major PFM reform over a ten year period. The anticipated achievements in its original design (covering a period of just 2.5 years – a period typical of many development partner-supported governance reforms) were unrealistic to the point of being wholly unachievable. The support would have resulted in abject failure and closure after (or even before) year 2.5 had it not been re-scoped shortly after it started – an interesting story in itself, but one for another time and another place. Even after re-scoping it remained over ambitious and was on the verge of closure on several occasions because of an apparent failure to deliver results (including when very good results were being delivered!).

The story of that reform is one where what was delivered after ten years was more or less what was possible within that period in the highly difficult change management environment of the country concerned (where what Matt refers to as ‘cultural cognitive influences’ are extremely strongly embedded). Interestingly enough, whilst a number of different development partner staff members came close to closing the project at various points in its lifetime because of ‘lack of results’, it managed to survive (sometimes narrowly) in part because it was subject to a regular and independent review process that explained to the development partner on most of these occasions that ‘context’ was being addressed and that progress was commensurate with context, even if it was not commensurate with the project’s over-ambitious design.

From my own experience, therefore, I have come to appreciate one of Matt’s conclusions with regard to ‘best practice’ and the short term nature of many governance reforms - the need for regular reviews to address context.

Overcoming the Limits of Institutional Reform in Development: The Magic of Rapid Results

 

The book argues that there are ways of doing reform that can help overcome limits like one sees with Uganda's anticorruption reforms and beyond (see previous post). Countries need not settle for new laws that are not implemented, or processes that people do not use, or systems that do not address the needs of citizens or organizations.

The book provides a detailed argument in favor of what Andrews, Pritchett and Woolcock call Problem Driven Iterative Adaptation.

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2102794

This is an approach whereby agents in the context gather together to face up to their problems and look for solutions. These agents make-do with what they have, creating hybrid structures through an iterative process. I argue that this is how most functional institutions came into being.

I also note that there are many variants of this approach in development today. One is Rapid Results, championed by Nadim Matta. Nadim was named #25 on Foreign Policy's 25th Top 100 Global Thinkers List that I just saw out today.

http://www.foreignpolicy.com/articles/2012/11/26/the_fp_100_global_thinkers?page=0,24#thinker25

Well done, Nadim. Your work shows that there are alternatives to the dominant way of doing development and there are ways of overcoming limits to change.

Here is how the Rapid Results Institute describes its 'magic':

http://rapidresultsinstitute.info/what-we-do/magic-of-rapid-results

We believe most people are looking in the wrong places for answers to vexing development problems. It is not more money. It is not more analytic work. It is not more workshops. It is not more training of government officials. It is not more participatory planning. It is not more high priced evaluations to determine which donor solution or government policy intervention is the most effective.

All these activities have their place in the development puzzle. But the fundamental engine of change is human motivation and confidence, and the courage to take a first step, to move into collective action boldly and with an intention and a commitment to achieve a seemingly impossible goal.

Once leaders discover how to tap into this engine of change, huge capacities for achievement are unleashed. And all the other pieces of the puzzle line up and begin to fit together in new and more effective ways. The work of the Institute – our Rapid Results Approach – is about leaders at all levels discovering how to tap into this dormant capacity, and about coaching these leaders so unleashing this capacity becomes part of the DNA of how they manage and lead.

We help leaders nudge people at all levels towards new mind sets and behaviors. 

* From spectators to owners and drivers of change

* From low expectations and risk aversion to making commitments to unreasonable goals 

* From a victimhood mentality to a “can-do” attitude of competent professionalism

The Limits of Institutional Reform in Development

I have a new book coming out in January 2013. It is called The Limits of Institutional Reform in Development. I intend to use this blog site to discuss topics related to the book. I hope you find it an interesting place to visit, and am asking for any readers to comment on their experience with reform in development--especially where that reform has been focused on changing institutions ('the rules of the game') in governments.

Here is the first paragraph from the preface to my book, which should give you an idea of its content:

This book emerged in response to mounting evidence that institutional reforms in development often do not work. Case studies and multi-country analyses show that many governments in developing nations are not becoming more functional, even after decades and hundreds of millions of dollars of externally sponsored reforms. These studies increasingly suggest that disappointing results cannot be explained by routine excuses either. One cannot simply blame governments in these countries for not doing reforms, for instance, because many governments remain deeply dysfunctional even after many satisfactorily completed projects introducing best practices advocated by international organizations. The work on institutional reform in development has seldom explored reasons for failure beyond such excuses, however. This has created a gap in the literature, which is important from both an academic and practical perspective. The academic challenge is to see if theory and evidence can help promote a better understanding of why many reforms do not lead to better governments. The practical imperative is more fundamental: Can a better understanding of past experience help improve the likelihood of more institutional reform success in more developing countries in the future?