MattAndrews

Where is the gap between form and function greatest (and why)? Some ideas

 
A variety of readers have asked if I can explain why the form-function gap is in place. Beyond this, if I have any idea how to close the gap.

I think there are a number of ways to think about this. First, we could look for instances where the gap is smallest and try to explain why. Second, we could try and think theoretically about whether and why we should expect the gap. Third, we can put the two together and see if there is any mixture of theory and empirics that could help suggest different ways of doing reforms--that foster functionality over form, for example, as recommended by Lant Pritchett.

In thinking about the evidentiary basis, I can draw on PEFA data on public financial management and suggest the following as starting observations about the form-function gap.

First, when looking across over 100 developing countries, it is apparent that most have gaps between laws and implementation of the laws. So there are very few exceptions to a general rule of form-function gaps. We should thus expect the gaps in most circumstances (as evidenced by the figure below, which shows the size of this gap for each country...where 0.5 is the equivalent of half a symbol in a range from A to D).

 

Second, there are some countries that have no form-function gap. Interestingly, these are mostly small island states (including poor ones). Even more interestingly, the same small island states have better implementation than laws in Global Integrity data as well. Johnathan Eyler-Werve from Global Integrity suggests this may be because such countries have "more of less functional traditions of less formal governance; it's a small country and they just make things work."

I like the idea and wonder if we could check this out more concretely. Is this the full story? What does it mean to have less formal governance systems that 'just make things work'? Is it also that smaller Pacific Island states are left alone more by donors, so don't face some of the 'fashion show' pressures?

Third, there are some countries that have small form-function gaps (32% of the countries have a gap less than 0.3). I have tracked this in detail in past work on African PFM (where the focus was on 31 African countries, not the 100 countries shown above).  Last week I showed that the form-function gap was smallest in a group of countries I called 'league 5' countries ('c5' in the figure below) and highest in the 'league 1 countries ('c1' below). I ask how the smaller form-function gap matters and what factors might be influencing it. 
I find that countries in higher leagues don't only have smaller form-function gaps, but also have better process and outcome scores. The following figure shows both the quality of PFM processes and PFM outcomes--as measured by the donor-generated PEFA indicators--for five leagues (league 1 countries had weakest processes and poorer outcomes; league 5 countries had strongest processes and outcomes). (Readers should see that the process scores are broken down by different areas in the PFM process--from strategic budgeting through external audit).

 
 

What I take from this is that some countries are doing better in reforms in every sense. They have better laws, better implementation of laws, and better outcomes. Others are really weak in all of these dimensions--weak forms, weak functions and weak outcomes. This suggests some important contextual endogeneity. 'International good practice' takes root in some places where things generally go better... and does not in others.

I then asked what kinds of factors differentiated countries in the different leagues. I found a range of factors that have various levels of influence over the quality of PFM processes and outcomes and--it seems, at least--on the extent of form-function gaps: (i) a growing economy (ten years of robust growth); (ii) political stability (for over ten years); (iii) larger domestic, non-mineral income sources; (iv) longer periods of broad reform commitment, and (v) an Anglophone colonial heritage.

These factors are commonly shown as positive influences on reform and governance (although other research of mine casts doubt on factors like the length of reform commitment). I find here that league 1 countries are characterized by few of these factors and league 5 countries are characterized by most of these factors (as in the Figure below). (I show the 31 countries on the horizontal axes, as they fall into the five leagues, and the vertical bars show the number of 'positive factors' in evidence...the higher the bar the more positive factors are evident).

 
**Multivariate analysis shows that some of thes factors matter more than others... (forthcoming in International Public Management Journal).

So it appears that context really does matter when talking about reform progress--countries with positive contextual factors in abundance adopt more reform, implement more reform, and have better outcomes than others. Countries with weaker contexts are likely to do worse in reforms, will adopt only new laws (no implementation) and see very little improvement in outcomes.

As with any blog, let me stop now and ask some questions (hopefully getting some responses): Does this kind of work help to think through ways of doing reform better? What ideas emerge from this kind of work?

 

Key dimensions of the form-function gap: Evidence from African public financial management

Yesterday I showed data from Global Integrity illustrating the gap between the quality of laws and implementation in areas considered important for combating corruption. I have been looking at these gaps in other areas as well, including public financial management.

The data in this domain helps tell a story about the dimensions of the form-function gap. Let me explain.

Donors use a mechanism called the PEFA performance assessment to measure the quality of PFM systems in developing countries. A few years ago I worked with some colleagues to identify which of the 73 PEFA dimensions measured defacto implementation issues and which measured dejure compliance issues. We then looked to see if a sample of 31 African countries (all the PEFAs in Africa at the time) scored differently on defacto and dejure dimensions. Not surprisingly, we found that there was quite a gap.

(see http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1724741).

The gap looks quite a bit like that emerging in the Global Integrity data. The data told us that governments pass laws, introduce new systems and adopt procedures better than they implement and use these.  When we controlled for a bunch of contextual factors, we found that the difference between the laws and implementation was--on average--about half a point (where the lowest score was 1 and the highest was 4). On average, countries were scoring about 3 on laws and 2.5 on implementation of the laws.

Obviously the variation in this implementation gap was different for different countries. It was particularly large in countries with the contexts least conducive to reforms; these countries were passing new laws with no hope of implementation. This is the first dimension of the form-function gap. I call it the de facto problem.

The research identified two other dimensions of this form-function gap. The first can be called the downstream problem and the second the deconcentrated problem:

• The first problem emerged in observations that PEFA scores were higher for budgeting processes than they were for downstream execution processes. Governments were making governments better than they were executing them (once again, the better budgets made the governments look better but actual spending was not much better).
• The second problem referred to evidence that PEFA scores were higher where performance required the action of a narrow set of concentrated agents--like the budget bureau, treasury, central procurement agency. PEFA scores were lower when they required action by a deconcentrated set of agents in line ministries, districts, etc.  Engagement between donors and core agents suggested things were better, but in reality they were not.

The three form-function gap problems are shown in the following figure. It shows different scores in defacto and dejure, downstream and upstream and deconcentrated and concentrated agent dimensions for five groups of countries. The five groups reflect different contexts: c1 countries had generally positive reform contexts, whereas c5 countries were in tough contexts and had much lower PEFA scores. C2, c3, and c4 sit in between.

You can see that the defacto, downstream and deconcentrated problems manifest in all country groups. Form is more impressive than function everywhere. Laws promise more than can be implemented, budgets look better than the spending they are meant to represent, and narrow groups of core government agents do more than the average deconcentrated agents.

This helps explain why reforms progress in many countries but governments do not get much better over time (in terms of real performance). Form is better than function. Again. It will likely continue in this vein as long as donors support reforms to make better formal products (like laws, budgets etc.) and continue to work almost exclusively with core counterparts in capital cities.

 
 
 

The governance fashion show

Measuring the gap between form and function: Global Integrity data

I am constantly on the look out for measures of the gap between forms and functions of good governance. This week I have been intrigued by the Global Integrity data on the topic (http://www.globalintegrity.org/report). Global Integrity Reports are described as follows:

"The Global Integrity Report is a tool for understanding governance and anti-corruption mechanisms at the national level. Written by local researchers and journalists, the Report is characterized by an innovative, award-winning research methodology; a robust peer review process; and start-to-finish transparency."

The reports show scores for a range of countries on the quality of laws and the implementation of the laws. It also shows the gap between laws and implementation--how much better form is than function. Here are the highlights and some important observations:

First, consider the following table, showing countries with the lowest implementation gaps.

• All countries have implementation gaps (legal forms are always better than functional implementation)
• All of the wealthier countries in the sample have implementation gaps smaller than 25.
• Very few developing countries have implementation gaps smaller than 25.

Global Integrity Index 2008	Country Score	In Law (L)	In Practice (P)	Implementation Gap (L - P)
Italy	79	82	73	9
Poland	88	92	82	9
Kazakhstan	76	80	69	11
United States (2007)	85	90	78	12
Japan	83	91	76	14
Korea	88	96	82	15
Bulgaria	87	95	79	16
South Africa	79	86	70	16
Hungary	77	85	67	18
Turkey	69	78	59	19
Macedonia (FYROM)	76	86	65	21
Chile	77	87	66	21
Lithuania	74	85	63	22
Canada	80	90	68	22
Morocco	48	58	35	22
Ghana	67	77	55	22
Kenya	69	79	56	23

Second, consider the quality of laws in some of the wealthier countries above, compared with those in the table below.

• A number of developing countries have better scores on their laws than wealthier countries like Italy. Colombia has better scores on laws than Chile.
• Most developing countries have large implementation gaps as well. Colombia's is nearly twice that of Chile's.
• The best laws in the world are in Uganda (100 out of 100). Uganda also has the highest implementation gap (51).
• Uganda's experience is similar to that of many other developing countries: High scores on laws and really low scores on implementation dominate.

Global Integrity Index 2008	Country Score	In Law (L)	In Practice (P)	Implementation Gap (L - P)
Cambodia	46	58	30	28
Indonesia	69	83	55	28
West Bank	53	68	39	29
Serbia	62	75	45	30
Egypt	54	67	38	30
Angola	41	56	26	30
Cameroon	55	70	39	31
Nigeria	64	79	48	31
Bangladesh	68	84	52	31
Ecuador	60	75	43	32
Nepal	64	81	48	33
Belarus	66	81	48	33
China	59	74	41	33
Romania	80	97	64	34
Russia	69	89	52	36
Georgia	59	78	41	37
Colombia	71	90	53	37
Philippines	71	91	54	38
Nicaragua	58	76	39	38
Argentina	70	90	51	39
Moldova	68	88	48	40
Somalia	35	54	14	41
Montenegro	56	77	36	42
Pakistan	72	94	52	42
Yemen	46	66	23	43
Iraq	53	75	32	43
Kyrgyz Republic	64	87	42	45
Bosnia and Herzegovina	63	87	41	46
Uganda	72	100	49	51
Azerbaijan	61	87	36	51
Ethiopia	56	82	31	51

Third, some questions:

• Is it better to be Uganda with 100 on laws and 49 on implementation, or Italy with 82 on laws and 73 on implementation?
• Should Somalia's governance reforms (if and when they start) focus first on improving the quality of laws (from 54) or implementation of existing laws (currently 14)?
• Do externally influenced governance reforms have any impact on implementation? Or are their effects limited to improving laws? What are donors doing to close the implementation gap?

My worry with the story emerging from these numbers is simply that governance reforms have created a gap between form and function and continue to do so. There are too many countries that have 'benefited' from costly governance reforms but still have extremely low implementation of laws (many of which were introduced via reforms). 

The data here is quite provocative in telling this story, and raises questions about what reforms have achieved in countries as varied as Cambodia, Ethiopia, Azerbaijan, Egypt and Georgia. Note that the IFC has awarded some of these countries awards as best reformers for Doing Business. (How should we tell this story? Do good 'Doing Business' reforms involve passing great laws you can't implement? Is the goal to look better without being better? Or do we really believe that in time the gap will close?) 

Other examples of data showing the gap will be welcomed. As will any examples of the 'development as functional improvement regardless of form'. As will any comments that tell me that this is all much ado about nothing.

 

Public sector reform, good government and the quest for functional thinking

I thought I would post this link to a transcribed interview I did with our media folks in response to my 2010 Governance article 'Good Government Means Different Things in Different Countries'.

http://www.hks.harvard.edu/news-events/publications/insight/international/andrews

My kids laughed at the video interview...

Malawi: another example of looking better but not being better

I am busy looking into the history of donor-driven anti-corruption reform in Malawi. Interesting. Frustrating.  Amusing. Another case of form and function not meeting.

 Global Integrity did some nice work on the subject (http://report.globalintegrity.org/Malawi/2007/). They show a time-line of events starting in 1999. Reforms actually began a few years prior, with elections in 1994, a new anti-corruption law in 1995, and the creation of an anti-corruption bureau (ACB) in the same year.  The first few years after this were full of positve signals--cabinet Ministers were referencing the struggle in speeches, the ACB was pursuing thousands of leads, etc.

But a scandal errupted in 2000 and 2001, where donor funds destined for classrooms ended up in government officials' pockets instead. Officials at all levels were implicated, including then-President Muluzi. Donors stopped funds.  And started looking at what was really going on in the anti-corruption reforms. They found that the ACB was poorly staffed and resourced, that the ACB had investigated thousands of people but only initiated prosecutions in a handful of cases--very few of which ever reached conclusion. The ACB actually lacked authority to prosecute any of its cases, and was never a major player at all.

The picture was clear: A law plus law enforcer were created but never really had any influence or legitimacy. One would expect a dramatic response from all parties. But this never happened.

The anti-corruption reforms have continued along much the same path, with donors coming behind the ACB with increased funding and assistance. They have sponsored a new strategy document, and have even helped develop new IT-based approaches to designing systems of prosecution. They have also helped consistently modernize and update laws. (I am not sure who did it, but my favorite is the recent law against 'fouling the air' http://www.bbc.co.uk/news/world-africa-12363852). Very few countries in the world have such robust legal frameworks!

The 2007 Global Intergity Report called the situation for what it is: Malawi now has world class laws with a problem of implementation. In fact, the country ranked 90th out of 100 on the quality of its laws and 56th on implementation. The Report identifies a 'Very Large' implementation gap. This dropped a bit in the 2009 report, but still remains significant (http://report.globalintegrity.org/Malawi/2009).

Interestingly, Malawi's laws were rated at similar levels to Italy and the United States, Canada, France, Poland, Korea and South Africa. There are small 'formal' differences between developed countries (and middle income leaders like South Africa) and developing countries like Malawi. Developing countries now 'look better'. The real difference is in implementation, where the implementation gap in places like Italy, the US and others is substantially lower than that in Malawi (and other lower income developing countries like Nigeria, Nepal, Guatemala and Ethiopia--where the legal framework ranked 94 out of 100 in 2010 but implementation lagged at 46...). Developing countries that look better are not better in a de facto sense.

A 2010 review of NORAD/DFID support to Malawi's ACB references similar problems with making forms functional, but in words (http://www.norad.no/en/Tools+and+publications/Publications/Publication+Page?key=207030). Consider the following statements: "The National Anti-Corruption Strategy (NACS) has been launched, but key processes, such as an implementation plan and secretariat, are not in place yet." "The Government of Malawi approved a new ACB staff establishment...However, unsecure funding calls for more donor engagement, risking ACB’s independence." "The  prosecution database has been developed but is not fully operational yet." "A prosecution strategy as proposed by the last review has not been developed and is seen as controversial within the ACB."

These kinds of descriptions raise questions about Malawi's ACB, which could also be raised in respect of legal reforms in many developing countries (especially given the implementation gaps evident in Global Integrity's work): Are these reforms square pegs in round holes? Or are they just examples of new pegs needing 'fitting'? Are they forms that will become functional? Or are they forms that cannot function because they are irrelevant in the context (as Richard Rose would say)?  

Beyond these questions, lies the obvious one: Were there other forms of addressing the corruption problem that could have yielded greater functionality, in Malawi and beyond? Perhaps there are, on the ground, in select corners of Malawi and other countries. It would be great to hear if this is the case. I already know of the Business Action Against Corruption initiative (BAAC) which has emerged quite endogenously in the private sector (http://www.aiccafrica.org/index.php?option=com_content&view=article&id=115:baac-malawi-honoured-with-world-bank-anti-corruption-collective-action-award&catid=35:latest-news&Itemid=81). This seems to be a  functional response by private companies to the problem. Note that it did not emerge because of a new law or specific form. It is really interesting, and begs questions: How is this different to the ACB initiatives?

Simple answers may be too simple: It is local, it is shaped to fit the context, it is the response of real agents to a real problem they care about. This is how 'real' solutions always look. What Lant Pritchett would call local trees with their own roots. My metaphor is a camel in the Sahara (instead of a Hippo).

ps. read Gerhard Anders' 'Like Chameleons..." for further information. http://apad.revues.org/137

 

 

 

 

 

 

 

 

 

 

 

 

Development as form without function; Lant Pritchett on air

Consider my colleague Lant Pritchett, on development as form without function--imported trees without roots he calls them. My idea is Hippos in the Sahara.

http://www.youtube.com/watch?v=DprwLnySaSs&feature=relmfu

 

See also the linked paper: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1824519

This is a major issue in development that we need to pay more attention to. Changes in the world right now show up many of the development community's reforms and indicators of governance and government:

• Consider that Iceland was ranked way up there in many governance indicators prior to the 2008 crash (http://info.worldbank.org/governance/wgi/pdf/c108.pdf). It was praised for being transparent, well managed and efficient, etc etc. Then the 2008 crisis hit, and everyone found that processes, systems and laws were not as solid as they seemed.  They looked great in form but there was a gap between form and function. Maybe a country can never be functional enough to survive a crash like one Iceland experienced, but how much was the crash exacerbated by poor transparency, weak de facto government, etc.?
• Or consider Ireland, which also scored really well on all dimensions of governance indicators (http://info.worldbank.org/governance/wgi/pdf/c105.pdf).  It was a better practice example for other aspiring European nations in areas like budgeting, debt management and public investment. These are precisely areas where Ireland is now struggling. Form versus function again? Or am I being unfair?
• Or consider that Tunisia and Egypt were both recognized as 'Doing Business' reform leaders in the past few years--recognizing legal etc. changes designed to make it easier to do business ( http://www.doingbusiness.org/reforms/top-reformers-2010/). I have not been engaged in either country in the past few months, but many stories I read suggest that small business people have been engaged in recent uprisings because of major problems in doing business (here is a reporter's perspective, which I can't validate but seems representative:  tp://www.businessweek.com/magazine/content/11_05/b4213013748688.htm). 

Better 'Doing Business' scores did not seem to equate with better doing business in these countries. The countries looked better--but do not appear to have been better. Form without function again. Lant's challenge is an important one: How do we focus less on forms and more on function, so that countries actually get better. This requires promoting organic and local processes of development, that may produce governance forms that look strange and even inferior--but could work to make things better. Striving for and promoting functionality regardless of form.

Transparency, Accountability and Governance

Alan Hudson recently tweeted a question: Are accountability and transparency always relevant concepts in the  good governance discussion (find him as alanhudson1 on twitter, and at his blog,   http://www.alanhudson.info/ and the ONE blog http://www.one.org/international/blog/author/alan-hudson/).

The question is interesting. It is an important effort to go beyond poorly considered ideas about governance that do not travel well and are not always relevant. It is an effort to say "do we know if specific aspects of governance hold in most contexts?" I will re-phrase the question with some of my own:

• What do we mean by accountability?
• What do we mean by transparency?
• Are they means towards better governance?
• If so, how do we think they facilitate better governance? Do we think they could ever compromise governance?
• Or are they ends of governance reforms themselves?

Let me give my thoughts on both concepts.

On transparency

Alan yesterday tweeted about an interesting post by my good friend Joel Turkewitz at the World Bank (http://blogs.worldbank.org/governance/node/895). It helps to think through some of these questions I raise above.

Joel notes that if you simply value transparency because it seems to improve accountability and facilitate lower corruption, evidence suggests it does not achieve these ends alone. He says, however, that transparency is an important means to various other ends. This yields three reasons he says we should care about transparency: 1. Transparency potentially changes the way government operates. 2. Transparency potentially changes the relationship between people and government officials. 3. Transparency enables groups that otherwise would not be able to participate, to participate in governance.

I like the ideas Joel is exploring. Let's think about them:

• First, transparency is a means and not an end--a tool that could help achieve important results.
• Second, transparency works by changing relational dynamics between agents (in the workplace of governments, the interaction of governments and citizens, and amongst citizen groups themselves, for example).
• Third, transparency may not be always be the right means--tool--and may not always have the desired effects (see Joel's use of the word 'potentially').

Given such thinking, I see transparency fitting into the governance discussion naturally and easily. Governance, in my approach, is all about the authority relationships between agents that facilitate getting things done (see earlier posts). Transparency is a means that can sometimes be used to improve these authorizing frameworks and help improve outcomes. It is 'potentially' a useful tool in such.

But note that it should not always or automatically be considered a good fit or useful tool. One can think of many cases where transparency does not improve governance and lead to better results; where opaqueness is the better tool. The obvious examples are intelligence and defense operations, but think also of the debate about transparency of patient records in the US health care industry (which many don't want released because patients rights may be violated), and all the professors out there know well that students do not always value transparency in grades (at least not when it comes to others seeing their grades).

To know when and how transparency matters as a tool of governance, one should start by asking who is engaged in the governance situation, what results are hoped for, and if there are transparency problems. So, transparency matters but not in a standardized manner and not all the time.

My thoughts on accountability

I feel similarly about accountability, but have some problems with the concept. The main problem is simply that I don't think we all understand the concept in the same way. Next time you have a room of experts together, ask them to write down what they understand by 'accountability' and an example of accountability in action. I guarantee multiple answers, laden with values and norms, providing fodder for difficult debate.

Beyond this, I think the concept fits into a governance discussion in the same way as 'transparency'. As with governance, accountability is about relationships. So the question is: who is meant to be accountable to whom? As with governance, accountability is also about results. So the question is: what is agent x accountable to agent y for?

Now we are talking about a concept that matters. But note a few problems. First, all agents have multiple accountabilities--to different agents and for different things. This is particularly true for govenment officials--whether politicians or bureaucrats. It is also true for all of us working on the outside of developing countries, accountable to ourselves as individuals, to the donor agency that funds a particular job, the officials we work with, the citizens of the country affected by our work, our families (for the dollars we bring home) etc.

When we speak of 'Improved accountability', what is it we are talking about? My concern is that accountability and governance come together in a normative discussion and not a real one, where 'accountability' is a generic concept that we should simply pursue. In fact, it is complex and contextual relational idea that needs to be unpacked for every situation. My bigger concern is that when we talk about developing countries being accountable we mean 'they do what we say'. We need to remember that accounatbility relationships are up, down and sideways for all of us and that the relationship of donors and governments is in fact one of the biggest challenges to achieving 'accounable' governments and 'good governance' in developing countries.

Transparency, accountability and governance

Given the above, my feeling is that transparency and accountability are central concepts in the discussion of governance but need to be contextualized, unpacked and treated with caution. I don't understand people who speak generally about thes concepts.

What does it mean to 'Improve budgetary transparency to improve accountability'? We need to be more specific, because any read of Wildavsky will tell us that accountability is a multi-dimensional construct in the world of public budgeting, and any read of the literature on fiscal rules will tell us that greater transparency often leads to more games and reduced accountability of bureaucrats and politicians to citizens.

Accountability and transparency matter in specific ways and specific contexts. Better information on education fund flows, provided to citizens, may help improve the accountability of teachers and principals to citizens and ultimately facilitate better school governance for instance. In all cases, governance, transparency and accountability are about agents engaging with agents and any intervention needs to think through who engages with whom, for what, how, when, why. These are the generic questions.

FIFA: The beautiful game in not so beautiful hands

Given yesterday's resignation by a FIFA official, and my great love for the beautiful game, let me start everyone thinking about the FIFA governance puzzle, and particularly about what a transparency-enhancing approach may look like to get FIFA more accountable and better governed.

Who are these officials meant to be accountable for, for what, when, and why? I fear that calls for more transparency and accountability are useless until we are clear about the answers. If they are a politically elected cabinet, who is the electorate and how can transparency help that relationship? If they see themselves as the braintrust of a clandestine syndicate, how could transparency help?

Inside the coin toss: And governance reforms as Hippos in the Sahara

Last week I blogged about research I am involved in that looks at whether governance reforms have led to better governance indicator scores (WGIs, QOG, etc) and more robust economic performance (growth in particular) in a sample of 40 countries. The work shows that countries undertaking governance reforms are as likely to see declining governance scores and slow growth records as they are likely to experience improvements in these indicators. Success is as reliable as the chance of getting heads when tossing a coin. 

I had some responses online and off. Some people challenged me to go deeper, into factors that could make a difference between the two groups of countries. Great suggestions that I try to act on in forthcoming work. Here is a taste of the findings.

First I ask whether the countries with better scores/growth records simply had more successful records of reform. I find this is not the case. World Bank project assessments are as likely to be 'satisfactory' in countries with declining and improving indicators. While these ratings are inherently political and unreliable, they are all we have, and they are reinforced by the fact that countries with declining and improving scores continue to get loans for new governance reforms and other interventions. Usually, donors explain that the new loans are a result of improvements stemming from other projects--in all countries, even with worsening governance scores and low growth.

 Second, I ask if the declining score countries did different things. The answer is no. Interestingly, I find it quite easy to see a common pattern of reform design across the 40highly variant countries. They typically all start with stabilization, privatization and liberalization interventions and public financial management and civil service reforms that formalize and rationalize the public administration. They then move into more 'modern' public management interventions like multi-year budgeting, meritocratic hiring, performance management and decentralization reforms, efforts to enhance transparency, promote participation and others. These interventions are focused around changes in laws in most countries, and the introduction of new systems and processes, as well as the strengthening of central agencies. Some countries that do these things see improvements in governance indicators and more robust growth, others do not.

Third, I ask if contextual factors at the country level matter. As with past research I find that they do. Extreme contextual stresses like instability certainly account for low governance scores and growth records in a number of countries. Past income levels also make a big difference, as do measures of democratic influence and a variety of others. What these variables tell us is that different countries have different challenges, endogeneity matters, and we face major limits to doing governance reform in many contexts. As one commentator noted last week, 'the coin toss may depend on context'. The coin toss does depend a lot on context. So, why do reforms look so similar in so many contextually different places? If we want to improve the odds of success, shouldn't we better fit reforms to context?

Fourth, I looked at whether time makes a difference: Did countries with longer reform histories produce better results, regardless of context? The argument is frequently made that governance reform takes time, and we should be patient. The assumption is that externally designed, defined and supported reform matters but only in places  where there is long term commitment to its implementation. But this is not what I found. I analyzed the degree to which the number and cost of World Bank governance projects influenced WGI and QOG scores and growth records over time, with key dates every year from 1998. I controlled for contextual factors (briefly noted above) and identified World Bank projects in four year intervals preceding each date when the dependent variable was captured.

I found that projects had a positive impact on dependent variables (WGIs, QOG scores in particular) in the short run, but not in the long run. In fact, it seems to me that countries get a bump in governance scores by introducing new projects and passing the new laws associated with such, but then commonly see scores drop when they cannot follow through on the reforms--the laws are not acted upon, the new procurement offices do not work, the multi-year budget and fiscal rule fails to improve spending quality and discipline. They then look for new projects to take eyes of the poorly implemented old ones. Often the new projects lead to new laws replacing the ones introduced in past projects.

Examples are easy to find: Argentina, Afghanistan, Georgia, Niger, Rwanda...countries that get short-term rewards for agreeing to do reforms that ultimately prove difficult (or impossible) to implement, and add new reforms every four to six years regardless of problems implementing those that went before. Reforms are signals, not changes.

This work is part of a bigger project on the failure of governance reform in development. It seems to me that the observations I am making are not simply that WGIs are poorly measured because they are based on perception. It seems to me that there are deeper and more salient issues in development that we need to look into more deeply:

• Why do we have a view of governance--and indicators--that we cannot say is actually conducive to development?
• Why do we continue with reforms that don't seem to improve things in many places?
• Why, when we know context matters, do we continually ignore it?
• Why do reforms look the same everywhere?
• Why do we seem continually satisfied with interventions in form with limited function?
• Is it even possible--or desirable--for external agents to engage in the internal change process of developing country governments?

I have recently been calling this the problem of Hippos in the Sahara. The developing world is full of countries with contexts that are as problematic as the Sahara. And as peculiar. They need some functional solutions to real problems; think of camels to help provide transport.

Reforms that look like camels are difficult for the development community to deal with, however. These evolve particularly for the context in which they emerge and fit a lot less outside of such. They are not found as best practices in more developed contexts (where 'best practices' arguably always come from).  Instead of helping developing countries find contextually frunctional camels, therefore, external agencies encourage these countries to adopt what they see as best practice in developed countries.

These are often the equivalent of hippos. They are impressive looking reforms, like the strong creatures themselves. When countries adopt them, they look fantastic--"How in the world did they manage to introduce a Hippo into the Sahara?" The problem, of course, is that the hippo takes over the scarce resources in the context (whatever the equivalent of water and shade is) and ultimately proves useless and impossible to keep going. It is a reform of form, with little functional promise.

I found this great picture of a Hippo in the Sahara by Leslie Sealey, a painter who has given me permission to show it here. (Her own blog is http://lesliesealey.blogspot.com/). I love the picture, especially with the bedouin sitting outside the tent--ousted by the large hippo. I have shared it with many officials in developing countries who initially chuckle and then quietly confide, "I am a hippo keeper, you know." I hope that their countries also have some good camel seekers.

 

Governance reform results are as reliable as a coin toss

What do governance reforms and coin tosses have in common? They both yield positive results 50 percent of the time...

If we assume that governance matters and even that governance indicators make sense, what is the record of externally influenced reforms on governance? 

• One would hope that reforms from organizations like the World Bank have helped countries to improve their economic growth records--given the regularly implied argument that good governance promotes growth and development.
• One would expect World Bank reforms to impact World Governance Indicator (WGI) scores directly, given that such interventions and WGI’s are both products of the same external development community. They are the ‘test’ around which reforms may have been shaped.
• One would also be led to believe that reforms sponsored by organizations like the World Bank facilitate improvements in alternative measures like the International Country Risk Guide (ICRG) Quality of Governance (QOG) indicators. These provide potentially more independent yardsticks than WGI scores but ostensibly reflect things we care about when talking about governance.      

Given such measures, the record of economic growth and government improvement varies across a random sample of 40 countries I am looking at. It includes Afghanistan, Algeria, Angola, Argentina, Azerbaijan, Bolivia, Bulgaria, Cape Verde, Central African Republic, Chile, China, Georgia, Ghana, Guinea, Haiti, Honduras, Kyrgyz, Laos, Madagascar, Malawi, Mauritius, Moldova, Mongolia, Mozambique, Nicaragua, Niger, Poland, Rwanda, Samoa, Senegal, Serbia, Sri Lanka, Tanzania, Thailand, Togo, Uganda, Ukraine and Uruguay. 

• While GDP per capita increased in 38 of the countries between 2000 and 2008, growth was more sluggish than the average for comparable countries in 25 cases. Personal incomes increased by about 20 percent in Bolivia in this period, for instance, which was below the mean for lower middle income countries.
• Similarly, 21 of the 40 countries went backwards on WGI Government Effectiveness scores between 2000 and 2008. For example, Cape Verde dropped from 0.35 to 0.05 on a measure that ranges from -2.5 to +2.5. Togo regressed from -0.67 to -1.43.
• Related patterns are seen in the Quality of Governance (QOG) data, where values decreased in 19 of the 28 nations included in the ICRG database in this period. Argentina fell from 0.69 to 0.52 where the indicator runs from 0 to 1. Moldova dropped from 0.55 to 0.44.

There is a fair amount of correlation between these indicators. This overlap ensures that 18 countries languish in the ‘declining’ group of at least two of the indicators. A similar set of 18 countries are ‘improvers’ on at least two indicators.

Interestingly, the governance reform experience looks similar across the two groups:

• There is no statistically significant difference between the groups when one looks at numbers of World Bank public sector reform projects, dollar commitments for such or reform start dates. 
• The same can be said when looking at growth in per capita incomes, WGI Government Effectiveness scores in 2008 and QOG records in 2008. In all cases countries with declining records had similar reform records to those that progressed.

While not statistically different, the average country with declining WGI scores actually had more projects (59) and higher costs ($1.3 billion) than the average country with improved WGI scores (48 projects at $1.1 billion). Reform starting dates were also similar. Of 19 countries that started reforms before 1985, 9 saw declining government effectiveness in the 2000s and 10 saw improvements. Of the 21 that started after 1985, 10 declined in the 2000s and 11 improved.

The message from this data is plain: Countries with similar records of externally influenced governance reform have seen varying results.

Taking the data at face value, it appears that there is about a 50 percent chance that such reforms make governments more effective. Half of the time countries pursue this reform but government gets less effective. Success is as reliable as a coin toss producing heads.

Is this good enough? Is it something we should expect? Is it because the reforms don't work or maybe because 'governance' is a problematic means or end in the development debate?

Thinking seriously about governance

Timothy Ogden has started an interesting discussion about governance in the Financial Access Initiative (FAI) blog (http://financialaccess.org/node/3746). He asks for serious thinking about governance in the microfinance industry. He argues that the governance arrangements in place in organizations are more important than their 'profit status'. Poor governance allows for scandal and wrongdoing in all kinds of organizations, whether they are for profit, non profit or partly profit based. He asks "What are proper governance arrangements for a pro-poor institution?"

This is a great question. It, and the discussion as a whole, offers a number of challenges to the broader discussions about governance in development. I see four in particular:

1. Conceptual content, and implications for measures and assessment:

'Governance' is given quite specific meaning in the posting. It reflects on how organizations are authorized, by whom, and with what effect.  This is a much more specific approach to the content of this concept than one usually finds in the broader application of 'governance' in development. Think about the World Governance Indicators, for instance, that seem to enjoy so much breadth of content as to defy definition.

This raises a challenge: what does the development community mean when it talks about governance? As a reluctant academic who also inhabits the world of practice and engages with practitioners, I tend to prefer concepts that we can make sense of in our research and reform discussion. I am thus supportive of Ogden's sensible application.

This application is actually found in the  literature on governance, where one sees greater conceptual clarity and content focus than is typical to the loose approach in international development. Indeed, the term was generally meant to reflect on the way in which authority is exercised in a particular context. This is how Michalski, Miller and Stevens (2001) define it. It is even how the World Governance Indicator (WGI) developers put it, though using different language.They speak of governance as  “[T]he traditions and institutions by which authority in a country is exercised” (Kaufmann, Kraay and Zoido-Lobaton 1999).

If the content of this concept is about the exercise of authority, what do we need to measure such? How do we evaluate such? Which of our indicators actually reflect on how authority is exercized? Have we shifted this concept into a realm of problematic but opportunistic vagueness?

2. Unit of analysis and the focus of assessment and reform: 

Ogden's discussion is of governance at two levels--within 'pro poor' microfinance organizations and across the microfinance community. He mentions lessons about governance in the organizations and implies that these should be learned and potentially applied in the field of 'pro poor' microfinance. I use the term 'field' here, in the spirit of DiMaggio and Powell's (1983) work on organizations. They recognise that organizations work with other organizations to make up activities in fields like micro-finance. The 'field' is that particular set of organizations that, “in the aggregate, constitute a recognized area of institutional life” (DiMaggio and Powell 1983).

It seems sensible to think about how authority is exercised at these levels. It seems that we can measure this at these levels. It seems also that indicators we might develop, and lessons we might learn, can be transformed into relevant reforms for the organizations and fields.

Now consider the way governance is analyzed and applied in international development. We have governance figures at the countrylevel (see the WGI definition of governance above). This level of analysis combines all organizations across all fields, assuming that this can be done and makes sense--practically or theoretically. So we say that Kiribati has better governance than the Marshall Islands, and Rwanda has better governance than Pakistan.

But what do these indicators mean, and what can we do with them? It seems to me that governance should be assessed and governance reforms be focused on less aggregated units of analysis. Authority is exercised in different ways in different fields and organizations within countries. Even more importantly, the challenges of exercising authority are different in different organizations and fields. Pro poor microfinance and education fields involve different agents, for example, and have different political pressures to contend with. Aggregated measures just do not capture this variation, or allow learning about how  to structure governance solutions given such variation.

If you buy this, what are the implications for governance indicators in the development community? How many indicators either build up from organizations or assess governance at the level of discrete fields? What could indicators like this look like? How could indicators like this be used for developing ideas about 'good governance'?

 3. Focus on form instead of function, and implications for the tone of discussion:

The thing I like most about Ogden's short note is his focus on bottom line issues: he emphasizes the importance of pro poor microfinance institutions actually serving the poor with micro finance. This is the goal of the organization and this is what we need better governance for. A simple idea. Stated simply, governance is important to improve the functionality of organizations and fields, reflected in results and outcomes. Governance is not a set of forms that we speak about because it has stand alone importance.

I think this approach is well reflected in theory on 'governance'. Tirole (2001) defines corporate governance as “the design of institutions that induce or force management to internalize the welfare of stakeholders.” Consider the basic elements of governance implied in this definition: (1) It is focused on how mechanisms (institutions) regulate (2) the way that authority is exercised by one set of agents (managers) (3) who act on behalf of a broad group of principals (stakeholders) (4) with the goal of maximizing the welfare of these principals (stakeholders). Combining these elements, we see that good governance emerges when specific agents exercise delegated authority in such a way that their specific principals enjoy improved welfare. The goal is better functionality. 

This definition has parallels in the political science and public management literatures. Kooiman’s (2003) characterization of governing, for example, points to “the totality of interactions, in which public and private actors participate, aimed at solving societal problems or creating societal opportunities.” Hill and Lynn (2004) describe public sector governance from a management perspective, as “Regimes of laws, rules, judicial decisions, and administrative practices that constrain, prescribe, and enable the provision of publicly supported goods and services through associations with agents in public and private sectors.”

The idea of delegated authority emerges across these definitions, as does the focus on outcomes as the purpose of delegated authority ("maximizing stakeholder welfare", “solving societal problems or creating societal opportunities” and ensuring the “provision of publicly supported goods and services”). In Ogden's world the question of governance centers on how poor people delegate and enforce delegation of authority to micro-finance institutions, to empower the poor. This is a functional problem. Real, tangible, specific and practical.

In the world of international development, I contend that governance is much less real, tangible, specific, practical or meaningful. Many indicators focus on form and not function. Indicators like the World Governance Indicators, Doing Business Indicators, and even various quality of governance indicators reward countries for having specific forms in place. They either assume that these lead to better results or de-link governance and results and outcomes. At this extreme, good governance is really about systems, processes, laws etc. that have a stand-alone importance. Looking better is more important than being better.

Many will disagree with me on this. They will argue that outcomes and results are central to the discussion of governance in development. I would love to be wrong, and welcome the debate. Arguments would need to show that indicators actually emphasize results and/or that where they emphasize processes we actually have evidence that the new forms lead to improved results and functionality. I don't think the arguments can be made, partly because the questions are best askd at differnet levels of analysis (organizations and fields) but any responses in this light will be welcomed.

4. Solutions and answers versus problems and questions:

The tone and approach to Ogden's note is admirable. It notes that governance is a problem we should care about and asks questions about what kinds of governance are needed to improve services in the micro-finance community. This stands in contrast to the international development community's approach to governance, where indicators impose solutions and answers without clear problems or questions.

I agree that governance matters. I agree that it is something we should treat seriously. I agree with Ogden that we have questions rather than answers. I wonder if it is possible to re-focus the international development dialog so that it too goes back to problems and questions...