MattAndrews

Governance indicators and governance outcomes: How do they relate?

 Alan Hudson asked yesterday how I relate my comments about constructing governance indicators to prior work I have done that argues governance is all about producing outcomes. I replied in comments, but let me draw a bit from a 2010 paper I wrote on this subject (in Oxford Development Studies). Think about this as the way I am conceptualizing the process of 'theorizing' governance indicators.

We need to have some idea what governance is, before starting. In a general sense, most see this as the exercise of authority. Which requires us asking: who the authority is exercised by, on behalf of, and for what reason?  Answers to such questions will help guide us to appropriate measures of the phenomenon.

I find the literature on corporate governance in publicly traded companies useful in addressing these questions. Tirole defines corporate governance in this context as “the design of institutions that induce or force management to internalize the welfare of stakeholders”. Consider the basic elements of governance implied in this definition: (1) it is focused on how mechanisms (institutions) regulate (2) the way that authority is exercised by one set of agents (managers) (3) who act on behalf of a broad group of principals (stakeholders) (4) with the goal of maximizing the welfare of these principals (stakeholders).

Combining these elements, it can be seen that good governance emerges when specific agents exercise delegated authority in such a way that their specific principals enjoy improved welfare.

This definition has parallels in the political science and public management literature. Kooiman’s characterization of governing, for example, points to “the totality of interactions, in which public and private actors participate, aimed at solving societal problems or creating societal opportunities”. Hill & Lynn describe public sector governance from a management perspective, as “Regimes of laws, rules, judicial decisions, and administrative practices that constrain, prescribe, and enable the provision of publicly supported goods and services through associations with agents in public and private sectors”.

The idea of delegated authority emerges across these definitions, as does the focus on outcomes as the purpose of delegated authority (maximizing stakeholder welfare, “solving societal problems or creating societal opportunities” and ensuring the “provision of publicly supported goods and services.”)

In the public sector context we are obviously dealing with governments as the agents to whom authority is delegated, by citizens (as principals), with the explicit goal of maximizing various kinds of social welfare (as the outcome). Governments can use this delegated authority in many ways: to garner and allocate resources, build capacities (human and physical), regulate via laws or force, and so forth. Governance is superior when authority is exercised in such a manner that social welfare is maximized and governance is inferior when authority is exercised in a manner that does not maximize welfare.

In this way of thinking, governance indicators should assess the degree to which organizations structure their authority relationships to produce outcomes--not just the degree to which they comply with a generic set of best practice processes.

Some roles, interactions and mechanisms might be more effective than others in facilitating better governance outcomes. Some may have generic import to all organizations producing the outcome, some may be relevant contingent on specific contexts, others may be cutting edge ideas we think will have value but are still untested.

These kinds of practices can only be identified after we look at specific organizational fields (e.g. education or health care), measure the outcomes of interactions in these fields, and consider contextual differences between countries. Function must lead form in this respect.

We need to theorize our indicators to make this happen. This starts with us having clear ideas about the outcomes we are looking at (and our normative approach to thinking about such) and the fields in which we are focused. It moves to having some theoretical idea about why we think some practices may facilitate better outcomes. It then goes further to the evidence we have about such relationships. The evidence helps us see which practices fall into which categories, such that we can say:

• “The following governance dimensions (what we measure as the generic base of our indicators) are non-negotiable institutions that we have evidence induce or force agents to produce welfare-maximizing outcomes for principals—in all cases”.
• “The following governance dimensions (what we measure as the contingent segment of our indicators) are negotiable institutions that we have evidence induce or force agents to produce welfare-maximizing outcomes for principals—in a contingent way (being relevant in specific contexts)”.
• “The following governance dimensions (what we measure as the cutting edge segment of our indicators) are untested institutions that we believe may induce or force agents to produce welfare-maximizing outcomes for principals (and we include to facilitate some testing)”.

In all three dimensions, we include the form because we theorize that it leads to better functionality and improved outcomes. I would like to say we are constructing indicators of more effective government...

Principles for developing governance indicators

I had an interesting meeting this morning to discuss a new initiative to devise indicators of good practice in observing human rights. It got me thinking about principles I see as important in devising indicators like this. Think of it as good practice for good governance indicators, to make sure there is some sense to the box ticking now dominating development.

 Here goes:

1. Be clear what you are devising the indicators for: Is it to hold organizations accountable? Is it to help organizations know how to do things better? These and other objectives are not necessarily mutually exclusive but not all indicator sets work to do both. If indicators are meant as guides on doing things better, don't use them to facilitate coercion. It could undermine their value.

2. Be clear about the indicator's normative bias: Indicators always have some kind of bias. Human rights groups develop indicators that are about observing human rights. The World Bank World Governance Indicators have a distinct neoliberal flavor (touting the norms of limited government). This should not be hidden, or considered a good or bad thing. It is, however, important for users to understand what the indicators are about.

3. Theorize the indicators: What I mean is simply that the designers should explain where each indicator measure, dimension or element comes from, and why measures are being incorporated (why do we think x is important as a contributor to good governance, for example). What is the theoretical argument underlying it? What is the empirical evidence supporting it? What examples can you provide as to why it is included? Too many indicators assess things that are not well theorized, leaving organizations in the dark as to whether they should comply and how.

4. Distinguish between dimensions in the indicator set: In most indicator sets there are issues being raised that one could theorize as non-negotiable good governance conditions. Others are interesting conditions or practices that are actually negotiable--working well in some conditions and not in others. A third set of practices could be considered innovative and new, but untested. Where indicator sets don't distinguish between the three, organizations face the same pressure to conform with non-negotiable, negotiable and cutting edge elements. This is problematic.

5. Assess de jure and de facto aspects of key practices: If an indicator set has an impact, organizations will face pressure to conform with required practices. This conformity can involve 'ticking boxes' by making changes in form--when in fact the change intended is in function. When this happens, indicators can foster a gap between form and function that undermines legitimacy of the prescribed practices. To counter this, measure aspects of the practice that involve de jure changes in form as well as aspects of the practice that involve de facto implementation. Then measure the gap between the two. 

6. Allow indicator analysis that has quantitative and qualitative dimensions: I don't like indicators that are just about numbers. A 1.5 this year and a 1.5 next year may not be the same. How do I know if there are side stories that need to be told? How do we learn about the fit of some indicator questions to context? I like indicators that provide both numbers and qualitative descriptions of the analysis underlying such. Contextual texture is important to understanding the indicator results.

Just my thoughts. I like Global Integrity data, Bertelsmann Transformation Index reports, and Economist Intelligence Unit reports because they satisfy quite a few of these criteria. 

Is the US federal budget process ungovernable?

I usually write exclusively on development and developing countries. I recently authored an article on the US federal budget process, however. The Association of Schools of Public Administration referenced it in discussing the debt ceiling debate/debacle.

I think there are some lessons about fitting reform to context in this piece. The lessons speak to the importance of theorizing both the reform and reform context to ensure you have the proper fit for a governance problem. When one theorizes the American context, focused on the logics driving political decisions, I come to wonder if any solution does fit. 

Let me get to the blog piece. Apart from some fun figures (see here) I also liked the blog title:  "The Budget Deal: A Repeat Phenomenon?" (http://aspanational.wordpress.com/2011/08/04/the-budget-deal-a-repeat-phenomena/).

 

It could also have been titled, "Governance Gaps in US Federal Budgeting: Gaping and Growing despite Reform." Or it could have kept the original article title, "The illogics of Federal Budgeting and Why Crisis Must Come." (http://onlinelibrary.wiley.com/doi/10.1111/j.1540-6210.2011.02352.x/abstract).  All of these titles show how well the story line of US federal budgeting reform relates to the problems of governance gaps in developing countries. Consider some of the observations I think are relevant:

• US budget reforms have been numerous in the past decades, just like they have in developing countries: Federal budgeting reforms since 1912 have included steps to introduce executive budgeting and fiscal rules, PPBS, ZBB and PBB and PART, the formalization of internal audit in agencies, process computerization, adoption of long term planning, and more. One could call the last century 'The period of great budgetary reform in the federal government' (Let's call it 'The Period'). 
• The US federal budget performance seems to have gotten worse in spite of these reforms, as it has in many developing countries: Interestingly, the country's fiscal sustainability problem has worsened in this 'Period'. Deficits have been more prevalent since these interventions than before. Federal spending has exceeded revenues in 75 percent of the years following the 1921 Act that introduced executive budgeting as a solution to spending problems, for instance, compared with 38 percent of the years between 1792 and 1920. Spending exceeded revenues in 16 of the 20 years after the 1990 Budget Enforcement Act (BEA) introduced measures to control deficits in the late 1990.  Deficits exceeded 3 percent of GDP in eight of the twenty years since 1990’s BEA, reaching such levels only 26 times in the prior 198 years (13 of which related to wars).
• Reforms that seem poorly fitted to the US context are constantly proposed, as they are in developing countries: Executive budgeting is an example. How can one have an executive driven budget as the 'solution' in a country where the Constitution gives Congress responsibility for budgeting? How can one think that fiscal rules will work in a country where politics is competitive, flexible and dynamic? 
• The country ends up being dependent on short-lived 'deals' as fixes to immediate problems, even though these offer no longer term solution--as one sees in many developing countries:  Short term fixes like the recent agreement between President Obama and the Congress are like band aids on a corpse. The agreement between President Clinton and the Republicans was not much better--a bandage, perhaps--and did not ensure the sustainability of budget surplus (a major short run achievement though that was). 
• In many ways reforms and agreements  are really just signals intended to suggest that those in authority are  trying to provide governance solutions--as they are in many developing countries: Beyond the signals, the interventions have limited effect--either holding for short periods only or penetrating the budgeting system superficially. Writing over twenty years ago, Aaron Wildavsky noted that we cannot even imagine what a budget reform would look like that penetrated further, disrupted politics, and forced real change where it is really needed.

These observations could be made about reform in many developing countries. It is often nothing more than a repeated game of signaling an intent to govern better while preserving the status quo. In the United States that status quo is a complex one that I think may prove ungovernable. I say this simply because it is informed by a set of principles or logics that interact in ambiguous ways.

• The first logic is reflected in the doctrine of Separation of Powers. A system of checks and balances allows each branch to restrain abuse by the others. Federal budget preparation and oversight is the responsibility of Congress, for example, but execution rests with the executive. The system purposefully fragments power, which inevitably fragments accountability as well. This makes developing a budget--or a budget reform--problematic. Who is accountable for the overspending? Who is accountable for developing the solution?
• Consider also the principle of limited government. Formalized in the Constitution, it reflects a deep seated American view that government should not encroach significantly in the lives of individuals. Applications in the Articles of Confederation relate directly to budgetary policies, limiting federal government powers of taxation. Many still believe these limits should hold, including civil society groups like the Club for Growth. Such groups see low taxes as appropriate fare, but this is a perspective that clashes with other ideals. 
• America is also a nation of people who value assertiveness and competition, for instance, and exhibit what Geert Hofstede calls a 'low uncertainty avoidance'. Put together, these values support aggressive, unfettered budgetary deal-making that are reinforced by regular competitive elections, expectations that representatives will bring home the pork, and the appropriateness of a vast structure of lobbyists set up to press for interests. These elements contribute to a logic of competitive competition that encourages politicians to act as snake-oil salesmen playing high-stakes games of chicken in the budgetary process. Winning is important, reflected in one's ability to get money for one's constituents. This clashes with the idea that government should be limited and constrained.
• Short-termism and a defining belief in the ‘American Dream’ and versions of American Exceptionalism further promote such behavior. These scripts generate expectations of America being ‘better’ than others, which arguably manifests in demands on government and on government expenditure. The demands come from multiple competitive representatives and their constituencies who all wish to be heard and expect to be satisfied with appropriations. Some expect and demand federal assistance in the aftermath of Hurricane Katrina and are willing to sponsor lobbyists to ensure this is forthcoming. Others demand military responses to provocation from Iran, Iraq and Afghanistan. Some work hard to ensure federal efforts are robust in stabilizing unstable financial markets. Additional interests call for support to small business, help for first-time home buyers, and much much more. The scripts suggest that an exceptional country needs exceptional government that can do all these things.  And still be limited. Budgets should accommodate it all, but taxes must still be low.

I am not sure how one crafts a governance solution in the face of these different values and logics. I am sure that the task is more complex that introducing a technical fix--like a fiscal rule (the balanced budget amendment, for instance), or mechanisms that strengthen the executive, or performance metrics, or program based budgeting.

Whatever the solution, reformers must have a theory about American government that takes such values into consideration--identifying the hole in which the reform peg must fit.

To be real and accommodate a fit, the debate about reform must involve open political contestation and discussion about the logics driving budgeting--the social and cultural ideas of what American government is, how it gets its money, the social trade-offs of providing or not providing services, and such. I think real crisis may be the only thing that fosters such debate. Until this happens, I fear that the US federal budget process will continue to be ungovernable--and get worse over time. Regardless of who holds the Presidency or what band aid deals they can facilitate.

In many senses, the situation in US federal budgeting may be more complex than that in developing countries! In both, however, we need to ensure that future reforms are better fitted to context--and to do this we need to theorize the reforms and context better. 

Theorizing case studies: check out ISS at Princeton

Princeton's new Innovations for Successful Societies is an interesting initiative. Jennifer Widner, Laura Bacon (a former student) and others are developing materials one can use to learn about innovative solutions to developmental problems. These include case studies, interviews and more. What I like about the materials is that they don't just present a 'best practice' idea or heroic story of success. They provide enough information to enable readers in theorizing about these practices and innovations: you can listen to the protagonists describing why the innovation emerged, what process it emerged from, how people came on board to make it happen, what kinds of political and operational demands it posed, and so forth. If only all development organizations were required to provide background stories like these for the innovations they so easily reproduce everywhere. One wonders if ministers of finance in French speaking African countries would so willingly agree to Medium Term Expenditure Framework reforms if they knew where these ideas came from (select advanced Anglophone countries like Australia), what kind of process MTEFs emerged from in these settings, what kind of political context they demand, what capacities they require etc. Ditto for just about every good, better or best practice one can think of. We need more of what ISS is providing: materials to help theorize what makes a best or innovative practice best or innovative and to assist us in thinking about whether these practices fit into new contexts...or what will be required to facilitate the fit. Visit their web site at http://www.princeton.edu/successfulsocieties/

Overcoming governance gaps from unwanted external best practice reforms

My blogs so far suggest that there is a growing governance gap created from many governance reforms. This is largely because reforms introduce best practice formal interventions that ultimately are not implemented. I suggest we should do things differently. What does that mean? Today I suggest a basic first step: ensuring reforms are 'wanted' by constituencies that have to do the implementation. Note I am not talking about 'ownership' in some vague sense. 'Want' is more direct, focused, immediate.  And the underlying story with many governance reforms is that they are not wanted by those required to ultimately implement the change. For good reasons.

 

Let me frame this idea with a short story and then get to the point, and some ideas on generating 'want' in a reform context.

The story is about adopting international accounting standards. These are an important 'best practice' element of the good governance and doing business regimes. Developing countries are encouraged to adopt them, with multilaterals and bilaterals sponsoring projects focused on such in locales across the less affluent parts of earth.

Mir and Rahaman write about the adoption of  international accounting standards (IAS) in Bangladesh (http://www.emeraldinsight.com/journals.htm?articleid=1528645&show=pdf).

They describe “evidence of immense pressure that major international donor/lending institutions put on the Bangladeshi government and professional accounting bodies to adopt IASs not only to provide credibility to foreign investors but also ensure that accountability arrangements with lending/donor agencies are tight enough. Clearly, the government and other institutions in Bangladesh have very little option (if any at all) because of the country's high dependence on aid…[even though]… a wholesale adoption of IASs constitutes a quick-fix that may not necessarily be in the long-term interest of Bangladesh and its accounting institutions.”

The description raises an important question of whether (and how regularly) donor insistence on best practice reform undermines local ownership--and the process of local problem solving and solution finding. Other descriptions in the story tie such questions to additional concerns about how well these imposed reforms are eventually implemented. The authors note that, "The perceived undemocratic nature of the adoption process appears to be creating and enhancing conflict among various constituencies, resulting in very low compliance with these standards."

The gap that these authors identify--between an imposed best practice reform design and an ultimate practice emerging from such--seems to emerge partly because the process of finding and fitting solutions is bypassed in favor of a solution coming from other countries. In their paper, the authors note that one reason this kind of imposed solution fails is simply that it is seen as being from the outside...emergent from an external community and not in response to local problems and through locally legitimate processes. They are not wanted internally, by the folks who ultimately have to implement and use them, digest them, allow them to interrupt their day-to-day work, change the way they think and contract and value things.

It reminds me of Richard Rose's admonition that reforms should involve “relevant” practices instead of “best” practices, where “Relevant practices are both politically acceptable and within the resources of government.” He argues that, “While relevance is no guarantee of a programme’s success, it avoids the certainty of failure due to the promotion of programmes that governments do not want or lack the resources to implement.”

While I cannot vouch for the accuracy of Mir and Rahaman's view that IAS adoption is failing in Bangladesh because it is unwanted by implementers, the story rings true enough to reproduce it. And in the story I see lessons for doing governance reform better--making it 'relevant' in the words of Richard Rose.

First, instead of focusing on reproducing solutions, reforms should emphasize  solving problems. The literature on change regularly emphasizes the importance of problems, crises and the like as motivatorsof change. Agents generally do not want change but can get to a 'want' position if they see that the change is needed to alleviate a threat, solve a problem, assist their survival, maybe even make them perform better.

Second, do no support reform design processes that lead to quick, lazy, narrowly engaged plucking of external best practices as the logical solution. If you want to implement a reform you need to spend time finding a solution that fits the context. This requires engaging with those who will implement, having them contribute to a shared understanding of why reform is needed and a shared process of experimenting with local ideas and scanning the globe for 'relevant' ideas. The process of finding and fitting is absolutely vital if one truly wants an implemented reform. This process is the only way one gets a clear perspective on political realities (not possible through an externally written political economy analysis) and local capacities to implement.

Third, create a high hurdle for external 'best practice' ideas to clear before adopting them. Evidence suggests that best practice ideas travel well from place to place where they are contextualized. Rose describes this as being "altering the original model to operate in a different administrative and political setting” noting that, “When altering a prototype, policymakers are not abandoning what they have learned from better practices elsewhere; they are applying a lesson learned elsewhere in ways relevant in their own national setting.” Some basic questions should always be asked where a best practice is being reproduced: What is the lesson learned elsewhere that you are replicating? How has the practice been altered to match your political realities? How has the practice been altered to match your administrative realities? This requires having a theory about the best praxctice (understanding what it is, and how it works) and being able to see if that theory works in your context--what the literature calls 'theorization'.

These three steps require that external agents take a very different perspective on the reforms they introduce. It requires a change in focus from (i) ready-made solutions to sticky and under-understood problems, (ii) quick answers with narrow groups to longer processes of asking questions in broader constituences, and (iii) easy acceptance of best practices to more skepticism about whether best practices are relevant. I am sure these ideas will not guarantee success, but as Rose says, they will help avoid certainty of failure when it comes to implementing reforms people just do not want.

What do you think?

The governance fashion show

Measuring the gap between form and function: Global Integrity data

I am constantly on the look out for measures of the gap between forms and functions of good governance. This week I have been intrigued by the Global Integrity data on the topic (http://www.globalintegrity.org/report). Global Integrity Reports are described as follows:

"The Global Integrity Report is a tool for understanding governance and anti-corruption mechanisms at the national level. Written by local researchers and journalists, the Report is characterized by an innovative, award-winning research methodology; a robust peer review process; and start-to-finish transparency."

The reports show scores for a range of countries on the quality of laws and the implementation of the laws. It also shows the gap between laws and implementation--how much better form is than function. Here are the highlights and some important observations:

First, consider the following table, showing countries with the lowest implementation gaps.

• All countries have implementation gaps (legal forms are always better than functional implementation)
• All of the wealthier countries in the sample have implementation gaps smaller than 25.
• Very few developing countries have implementation gaps smaller than 25.

Global Integrity Index 2008	Country Score	In Law (L)	In Practice (P)	Implementation Gap (L - P)
Italy	79	82	73	9
Poland	88	92	82	9
Kazakhstan	76	80	69	11
United States (2007)	85	90	78	12
Japan	83	91	76	14
Korea	88	96	82	15
Bulgaria	87	95	79	16
South Africa	79	86	70	16
Hungary	77	85	67	18
Turkey	69	78	59	19
Macedonia (FYROM)	76	86	65	21
Chile	77	87	66	21
Lithuania	74	85	63	22
Canada	80	90	68	22
Morocco	48	58	35	22
Ghana	67	77	55	22
Kenya	69	79	56	23

Second, consider the quality of laws in some of the wealthier countries above, compared with those in the table below.

• A number of developing countries have better scores on their laws than wealthier countries like Italy. Colombia has better scores on laws than Chile.
• Most developing countries have large implementation gaps as well. Colombia's is nearly twice that of Chile's.
• The best laws in the world are in Uganda (100 out of 100). Uganda also has the highest implementation gap (51).
• Uganda's experience is similar to that of many other developing countries: High scores on laws and really low scores on implementation dominate.

Global Integrity Index 2008	Country Score	In Law (L)	In Practice (P)	Implementation Gap (L - P)
Cambodia	46	58	30	28
Indonesia	69	83	55	28
West Bank	53	68	39	29
Serbia	62	75	45	30
Egypt	54	67	38	30
Angola	41	56	26	30
Cameroon	55	70	39	31
Nigeria	64	79	48	31
Bangladesh	68	84	52	31
Ecuador	60	75	43	32
Nepal	64	81	48	33
Belarus	66	81	48	33
China	59	74	41	33
Romania	80	97	64	34
Russia	69	89	52	36
Georgia	59	78	41	37
Colombia	71	90	53	37
Philippines	71	91	54	38
Nicaragua	58	76	39	38
Argentina	70	90	51	39
Moldova	68	88	48	40
Somalia	35	54	14	41
Montenegro	56	77	36	42
Pakistan	72	94	52	42
Yemen	46	66	23	43
Iraq	53	75	32	43
Kyrgyz Republic	64	87	42	45
Bosnia and Herzegovina	63	87	41	46
Uganda	72	100	49	51
Azerbaijan	61	87	36	51
Ethiopia	56	82	31	51

Third, some questions:

• Is it better to be Uganda with 100 on laws and 49 on implementation, or Italy with 82 on laws and 73 on implementation?
• Should Somalia's governance reforms (if and when they start) focus first on improving the quality of laws (from 54) or implementation of existing laws (currently 14)?
• Do externally influenced governance reforms have any impact on implementation? Or are their effects limited to improving laws? What are donors doing to close the implementation gap?

My worry with the story emerging from these numbers is simply that governance reforms have created a gap between form and function and continue to do so. There are too many countries that have 'benefited' from costly governance reforms but still have extremely low implementation of laws (many of which were introduced via reforms). 

The data here is quite provocative in telling this story, and raises questions about what reforms have achieved in countries as varied as Cambodia, Ethiopia, Azerbaijan, Egypt and Georgia. Note that the IFC has awarded some of these countries awards as best reformers for Doing Business. (How should we tell this story? Do good 'Doing Business' reforms involve passing great laws you can't implement? Is the goal to look better without being better? Or do we really believe that in time the gap will close?) 

Other examples of data showing the gap will be welcomed. As will any examples of the 'development as functional improvement regardless of form'. As will any comments that tell me that this is all much ado about nothing.

 

Public sector reform, good government and the quest for functional thinking

I thought I would post this link to a transcribed interview I did with our media folks in response to my 2010 Governance article 'Good Government Means Different Things in Different Countries'.

http://www.hks.harvard.edu/news-events/publications/insight/international/andrews

My kids laughed at the video interview...