MattAndrews

Can governance indicators focused on outcomes still be about governance?

MEASURES SHOULD CAPTURE THE SYSTEMS

BUT ONLY THOSE THAT HELP PRODUCE THE OUTCOMES.

A reader, David, asked an important question in response to my blogs about governance and outcomes. Essentially, he noted that the world of development has come a long way in recognizing that governance matters. Nowwe have a focus on administrative structures, rules of the game and the like, under the title 'Governance'. If we start saying that governance must be about outcomes, could we lose this focus...

A great question (which I hope I have represented fairly). Here are my thoughts on it.

While my past work may seem to suggest I see outcomes as measures of governance themselves, I do not. Outcomes come into the picture because governance must be about outcomes. So, governance is about the institutions, structures, systems etc. that shape authority mechanisms principals use to ensure that agents produce outcomes that lead to welfare gains for the principals.

Consider the figure below. The indicator should measure institutions, structures, etc. that we believe help achieve outcomes (or otherwise enhance the welfare of principals). In the case of governance in the child health arena, the indicator would measure whether a country has systems etc. that we believe facilitate the exercise of authority in such a way as to maximize the survival rate of under five children. It would aggregate many components for sure, to be a complex measure of the governance structures deemed necessary to keep kids alive. But it would not be a vague summary indicator of something that someone somewhere can postulate about.

Note that the outcome is central here, and the test is whether the indicator, and its components, reflect on the outcome. See the discussion on construct validity from a few days ago. You cannot assess construct validity unless you know what the governance focus is. And you cannot vouch for the construct validity of your indicator unless you have some way of testing whether its components are actually related to the outcome you care about.

 
 If one thinks of governance indicators--and the relationship between governance, indicators and outcomes--in this way, it should alleviate concerns that the focus on institutions, systems, strcutures etc. will be lost in the future. They are still front and center. But they cannot be introduced in a stand-alone fashion. In this way of thinking they are chosen and included in the indicator and in our lexicon of reform because we believe that they help improve the way society produces outcomes.

I have used the word 'believe' above to be a little open and fluffy. I would love to see it replaced with 'know' or 'believe, on the basis of theory and evidence'. All the systems and structures and institutions we include when constructing an indicator should be things we believe, on the basis of theory and evidence, facilitate the exercise of authority by governments (and other entities) in such a way as to produce outcomes that advance the welfare of citizens. Sorry such a mouthful.

If we had this kind  of theoretical, evidence based story, we would not weaken the governance agenda--or dilute it. We would surely strengthen it. Being able to say that having a multi-year budget in the health sector is empirically associated with having better  child health outcomes is surely the best way of ensuring that these reforms gain traction. But we cannot say this. So, we depend upon the development community accepting that multi-year budgets have some intrinsic, stand-alone value...things to be done even if we don't know their value. I cannot really think of too many governance practices to put in this category.  

I know I have not addressed the fullness of David's question, which asks about the impact of other factors on outcomes, potential interactions of these factors with governance reforms, etc. I hope  the commentary following this--and future posts--will go some way to do this. In the mean-time, as usual, what do you think about the perspective?

 

 

 

Stop saying 'Country x has a governance score of y': It makes no sense

If we focus on Governance Outcomes, we need to focus indicators on specific Organizational Fields: Whole of government indicators—or government only indicators—make little sense

Last week I did a blog post on my ideas about linking governance indicators to outcomes. A range of people have responded positively to the idea, though most via email. Try writing comments in and let’s get a conversation going on the topic.

Anyway, I thought I would provide some more discussion on what I see as the implications of focusing governance indicators on outcomes and share specific examples of how this may be done. I will do this in the next few days. I aim to be a little provocative and get some responses…so fire away, please!

What is the appropriate unit of analysis in governance?

To kick things off, I thought it might be useful to think about the way in which an outcomes focus might challenge the unit of analysis in governance discussion and measurement. Given the theoretical approach I outlined last week, it is my belief that governance can, by definition, be assessed only in the context of specific fields in which outcomes are produced. I mean something specific about ‘field’ here, building on the definition in organizational theory: “The particular set of organizations that, in the aggregate, constitute a recognized area of institutional life” (see DiMaggio & Powell, 1983). The field includes “suppliers, resource and product consumers, regulatory agencies, and other organizations” involved in getting things done in a particular functional area.

Such a system is an appropriate unit of analysis for governance research because it is the domain in which outcomes are produced (yes, a bit repetitive but important). Note a few things. First, The Government is in a sense too large a unit of analysis for a governance indicator. Governments are engaged in multiple fields producing multiple outcomes. Governance indicators that pretend to say something about an entire government are problematic from this perspective. Do they refer to governance as it relates to child health outcomes or to educational outcomes or to safety and security outcomes. Or do we think that one set of questions gives valid information about governance issues relating to all three? Second, The Government is in a sense too small a unit of analysis for a governance indicator. Fields involve multiple players, particularly principals and agents, who are constantly engaged in allocating, receiving and managing authority to do specific things, with particular functional results expected. The quality of governance is reflected in how well the entire field engages to produce these results. National governments (the focus of many governance indicators) are usually only one player in any given field. We need to go beyond measuring governance within and through national governments to link outcomes and governance indicators.

The implication, therefore, is that it really makes no sense to have a ‘governance score’ for governments in Mauritius or Botswana or China or Bolivia. We should have indicators for the fields in which outcomes are produced in these countries, thus ‘the child health sector in Mauritius’ and ‘the safety and security domain in Botswana’ etc.

If we focus on fields, we need to expect variation and complexity

One should note some additional important features of organizational fields (as discussed by Richard Scott in his classic text on organizations). They come in different sizes and with different relational structures (often driven by resource access). This can yield fields that are placid and do not change very much with time, disturbed or turbulent and subject to many shocks, randomized, reactive or clustered (See Emery & Trist, 1965). In each variation the relational and normative governance challenges may be different, and one should expect very different governance processes to have become embedded. One should also expect different structural and process solutions in each.

Organizational fields also reflect different authority typologies—or ways in which authority is cognitively and normatively influenced. In some places one should expect social choice to determine who has authority and how it is exercised. In others authority will be more coalitional, and in others it will be federative. Authority structures could also be unitary. Once again, the different authority typologies will probably yield different solutions to governance problems. These differences also emerge because of physical differences in fields—some being geographically centered and some being dispersed. A central organizing mechanism may make the first work. A decentralized mechanism may be appropriate to facilitate best outcomes in the second.

Building on this idea, governments can be expected to play different roles in different fields, largely because the sets of players and institutional logics differ in different fields, rates of change in fields vary, and demands on controlling and shaping behavior are not always the same. The education and health care fields look very different in the USA, for example. The roles of federal, state and local governments differ between these two fields, as does the relative size and scope of engagement by private and non-profit players.

These differences matter because public sector governance is about the exercise of authority by government agents, on behalf of citizen principals, to maximize the welfare of those principals; and all of these elements are field-specific. In some fields citizens delegate authority to government to regulate, in other fields to provide services directly.

So, governance indicators should focus on fields…where outcomes emerge…but with flexibility to reflect context.

As a result of the expected variation in structures between fields within and across countries, and the implications of such for governmental roles, we believe that governance indicators should be defined by the field in which outcomes are produced, players can be identified, and authority structures delineated. This allows governance indicators to look like specific rather than general health measures. Prominent governance indicators pay indirect homage to this, including the WGI, which separates “government effectiveness” from “rule of law” in its components; but these are categories of interest and not defined areas in which authority is exercised—the key driver of the governance definition. As a result, the measures become difficult to interpret. Practitioners, policy-makers and researchers would be better served with indicators focused on specific outcomes in specific fields that make immediate sense and can be acted upon.

I see the child health arena as one field, with identifiable outcomes and navigable organizational fields—where one can identify key agents, organizations, organizing logics and systems connecting and structuring such (even though these do vary across countries and over time, shown in studies on changes to health care fields). Child health is the complex product of multiple players bringing multiple inputs to bear in these fields, under specific authority structures, with different effects in different settings. (Other field-based studies include Beaumont, 2003, on anti-poverty systems and Parto, 2005, on waste subsystems in the Netherlands and the UK and Knill & Lehmkuhl, 2002, on regulation of the internet.)

Making good good governance indicators--the construct validity question

IF YOU HOPE PEOPLE WILL DO WHAT YOU MEASURE

MAKE SURE YOU MEASURE WHAT YOU WANT THEM TO DO

I was asked to give some thoughts on construct validity in governance-type indicators. Here are some thoughts and references to sources that help me in developing such thoughts.

First, I would start with the critique Melissa Thomas provides of the world governance indicators. She asks simply, 'what do they measure' and in so doing raises a major construct validity question, with two dimensions: (1) is there underlying theory about how the things measured in the indicator affect the outcomes designers care about? (2) is there evidence that the things being measured are related to these outcomes?

See her full paper at, http://www.sais-jhu.edu/sebin/q/r/What%20Do%20the%20Worldwide%20Governance%20Indicators%20Measure.pdf

See also the response by Kaufmann, Kraay and Mastruzzi. http://info.worldbank.org/governance/wgi/pdf/kkmresponseejdr2final.pdf They argue a blend of 'construct validity isn't relevant' and 'you don't prove we have a construct validity problem anyway.'

Whatever you think of Thomas's paper and the response, the issue is simply whether the thing being measured correlates with the thing it is meant to be an indicator of. As with simple ideas, however, there is more complexity to consider. Indeed, there are multiple dimensions of this construct validity test.

1. Construct validity requires content validity: does the measure reflect on all facets of the construct? Lawshe proposed a way of testing this, suggesting that panels of experts be given a list of all the questions one intends to ask about a construct, and asked whether each question relates to an essential, useful bot not essential, or Unnecessary aspect of the construct. They could also propose questions that have been left out.

With enough feedback, one can then get an idea about the core content of a construct. If a question is roundly endorsed as 'essential' for example it would be part of the non-negotiable part of the indicator. If it were anything less, one would have to decide if the question was a negotiable, cutting edge or unnecessary element in the indicator.

I have some cautionary notes. First, make sure the experts explain their results- providing theoretical reasoning to back their argument. Make sure they are not just making a normatively biased call. Second, make sure your panel of experts has a good understanding of the construct you are trying to measure and why you are measuring it (see yesterday's discussion on indicators and outcomes). Third, make sure your panel is not biased, reflecting only one normative opinion of the construct. This is one of the problems I think the world governance indicators and other governance indicators has encountered. Kurtz and Schranck and others simply say (with justification) that these indicators reflect neoliberal bias and the perspectives of private sector operators. Fourth, I think you need to go beyond lawshe to actually test the correlation with questions and outcomes, continually, so that over time you build evidence in support of your content decisions.

I have included a couple of links to studies that do this ( in areas like education and health care).

• This is a citation to a study on Ccnstruct validity of 3 indicators of psychological distress in relation to perceived health by Koopmans and Lamers. http://www.mendeley.com/research/assessing-construct-validity-three-indicators- psychological-distress-relation-perceived-health-physical-illness/ The study asks about how each indicator relates to the outcome in question. It uses regression analysis to test the relationship. Combining theoretical and empirical work like this helps foster content validity.
• See also Willis, Stoelwinder and Harris. Their work focuses on interpreting indicators in health care. The authors test the assumption that quality indicators reflect on the quality of care in hospitals. http://www.ncbi.nlm.nih.gov/pubmed/18603538
• See also Mathews, Hackett and Pennell on construct validity of accountability measures and performance indicators in schools.http://www.learningpt.org/sipsig/mathews.html

2. Convergent validity is another aspect of construct validity. This asserts that a measure parading as an indicator of x should converge with other indicators of x. The Tests here are quite simple. Look at indicators that purport to reflect on the same construct and see if they are correlated. An example is Townsend and Kaiser who looked at indicators of fruit intake and psychosocial health. http://www.ncbi.nlm.nih.gov/pubmed/16029687

The world governance indicator folks have done this, and argue emphatically that their indicators correlate highly with other indicators of constructs implied in the broad idea of good governance. There are some problems with this, though. First, the 'good governance' concept is so broadly presented in this work that one is bund to found overlap. Second, the world governance indicators are calculated on the basis of many of these 'related measures'. So, one should expect correlation. But this is mathematical convergence, not theoretical convergence.

Another approach to testing construct validity of a new indicator involves taking the new indicator and testing it side by side with a prior indicator that is acknowledged as having some construct validity already. I like Burke, Burke and Crowder in this regard. http://aei.sagepub.com/content/31/4/1.abstract

The article looks at how student skills of a new indicator of early literacy scores correlated with an older indicator of a similar construct that was already known to predict Later reading efficiency.

3. Discriminant validity is the third aspect I will draw on now. It notes that a measure meant to indicate x should not indicate y, z and other constructs. It is kind of like the opposite of convergent validity in that you should be able to see that it does not correlate with (and rather diverges from) indicators of other things.

This is a dimension where world governance indicators have struggled. The argument of people like me is that these indicators correlate with measures of economic well being, such that it is hard to say they are not just another way of measuring this. Neumann and Graeff suggest this is one of the questionable parts of the good governance indicators they look at. http://www.springerlink.com/content/t417158753238u2q/

An empirical consequence of this problem is that governance indicators typically correlate with other economic, social and political indicators. This means that there could be measurement problems with regressions with governance indicator scores as explanatory variables and democracy, GDP level, etc also as explanatory variables.

Governance indicators and governance outcomes: How do they relate?

 Alan Hudson asked yesterday how I relate my comments about constructing governance indicators to prior work I have done that argues governance is all about producing outcomes. I replied in comments, but let me draw a bit from a 2010 paper I wrote on this subject (in Oxford Development Studies). Think about this as the way I am conceptualizing the process of 'theorizing' governance indicators.

We need to have some idea what governance is, before starting. In a general sense, most see this as the exercise of authority. Which requires us asking: who the authority is exercised by, on behalf of, and for what reason?  Answers to such questions will help guide us to appropriate measures of the phenomenon.

I find the literature on corporate governance in publicly traded companies useful in addressing these questions. Tirole defines corporate governance in this context as “the design of institutions that induce or force management to internalize the welfare of stakeholders”. Consider the basic elements of governance implied in this definition: (1) it is focused on how mechanisms (institutions) regulate (2) the way that authority is exercised by one set of agents (managers) (3) who act on behalf of a broad group of principals (stakeholders) (4) with the goal of maximizing the welfare of these principals (stakeholders).

Combining these elements, it can be seen that good governance emerges when specific agents exercise delegated authority in such a way that their specific principals enjoy improved welfare.

This definition has parallels in the political science and public management literature. Kooiman’s characterization of governing, for example, points to “the totality of interactions, in which public and private actors participate, aimed at solving societal problems or creating societal opportunities”. Hill & Lynn describe public sector governance from a management perspective, as “Regimes of laws, rules, judicial decisions, and administrative practices that constrain, prescribe, and enable the provision of publicly supported goods and services through associations with agents in public and private sectors”.

The idea of delegated authority emerges across these definitions, as does the focus on outcomes as the purpose of delegated authority (maximizing stakeholder welfare, “solving societal problems or creating societal opportunities” and ensuring the “provision of publicly supported goods and services.”)

In the public sector context we are obviously dealing with governments as the agents to whom authority is delegated, by citizens (as principals), with the explicit goal of maximizing various kinds of social welfare (as the outcome). Governments can use this delegated authority in many ways: to garner and allocate resources, build capacities (human and physical), regulate via laws or force, and so forth. Governance is superior when authority is exercised in such a manner that social welfare is maximized and governance is inferior when authority is exercised in a manner that does not maximize welfare.

In this way of thinking, governance indicators should assess the degree to which organizations structure their authority relationships to produce outcomes--not just the degree to which they comply with a generic set of best practice processes.

Some roles, interactions and mechanisms might be more effective than others in facilitating better governance outcomes. Some may have generic import to all organizations producing the outcome, some may be relevant contingent on specific contexts, others may be cutting edge ideas we think will have value but are still untested.

These kinds of practices can only be identified after we look at specific organizational fields (e.g. education or health care), measure the outcomes of interactions in these fields, and consider contextual differences between countries. Function must lead form in this respect.

We need to theorize our indicators to make this happen. This starts with us having clear ideas about the outcomes we are looking at (and our normative approach to thinking about such) and the fields in which we are focused. It moves to having some theoretical idea about why we think some practices may facilitate better outcomes. It then goes further to the evidence we have about such relationships. The evidence helps us see which practices fall into which categories, such that we can say:

• “The following governance dimensions (what we measure as the generic base of our indicators) are non-negotiable institutions that we have evidence induce or force agents to produce welfare-maximizing outcomes for principals—in all cases”.
• “The following governance dimensions (what we measure as the contingent segment of our indicators) are negotiable institutions that we have evidence induce or force agents to produce welfare-maximizing outcomes for principals—in a contingent way (being relevant in specific contexts)”.
• “The following governance dimensions (what we measure as the cutting edge segment of our indicators) are untested institutions that we believe may induce or force agents to produce welfare-maximizing outcomes for principals (and we include to facilitate some testing)”.

In all three dimensions, we include the form because we theorize that it leads to better functionality and improved outcomes. I would like to say we are constructing indicators of more effective government...

Principles for developing governance indicators

I had an interesting meeting this morning to discuss a new initiative to devise indicators of good practice in observing human rights. It got me thinking about principles I see as important in devising indicators like this. Think of it as good practice for good governance indicators, to make sure there is some sense to the box ticking now dominating development.

 Here goes:

1. Be clear what you are devising the indicators for: Is it to hold organizations accountable? Is it to help organizations know how to do things better? These and other objectives are not necessarily mutually exclusive but not all indicator sets work to do both. If indicators are meant as guides on doing things better, don't use them to facilitate coercion. It could undermine their value.

2. Be clear about the indicator's normative bias: Indicators always have some kind of bias. Human rights groups develop indicators that are about observing human rights. The World Bank World Governance Indicators have a distinct neoliberal flavor (touting the norms of limited government). This should not be hidden, or considered a good or bad thing. It is, however, important for users to understand what the indicators are about.

3. Theorize the indicators: What I mean is simply that the designers should explain where each indicator measure, dimension or element comes from, and why measures are being incorporated (why do we think x is important as a contributor to good governance, for example). What is the theoretical argument underlying it? What is the empirical evidence supporting it? What examples can you provide as to why it is included? Too many indicators assess things that are not well theorized, leaving organizations in the dark as to whether they should comply and how.

4. Distinguish between dimensions in the indicator set: In most indicator sets there are issues being raised that one could theorize as non-negotiable good governance conditions. Others are interesting conditions or practices that are actually negotiable--working well in some conditions and not in others. A third set of practices could be considered innovative and new, but untested. Where indicator sets don't distinguish between the three, organizations face the same pressure to conform with non-negotiable, negotiable and cutting edge elements. This is problematic.

5. Assess de jure and de facto aspects of key practices: If an indicator set has an impact, organizations will face pressure to conform with required practices. This conformity can involve 'ticking boxes' by making changes in form--when in fact the change intended is in function. When this happens, indicators can foster a gap between form and function that undermines legitimacy of the prescribed practices. To counter this, measure aspects of the practice that involve de jure changes in form as well as aspects of the practice that involve de facto implementation. Then measure the gap between the two. 

6. Allow indicator analysis that has quantitative and qualitative dimensions: I don't like indicators that are just about numbers. A 1.5 this year and a 1.5 next year may not be the same. How do I know if there are side stories that need to be told? How do we learn about the fit of some indicator questions to context? I like indicators that provide both numbers and qualitative descriptions of the analysis underlying such. Contextual texture is important to understanding the indicator results.

Just my thoughts. I like Global Integrity data, Bertelsmann Transformation Index reports, and Economist Intelligence Unit reports because they satisfy quite a few of these criteria. 

Is the US federal budget process ungovernable?

I usually write exclusively on development and developing countries. I recently authored an article on the US federal budget process, however. The Association of Schools of Public Administration referenced it in discussing the debt ceiling debate/debacle.

I think there are some lessons about fitting reform to context in this piece. The lessons speak to the importance of theorizing both the reform and reform context to ensure you have the proper fit for a governance problem. When one theorizes the American context, focused on the logics driving political decisions, I come to wonder if any solution does fit. 

Let me get to the blog piece. Apart from some fun figures (see here) I also liked the blog title:  "The Budget Deal: A Repeat Phenomenon?" (http://aspanational.wordpress.com/2011/08/04/the-budget-deal-a-repeat-phenomena/).

 

It could also have been titled, "Governance Gaps in US Federal Budgeting: Gaping and Growing despite Reform." Or it could have kept the original article title, "The illogics of Federal Budgeting and Why Crisis Must Come." (http://onlinelibrary.wiley.com/doi/10.1111/j.1540-6210.2011.02352.x/abstract).  All of these titles show how well the story line of US federal budgeting reform relates to the problems of governance gaps in developing countries. Consider some of the observations I think are relevant:

• US budget reforms have been numerous in the past decades, just like they have in developing countries: Federal budgeting reforms since 1912 have included steps to introduce executive budgeting and fiscal rules, PPBS, ZBB and PBB and PART, the formalization of internal audit in agencies, process computerization, adoption of long term planning, and more. One could call the last century 'The period of great budgetary reform in the federal government' (Let's call it 'The Period'). 
• The US federal budget performance seems to have gotten worse in spite of these reforms, as it has in many developing countries: Interestingly, the country's fiscal sustainability problem has worsened in this 'Period'. Deficits have been more prevalent since these interventions than before. Federal spending has exceeded revenues in 75 percent of the years following the 1921 Act that introduced executive budgeting as a solution to spending problems, for instance, compared with 38 percent of the years between 1792 and 1920. Spending exceeded revenues in 16 of the 20 years after the 1990 Budget Enforcement Act (BEA) introduced measures to control deficits in the late 1990.  Deficits exceeded 3 percent of GDP in eight of the twenty years since 1990’s BEA, reaching such levels only 26 times in the prior 198 years (13 of which related to wars).
• Reforms that seem poorly fitted to the US context are constantly proposed, as they are in developing countries: Executive budgeting is an example. How can one have an executive driven budget as the 'solution' in a country where the Constitution gives Congress responsibility for budgeting? How can one think that fiscal rules will work in a country where politics is competitive, flexible and dynamic? 
• The country ends up being dependent on short-lived 'deals' as fixes to immediate problems, even though these offer no longer term solution--as one sees in many developing countries:  Short term fixes like the recent agreement between President Obama and the Congress are like band aids on a corpse. The agreement between President Clinton and the Republicans was not much better--a bandage, perhaps--and did not ensure the sustainability of budget surplus (a major short run achievement though that was). 
• In many ways reforms and agreements  are really just signals intended to suggest that those in authority are  trying to provide governance solutions--as they are in many developing countries: Beyond the signals, the interventions have limited effect--either holding for short periods only or penetrating the budgeting system superficially. Writing over twenty years ago, Aaron Wildavsky noted that we cannot even imagine what a budget reform would look like that penetrated further, disrupted politics, and forced real change where it is really needed.

These observations could be made about reform in many developing countries. It is often nothing more than a repeated game of signaling an intent to govern better while preserving the status quo. In the United States that status quo is a complex one that I think may prove ungovernable. I say this simply because it is informed by a set of principles or logics that interact in ambiguous ways.

• The first logic is reflected in the doctrine of Separation of Powers. A system of checks and balances allows each branch to restrain abuse by the others. Federal budget preparation and oversight is the responsibility of Congress, for example, but execution rests with the executive. The system purposefully fragments power, which inevitably fragments accountability as well. This makes developing a budget--or a budget reform--problematic. Who is accountable for the overspending? Who is accountable for developing the solution?
• Consider also the principle of limited government. Formalized in the Constitution, it reflects a deep seated American view that government should not encroach significantly in the lives of individuals. Applications in the Articles of Confederation relate directly to budgetary policies, limiting federal government powers of taxation. Many still believe these limits should hold, including civil society groups like the Club for Growth. Such groups see low taxes as appropriate fare, but this is a perspective that clashes with other ideals. 
• America is also a nation of people who value assertiveness and competition, for instance, and exhibit what Geert Hofstede calls a 'low uncertainty avoidance'. Put together, these values support aggressive, unfettered budgetary deal-making that are reinforced by regular competitive elections, expectations that representatives will bring home the pork, and the appropriateness of a vast structure of lobbyists set up to press for interests. These elements contribute to a logic of competitive competition that encourages politicians to act as snake-oil salesmen playing high-stakes games of chicken in the budgetary process. Winning is important, reflected in one's ability to get money for one's constituents. This clashes with the idea that government should be limited and constrained.
• Short-termism and a defining belief in the ‘American Dream’ and versions of American Exceptionalism further promote such behavior. These scripts generate expectations of America being ‘better’ than others, which arguably manifests in demands on government and on government expenditure. The demands come from multiple competitive representatives and their constituencies who all wish to be heard and expect to be satisfied with appropriations. Some expect and demand federal assistance in the aftermath of Hurricane Katrina and are willing to sponsor lobbyists to ensure this is forthcoming. Others demand military responses to provocation from Iran, Iraq and Afghanistan. Some work hard to ensure federal efforts are robust in stabilizing unstable financial markets. Additional interests call for support to small business, help for first-time home buyers, and much much more. The scripts suggest that an exceptional country needs exceptional government that can do all these things.  And still be limited. Budgets should accommodate it all, but taxes must still be low.

I am not sure how one crafts a governance solution in the face of these different values and logics. I am sure that the task is more complex that introducing a technical fix--like a fiscal rule (the balanced budget amendment, for instance), or mechanisms that strengthen the executive, or performance metrics, or program based budgeting.

Whatever the solution, reformers must have a theory about American government that takes such values into consideration--identifying the hole in which the reform peg must fit.

To be real and accommodate a fit, the debate about reform must involve open political contestation and discussion about the logics driving budgeting--the social and cultural ideas of what American government is, how it gets its money, the social trade-offs of providing or not providing services, and such. I think real crisis may be the only thing that fosters such debate. Until this happens, I fear that the US federal budget process will continue to be ungovernable--and get worse over time. Regardless of who holds the Presidency or what band aid deals they can facilitate.

In many senses, the situation in US federal budgeting may be more complex than that in developing countries! In both, however, we need to ensure that future reforms are better fitted to context--and to do this we need to theorize the reforms and context better. 

Theorizing case studies: check out ISS at Princeton

Princeton's new Innovations for Successful Societies is an interesting initiative. Jennifer Widner, Laura Bacon (a former student) and others are developing materials one can use to learn about innovative solutions to developmental problems. These include case studies, interviews and more. What I like about the materials is that they don't just present a 'best practice' idea or heroic story of success. They provide enough information to enable readers in theorizing about these practices and innovations: you can listen to the protagonists describing why the innovation emerged, what process it emerged from, how people came on board to make it happen, what kinds of political and operational demands it posed, and so forth. If only all development organizations were required to provide background stories like these for the innovations they so easily reproduce everywhere. One wonders if ministers of finance in French speaking African countries would so willingly agree to Medium Term Expenditure Framework reforms if they knew where these ideas came from (select advanced Anglophone countries like Australia), what kind of process MTEFs emerged from in these settings, what kind of political context they demand, what capacities they require etc. Ditto for just about every good, better or best practice one can think of. We need more of what ISS is providing: materials to help theorize what makes a best or innovative practice best or innovative and to assist us in thinking about whether these practices fit into new contexts...or what will be required to facilitate the fit. Visit their web site at http://www.princeton.edu/successfulsocieties/

Overcoming governance gaps from unwanted external best practice reforms

My blogs so far suggest that there is a growing governance gap created from many governance reforms. This is largely because reforms introduce best practice formal interventions that ultimately are not implemented. I suggest we should do things differently. What does that mean? Today I suggest a basic first step: ensuring reforms are 'wanted' by constituencies that have to do the implementation. Note I am not talking about 'ownership' in some vague sense. 'Want' is more direct, focused, immediate.  And the underlying story with many governance reforms is that they are not wanted by those required to ultimately implement the change. For good reasons.

 

Let me frame this idea with a short story and then get to the point, and some ideas on generating 'want' in a reform context.

The story is about adopting international accounting standards. These are an important 'best practice' element of the good governance and doing business regimes. Developing countries are encouraged to adopt them, with multilaterals and bilaterals sponsoring projects focused on such in locales across the less affluent parts of earth.

Mir and Rahaman write about the adoption of  international accounting standards (IAS) in Bangladesh (http://www.emeraldinsight.com/journals.htm?articleid=1528645&show=pdf).

They describe “evidence of immense pressure that major international donor/lending institutions put on the Bangladeshi government and professional accounting bodies to adopt IASs not only to provide credibility to foreign investors but also ensure that accountability arrangements with lending/donor agencies are tight enough. Clearly, the government and other institutions in Bangladesh have very little option (if any at all) because of the country's high dependence on aid…[even though]… a wholesale adoption of IASs constitutes a quick-fix that may not necessarily be in the long-term interest of Bangladesh and its accounting institutions.”

The description raises an important question of whether (and how regularly) donor insistence on best practice reform undermines local ownership--and the process of local problem solving and solution finding. Other descriptions in the story tie such questions to additional concerns about how well these imposed reforms are eventually implemented. The authors note that, "The perceived undemocratic nature of the adoption process appears to be creating and enhancing conflict among various constituencies, resulting in very low compliance with these standards."

The gap that these authors identify--between an imposed best practice reform design and an ultimate practice emerging from such--seems to emerge partly because the process of finding and fitting solutions is bypassed in favor of a solution coming from other countries. In their paper, the authors note that one reason this kind of imposed solution fails is simply that it is seen as being from the outside...emergent from an external community and not in response to local problems and through locally legitimate processes. They are not wanted internally, by the folks who ultimately have to implement and use them, digest them, allow them to interrupt their day-to-day work, change the way they think and contract and value things.

It reminds me of Richard Rose's admonition that reforms should involve “relevant” practices instead of “best” practices, where “Relevant practices are both politically acceptable and within the resources of government.” He argues that, “While relevance is no guarantee of a programme’s success, it avoids the certainty of failure due to the promotion of programmes that governments do not want or lack the resources to implement.”

While I cannot vouch for the accuracy of Mir and Rahaman's view that IAS adoption is failing in Bangladesh because it is unwanted by implementers, the story rings true enough to reproduce it. And in the story I see lessons for doing governance reform better--making it 'relevant' in the words of Richard Rose.

First, instead of focusing on reproducing solutions, reforms should emphasize  solving problems. The literature on change regularly emphasizes the importance of problems, crises and the like as motivatorsof change. Agents generally do not want change but can get to a 'want' position if they see that the change is needed to alleviate a threat, solve a problem, assist their survival, maybe even make them perform better.

Second, do no support reform design processes that lead to quick, lazy, narrowly engaged plucking of external best practices as the logical solution. If you want to implement a reform you need to spend time finding a solution that fits the context. This requires engaging with those who will implement, having them contribute to a shared understanding of why reform is needed and a shared process of experimenting with local ideas and scanning the globe for 'relevant' ideas. The process of finding and fitting is absolutely vital if one truly wants an implemented reform. This process is the only way one gets a clear perspective on political realities (not possible through an externally written political economy analysis) and local capacities to implement.

Third, create a high hurdle for external 'best practice' ideas to clear before adopting them. Evidence suggests that best practice ideas travel well from place to place where they are contextualized. Rose describes this as being "altering the original model to operate in a different administrative and political setting” noting that, “When altering a prototype, policymakers are not abandoning what they have learned from better practices elsewhere; they are applying a lesson learned elsewhere in ways relevant in their own national setting.” Some basic questions should always be asked where a best practice is being reproduced: What is the lesson learned elsewhere that you are replicating? How has the practice been altered to match your political realities? How has the practice been altered to match your administrative realities? This requires having a theory about the best praxctice (understanding what it is, and how it works) and being able to see if that theory works in your context--what the literature calls 'theorization'.

These three steps require that external agents take a very different perspective on the reforms they introduce. It requires a change in focus from (i) ready-made solutions to sticky and under-understood problems, (ii) quick answers with narrow groups to longer processes of asking questions in broader constituences, and (iii) easy acceptance of best practices to more skepticism about whether best practices are relevant. I am sure these ideas will not guarantee success, but as Rose says, they will help avoid certainty of failure when it comes to implementing reforms people just do not want.

What do you think?

Fitting best practice governance solutions to reform contexts: vital questions

Yesterday I blogged about a study of insolvency regimes. The study noted that many countries have adopted insolvency laws that are not now implemented. The problem is often a poor fit between norms implicit in the new laws and norms in place in the reforming country. This problem of fit is something I see a lot. It plagues development, in my opinion, and is a particular problem for those working on governance. Good governance solutions often come in the form of formal best practice processes that appear to make the difference between governments that work in the developed world and governments that do not work in developing countries. The solutions frequently don't work in the same way in developing countries, however. I was discussing this in class today and one of the great students said that when we talk about best practice we should be talking about what the practice is, what its content looks like, how it emerged, what its underlying norms look like, etc.etc. An insolvency law is best practice if it is used in a specific way, by specific users who see the world in a specific way. As the student said, we shouldn't talk about the law alone being the best practice...it is the full package that makes the practice work. I couldn't agree more. The problem is that most research notes that those who replicate such practices don't really know what the full package is. So they gravitate to what they see, and replicate that...on its own...without a theorized understanding of what can make it function. So, we find privatized firms failing in contexts that don't support them, internal audit laws flailing because countries with the laws lack accountants to do the audits, and insolvency laws lacking influence because legal capacities are too weak to give them life. The questions this raises for governance are many: How much of the good governance agenda is made up of best practice ideas? How many of these ideas have been effectively unbundled or theorized (so we know what makes them matter)? How often do we introduce best practice good governance solutions into contexts where they do not fit? Do we even know what criteria need to be met for a new practice to fit into a context?

Governance gaps and insolvency regimes

Thus far in this blog I have spoken about governance gaps in areas like public financial management and anti corruption. The observation in both has been that countries are adopting reforms in form but these are not yielding improved functionality because of limited practice. Building on this line, I like the work of Terence Halliday on insolvency reforms in development. http://www.oecd.org/dataoecd/42/47/38184739.pdf He starts the paper by saying that, "It is now a truism to affirm that In all lawmaking a gap opens Up between law on the books and law in action." He goes on to discuss the problems of basing reform on legal adjustment only, noting that law alone does not make an institution. Further, he questions the value of basing reform on global norms instead of local norms. He argues that this can create a major mismatch in institutional dimensions-because the norms underpinning new laws simply are not supported by informal norms implicit in political and social systems of reforming countries. This, he argues, leads to problems with implementation. The paper is really worth a read. It provides another example of reforms that make government look better but don't necessarily make government function better. It also raises some ideas about what a reform agenda focused on making effective governments or states might look like-more aware of extant institutions, for instance, less prescriptive of international norms as answers, more focused on fitting solutions to context.