The Limits of Institutional Reform in Development

Governance indicators that really show what's going on...

The immune system is a really good metaphor for thinking about good governance, especially if one is asking 'is the governance systemgood enough to do business in a country...or to live there...or to engage in an infrastructure project'. We want a governance (and immune) system that protects the country (body) from known threats and can ward off future threats and help to ensure functionality of the entire system (fostering a context that is conducive for really doing business or raising a family or traveling around or going to school in).

What we worry about is that the governance (or immune) system actually fights with the body, or fails to identify or respond to threats or to institutionalize successful methods of addressing threats. 

The biggest problem with the governance agenda today is that indicators commonly fail to show where governance systems fail. Indicators tell us that systems are comparatively strong or weak (often according to a comparator we don't get to chooose) but not where the strengths and weaknesses are. This is really problematic in most countries, where real people assume that there are governance problems and are interested in knowing where these are and if anything can be done to mitigate the risks of such. 

One of the ways to overcome this bias is to pay less attention to the form-based indicators that currently dominate thinking about governance. These can be replaced with indicators that provide a more serious focus on the ends or results of governance--showing how well governments are doing in a variety of real areas. I emphasize 35 key ends in my recent work, in areas where governance matters a great deal (like ‘defense, public safety, law and order’, ‘public infrastructure’, and ‘economic progress and adaptation’). I use these to construct a picture of how well a government is using its authority to do the things citizens need (and potential investors are interested in considering as they choose between emerging market opportunities).  

I provide the dashboard for my own country (South Africa) below. The dashboard shows how the country’s performance compares with that in other BRIC countries (Brazil, Russia, India and China). I show two of these three countries' comparative scores as well. I categorize this comparative performance by calculating (with continuous data) the number of standard deviations a score is away from its comparator group mean. For example, when this is done for road safety and the BRIC group, South Africa ends up scoring 1.73 standard deviations above the mean of fatalities per 100,000. I use color codes to reflect this relative performance, with different colors suggesting performance that is ‘comparatively weak’, ‘below average’, ‘average’, ‘above average’ and ‘comparatively strong’. I also work with a clear color, which reflects instances where data is not available to allow an assessment. In the road safety example, for instance, South Africa is categorized as ‘comparatively weak’ (and the block is red).

 

 

As one can see, the South African story of comparative governance is not all red, but it is not very greeen either. The comparative governance performance is mixed. This suggests that investors will face specific risks in some areas—that they will need to factor into any investment decision—but not in others. If choosing between an investment opportunity in South Africa and other BRIC countries, for instance, an investor would need to recognize that any engagement in South Africa will require managing serious (comparative) governance deficiencies in the area of defense, public safety, law and order and economic progress and adaptation. This helps investors to know which gaps they would need to cover themselves in the firms or projects they support (in this case they would need to pay for security, have a strategy to address trade issues, and more).

The dashboard provides a view onto the governance performance in a given country, much like one might get a picture of the effectiveness of an immune system in a body—identifying where it is effective and where it is not effective. This allows potential investors a window into how good governance appears on average (which one may perceive by considering the number of orange and red blocks as compared with the number of green blocks). It also allows investors to see where the governance risks exist (every country has some red and orange areas) and where they should be looking for signs of real reform, not signals.

This picture is completely different from the one obtained by governance indicators that capture (predominantly) whether countries have best practice forms in place, and reflect only average performance. This point is made when noting that South Africa performs better than other BRIC countries on most of the commonly used governance indicators. For instance, South Africa scores 0.33 on the Worldwide Governance Indicators (WGI) measure of government effectiveness (in 2012); China scores 0.01; India scores -0.18. South Africa ranks 41 on Doing Business scores with Russia at 92, China at 96, Brazil at 116, and India at 134 (in 2013). These data points leave an impression of South Africa that contrasts dramatically with the detailed dashboard above, partly because the data points capture form over function (and South Africa’s governance ‘looks’ more legitimate than the others) and partly because the data points average a highly varied performance).

I find that the governance dashboard approach is an empowering and useful approach to thinking about and assessing governance in developing countries. It helps one see where the governance system is compromised, like a poorly functioning immune system, and cannot be relied upon. It then helps one to think abiout ways to mitigate the risks associated with such compromise.

 

Good Governance and Cheese... (What?)-part 2

I believe that good governance involves 'governments effectively exercising the authority given to them.' I hold that many of the expressions of good governance go unnoticed, but make the real difference. One such expression happens when governments do things like licensing, regulation, inspection, and such well (notice: I'm not saying when governments do well on the doing business indicators, that measure effectiveness by the number of regulations that exist). Governments have key roles in providing effective regulation, enforcing standards, and more. When governments play this role well, they help countries do well.

A few months ago I shared an example of this not happening. It was a country where a food safety issue over a decade ago led to some food exports being stopped, by key trade partners. Over a decade later, the food products are still not exported to those trade partners. Indeed, one sees a chart of past exports that were growing to a date and then dropped to zero and just stayed there. The government has never responded; by investigating why the problem occured, swarming the problem with inspectors, and solving it...

I contrasted this with a food safety issue that arose in the UK in May: Chinese authorities argued that there were food safety issues in a British dairy and they stopped imports of British cheese as a result. When I posted about this I said I'd come back to the issue and reflect on how long it took for the British authorities to respond and deal with the problem.

 

Well, the following news articles chronicled the response--which began immediately, was quick and comprehensive, and led to cheese exports from the UK to China resuming about a month afterwards. This is good governance: governments doing what they are authorised to do, effectively and comprehensively. Gritty, real, tangible expressions of this kind of governance are evidenced every day in functional countries, where governments identify problems and resolve the problems. Unfortunately, most governance indicators don't capture this kind of expression of good governance ....

http://www.bbc.com/news/business-27280986

http://www.dairyreporter.com/Markets/Chinese-UK-cheese-export-ban-will-be-lifted-soon-Defra

http://www.thegrocer.co.uk/fmcg/fresh/china-lifts-temporary-export-ban-on-british-cheese/358316.article

http://www.dairyreporter.com/Regulation-Safety/China-lifts-temporary-ban-on-British-cheese-exports

Promoting change strategies that folks actually embrace is...too hard?

This is just so great....

Just Development: Ideas to Do Development Differently

These are interesting perspectives on doing development differently, for donors and for those working in country. I take them from a recent document produced by justice professionals in the World Bank in the Just Development series:

.......................................................................................................................................................................

IMPLICATIONS FOR WAYS OF WORKING

How do we apply these lessons and concepts more broadly in the justice sector, in order to achieve better outcomes? Below, we set out emerging implications for those involved in programming.

For Funders

1. Realism. There is an overarching need to be realistic about objectives ascribed to justice programming, recognising the often limited role that external assistance can play in reforming local processes of power and contestation. Moreover, complete solutions are unlikely to be able to be determined at the outset of programming but rather emerge through trial and error over time.

2. Adaptable operational modalities. Com- promise must be found between the need for bureaucratic certainty to meet organisational rules and the need for country level flexibility to ensure programming remains relevant. Po- litically smart and locally led approaches will benefit from adaptable terms of reference and theories of change, ‘rolling’ country-level log- frames (if they must be used at all), shorter term workplans and grants with relatively loose conditions. Working politically does not neces- sarily mean doing different things, but rather doing things differently by changing processes and dominant ways of thinking.

3. Risk mitigation. Funders currently require high levels of bureaucratic and managerial certainty specifying results, delivery timeframes and overall cost, especially at the political level where concerns about reputational and fiduciary risk are keenly felt. To encourage change in ways of working it is thus important to document and communicate successes achieved through politically smart and locally led approaches, while accepting this will necessarily involve some degree of failure.

4. Partnership between funders and imple- menters. Funders need to be comfortable with a high level of delegation, empowering politi- cally savvy operators at the country level to respond in real time to issues as they arise in justice programming. This does not mean that funders should be unengaged in programming decisions, effectively outsourcing political re- lationships. Rather, funders and implementers should develop partnerships that go beyond a contractual relationship to include regular dis- cussions on the substance (not merely the pro- cess) of programmes. This should allow imple- menters the space to operate as full partners and at arms length from the funder, but also keep the funder engaged, especially on politi- cal issues that arise with counterparts.

5. Creative funding modalities. Many justice programmes tend to be single-donor funded with implementation units operating in parallel to government institutions (with some exceptions, such as multilateral basket funds). But working in politically smart and locally led ways requires the ability to work directly with partner governments and through their systems where possible; moving from risk avoidance to risk management; and encouraging donors to work together to ensure coherence. One way to achieve this is through greater use of pooled or basket funds, with ideally limited donor earmarking, and, where appropriate, ex-post reimbursement processes to support government systems while ensuring accountability of financial expenditure.

For Country Level Implementation

1. Focus on justice problems, not the justice sector. If the aim is to improve justice outcomes for people then it is important to hone in on problems that people face (such as protecting girls from violence on the way to school). While sector-wide approaches may seem more comprehensive, they are often too ambitious for what external actors can realistically achieve in short timeframes and create unrealistic expectations. Solving tangible problems is one way to get around this while making genuine improvements in peoples’ lives.

2. Realism. While competitive procurement processes, as well as unrealistic terms of reference can create an overly ambitious environment, programme implementers should be more upfront about what is feasible within the scope of a programme and not over- promise on what can be delivered. Otherwise justice programmes end up failing the most important group – the poor and vulnerable. 

3. Political savvy staff is paramount. As we have noted, justice reform is all about the politics. As a result, smart, politically attuned operators are needed to navigate complex and changing political realities. Technical expertise can always be brought in but smart operators who have a deep understanding of the context and engage in continuous ‘learning by doing’ are more difficult to find.

4. Build trust with local counterparts. Early success may be achieved by finding entry points that are less politically contested. Working alongside and supporting local partners to address and solve problems they identify should be the main objectives, particularly at the outset. The trust built can then be leveraged into more controversial areas of the justice sector.

5. Keep money out of the equation. Even if the funder imperative is to spend large pots of money quickly, this should not be central to discussions with counterparts as it can disincentivise locally led problem solving. This means moving away from justice programmes that offer new buildings, equipment and trainings as incentives for partnership. These tools may still be required, but should not be the default entry point.

6. Invest in small bets, that trial multiple approaches to solving the same problem, with effective monitoring systems in place that can then determine which work best and, where possible, take them to scale. This will involve some degree of failure but losses will be more limited than where investment focuses on just one solution over a 4-5 year timeframe. Not only will this approach help to resolve problems more efficiently, but it will also contribute to building the evidence base for what works in the sector.

None of this is to suggest that politically smart, locally led approaches are a panacea and that by working in these ways programming will be perfected and justice reform rendered simple. However, based on early results across multiple sectors, the changes suggested here do, we believe, offer a more promising avenue for improving people’s access to quality justice by recognising the specific role of development assistance in supporting locally brokered solutions to what are inherently political problems. 

Constructing the problem as an entry point for change, broadening engagement (part 4 of class 14 in GTD)

I have not written about my course for a few weeks, given the Doing Development Differently posts. 

The last post on my course focused on part 3 of class 14, and focused on taking readers through the process of problem construction (constructing problem statements that draw attention, mobilize action, and provoke a search for new ways of doing things). I was using examples from places like Mozambique to explain how I work with an authorizer first to identify a problem area; and then I work with teams, asking them questions like 'why does it matter?' and 'who cares?' In these meetings I am pushing the teams to identify evidence--in data, stories and reflections on crises--that help to construct the problem.

I finished the last post with this text:

"Ultimately, the strategy was to get key data and stories about how the performance deficiency affected end users in the system--given an argument that these end users were the constituents of politicians whose support would be needed, or businesses paying taxes (who often had louder voices than citizens). The group agreed to get data about these groups, via past reports and surveys (like the Afrobarometer survey and the Doing Business Survey). 

This was another brief iteration, lasting a few hours and coming one day after the first meeting, where the group was constructing its problem to draw attention, create space for action, and ultimately move towards a new reform. The groups were asked to come back the next day with the new information."

Hopefully readers can see that the process of constructing the problem is (i) rapid, (ii) iterative, (iii) action based, and (iv) focused on the substantive engagement of local agents who will be needed to solve the problem.

The process of going back to past reports and to other constituents is purposeful. It forces the team to broaden its view of the problem, in a disciplined manner. Others' views are introduced, via past written work and even interviews and other forms of active, primary engagement. In this process I expect that team members will start contacting others who will be needed in the change process, and these folks will have opinions on the problem construction process that are often important. These folks will bring their authorizers into the process, bridging some organizational gaps that often fester.

• In the group I was working with on Justice (discussed in prior posts), the team started engaging with key people across the justice sector (as well as past analytical reports of the sector performance). They came back with real data to show the performance deficiencies--and a few new team members who actually furnished the data. This was an important iteration for existing team members; they got to see that contact with others can generate new resources that help action. Most importantly: The new team members had engaged their authorizer to ask about joining the team, and the authorizer had become engaged with the minister who initially provoked the work. So the authorization of the team grew as well. The figure below shows this, with numbers showing how the relational aspect of problem construction worked: 1. The minister engaged a facilitator; 2. The minister convened a team; 3. The team reached out to colleagues in other areas of the system; 4. The different members reached out to users and to analytical work to ask about the problem; 5. A new authorizer was engaged. The result: a broadened team, and broadened authorization, and broadened engagement.

• In another country I worked on maternal mortality problems, and the central team I had been engaged with decided to share ideas about their problem with health workers in a select set of provincial and district centers. They took two days to travel to the centers and came back with a variety of vital new ideas, data, and distributed members who felt a vital connection to the change process.
• In yet another example I worked with a small team on an industrial regeneration reform. The team had defined the problem in a specific way (focused on deficiencies in legislation and in doing business regulations, where the emphasis was on the number of days to start a business, etc.). The team decided to float this problem definition with business people in the sector they were focused on. The business people gave them new data, new views, and new points of interaction. The team came back with a completely different view on the problem, emphasizing constraints to actual production.

In all cases, this was another iteration in constructing the problem, that took the teams into a deeper understanding of their contexts and helped them to communicate a problem that they knew key players cared about... The process took days in all cases, and galvanized the team in all cases. It also helped to broaden engagement to those distributed agents who would be affected by the change. In most cases this broadening leads to expanded teams, and connections to new authorizers as well.

These are key factors that help success later on, especially when dealing with complex problems: One gets a greater handle on complexity the more one engages and the more one learns.

After this iteration, I often find folks start getting concerned about how 'hard' or 'big' their problem is: So I ask them if it's possible to deconstruct the problem into smaller bites. I will post on this process of deconstruction tomorrow.

The PDIA anthem...

In a minute and a half, one of my students breaks it down...  

"We don't have to recreate the wheel" is (often) a stupid statement in development

I often hear people tell me that we don't need adaptive reform processes in which countries 'find and fit' what works for their contexts. "We don't have to recreate the wheel" is what I often hear, where people are talking about many different things as "the wheel": civil service systems, budgeting mechanisms, accounting systems, democratic processes, procurement rules, 'doing business' requirements, and the list goes on.

Development is like recreating wheels?

 

I just think the statement is (generally/mostly) a stupid statement in development. Largely because the list of things people refer to as "the wheel" (as in the list above) are not anything like wheels at all. A wheel is a relatively simple thing, that is one complete and consistent object. It is not like a complex governance system or market where there are multiple moving parts, changing configurations, etc. These kinds of challenges are deeply contextual and require context-specific design and implementation.

Even if you think they are like wheels, the process of taking a wheel from one place to another (so that you don't ned to recreate it or substantially adapt it...so that it's almost like recreating it) is so extremely difficult.

Let me explain what I'm saying by talking about actual wheels. Of cars.

The best selling cars in many developed European countries are small; Audi holds sway in Andorra, Fiat in France. These cars have small wheels. But the best selling car at the moment in Kenya is the Toyota landcruiser. Imagine taking the wheel from the best selling French or British car and saying to a Kenyan "use this on your landcruiser...you don't have to recreate the wheel..." It's just a silly idea. Yes, the shape is the same, and yes the end goal is similar. But the adaptation required from the fiat to the landcruiser is so huge. One wonders if it is more work to try and adapt the fiat wheel or just to start from scratch.

This wheel does not fit...

This car....

I find it really interesting that different countries have different best selling cars, just like they have different administrative traditions, political legacies, cultural trends, tastes, etc. And just as each country's best selling car requires a specific wheel, so do all counries' governance and development systems require specific solutions....

The wheels needed by the following cars are not the same...so you need to adapt the wheel so much each time that it probably feels like you are recreating the wheel.

It is also possible that we don't need wheels at all... but rather require a totally new solution that others didn't think of (so recreating the wheel....even if there is a wheel to recreate...is the wrong solution. The best solution may be a set of skis!).

Beyond this, it's also possible that the problem isn't the wheels only. Go ahead, don't recreate the wheel: put wheels on the following and pat yourself on the back for not 'recreating the wheel'. But you may need a new engine, lights, steering column, etc. And you may find that these take a bit more work. 

Or just bring in a new car. Just make sure you know what kind of car is likely to sell in Tajikistan, or Cambodia, or Honduras, or wherever you want to go. Chances are it isn't the same wheel, or car that you have in the USA or the UK or Germany (where, by the way, the most popular cars are also different).

You often need more than a wheel....

 

 

Let me encourage you about Doing Development Differently: "There's good happening where it should not be done."

This is my closing presentation at the doing development differently workshop. It was my personal reflection on why we need a community of practice about doing development differently--not because we have the perfect solution to the problems of development but rather because we proved that there are many folks out there--across the world--who are doing development differently. There is an alternative to the dominant incumbent approach. 

We need to work out how to learn from each other, and to encourage each other--about doing this kind of work in our host organizations, across boundaries, and with passion. We created the DDD Manifesto to follow up on this kind of sentiment... join if you feel that this kind of group is for you. (just click)

 

If you want to Do Development Differently but it sounds too hard...

Arnaldo Pellini recently wrote an interesting personal blog post about the Doing Development Differently workshop and manifesto. He concludes his thoughts in the following sentence:

• I agree with these ideas and  I can share and discuss these ideas with the team with whom I work  but what difference can it make if the systems around us due to organizational culture, history, circumstances, and traditions struggle to  embrace flexibility, uncertainty,  untested experimentation, and slow incremental changes?

This is an honest reflection from a practitioner in the field; and one that I hear often--from folks working in multilateral and bilateral agencies, as contractors, and beyond. It captures a concern that the development machinery (organizations, monitoring and reporting devices, profession-alliances, government counterparts, etc.) is structurally opposed to doing the kind of work one might call DDD or PDIA.

It's like this cartoon...where our organizations say "let's innovate but stay the same."

I have been thinking about this a lot in the last few years, ever since I wrote chapter ten of my book...which asked whether the development community was capable of changing. In that chapter I was not especially confident but (I hope) I was still hopeful.

Since then, I think I'm more hopeful.

• Partly because we have found many folks in the multilaterals, bilaterals, contractors etc. who are doing development in this more flexible way. We invited a range of them to the DDD Workshop and have over 250 signed on to the DDD Manifesto. One of the goals of our work in the next while is to learn from these folks about HOW they do development differently even with the constraints they face. How do they get funders to embrace uncertainty? How do they get ministers in-country to buy-into flexibility and give up on straight isomorphism? 
• I am also working on research projects that tackle this question; doing PDIA in real time in places where development is predominantly done through the incumbent mechanisms. It is hard work, but I am finding various strategies to get buy-in to a new approach (including showing how problematic the old approach is, by working in the hardest areas where one has a counterfactual of failed past attempts, and more). I am also finding strategies to keep the process alive and buy more and more space for flexibility (by iterating tightly at first, for instance, and showing quick wins...and telling the story of learning and of increased engagement and empowerment). So far, I have not experienced complete success with what I have done, but I have certainly not struggled in getting support from the practitioners and authorisers we work with. (In my world it is harder to get support from academics, who think action research on implementation is a hobby and consultancy work... indeed, anything that does not say 'RCT' is considered less than academic. Sigh.)"

All this is to say that I think Arnaldo is emphasizing a really important constraint on those working in development agencies. But a constraint that we should work through if we really do agree that these more problem driven, flexible approaches are what is needed. To Arnaldo and others I would suggest the following:

• Separate the conversation about which way we should do development from the conversation about how much our organizational realities ALLOW us to do it. The first conversation is: "Should we DDD/PDIA?" The second conversation is: "How do we DDD/PDIA?" If we conflate the conversations we never move ahead. If we separate them then de can develop strategies to gradually introduce PDIA/DDD into what we do (in essence, I'm suggesting doing PDIA ourselves, to help change the way we do development...see an earlier blog).
• I also constantly remind myself that we (external folks in development organizations) are not the only ones facing a challenge of doing new stuff in existing contexts--with all the constraints of such. This is what we are asking of our counterparts and colleagues in the developing countries where we work. Dramatic and uncomfortable and impossible change is in the air every time we are introducing and facilitating and supporting and sponsoring work in developing countries. I always tell myself: "If we can't work it out in our own organizations--when we think that our own organizational missions depend on such change--then we have no place asking folks in developing countries to work it out."
• So, it's a challenge. But a worthy one. And if we care about doing development with impact, I think it behooves us to face up to this challenge.

Good luck, Arnaldo, thanks for your honesty and for the obvious commitment that causes you to share your reality. It is really appreciated. 

Vivek Srivastava of the World Bank on Doing Development Differently in Sierra Leone

This video is also from the Doing Development Differently workshop. Note some of the key lessons from the short presentation: 

• Shared problems create space for reform
• Engaging local agents (in this case those in the government) in designing and implementing reform ensures ownership
• Flexibility can exist (even in World Bank projects) between milestones... where you set targets for where you want to get to but allow agents on the ground the 'get there' through flexible means
• Many folks need to be engaged in civil service reform: in political leadership, in the middle of the bureaucracy, and at the frontlines...
• and more.