Regular reviews and feedback help reformers manage contextual complexities and adapt reform designs

« We should care more about the many institutional reform interventions that have 'so little impact' | Main | Why development and institutional reform must be step-by-step »

02/06/2013

Regular reviews and feedback help reformers manage contextual complexities and adapt reform designs

The following post was received as a comment from Martin Johnson. I have worked with Martin in various countries and always find his stories of working on long term reform in places like Albania, Uganda and Pakistan fascinating. He I also one of the founders of the PFM Board (with Mauro Napodano), a blog space where you can go for all things Public Financial Management (http://pfmboard.com/)

By Martin Johnson, Principal Consultant REPIM

Matthew Andrew’s 'The Limits to Public Financial Management Reform in Development' draws attention, among other things, to the widespread use of governance and related reforms that are ‘best-practice’ in design but are short-term and seem to ignore context and the myriad layers of minor and major context-specific issues that must be negotiated before technical solutions have even a chance of being adapted properly, never mind implemented at the front line by people that matter (however insignificant or distant those people may appear to project designers or project implementers).

He is not alone in being frustrated by observing the widespread use of this approach and it is no surprise that he presents so much evidence of success in terms of ‘signals’ and so (relatively) little impact from that apparent success.

The time is ripe for a weighty tilt against this paradigm. The evidence that Matt has assembled is certainly weighty. It is also persuasive and certainly has the potential to begin shifting the paradigm.

A fair amount of my own practical experience chimes with much of the analysis and evidence in Matt’s book. This includes coming into contact with projects that aim to support governance reforms but have unrealistic or inappropriate log-frame design, both in terms of nonsensical periods of time over which fully-formed reforms are expected to be designed and delivered and in terms of the contextual “constraints” that are simply overlooked or assumed away.

In one case I had an unusual opportunity to periodically review a major PFM reform over a ten year period. The anticipated achievements in its original design (covering a period of just 2.5 years – a period typical of many development partner-supported governance reforms) were unrealistic to the point of being wholly unachievable. The support would have resulted in abject failure and closure after (or even before) year 2.5 had it not been re-scoped shortly after it started – an interesting story in itself, but one for another time and another place. Even after re-scoping it remained over ambitious and was on the verge of closure on several occasions because of an apparent failure to deliver results (including when very good results were being delivered!).

The story of that reform is one where what was delivered after ten years was more or less what was possible within that period in the highly difficult change management environment of the country concerned (where what Matt refers to as ‘cultural cognitive influences’ are extremely strongly embedded). Interestingly enough, whilst a number of different development partner staff members came close to closing the project at various points in its lifetime because of ‘lack of results’, it managed to survive (sometimes narrowly) in part because it was subject to a regular and independent review process that explained to the development partner on most of these occasions that ‘context’ was being addressed and that progress was commensurate with context, even if it was not commensurate with the project’s over-ambitious design.

From my own experience, therefore, I have come to appreciate one of Matt’s conclusions with regard to ‘best practice’ and the short term nature of many governance reforms - the need for regular reviews to address context.

Posted at 01:00 PM in Book endorsements, Finding and fitting solutions that work, Overlooking the change context | Permalink

Tags: development, governance, limits of institutional reform in development, martin Johnson, matt Andrews, pfm, pfm board, reform, repim

Comments

You can follow this conversation by subscribing to the comment feed for this post.

The comments to this entry are closed.

My course at Harvard: See the series of lecture materials here

Post 2015 governance indicators...past blog posts reflecting a variety of thoughts on this topic

'is a landmark analysis that will change the way we both understand and design institutional reform.' Ha-Joon Chang, University of Cambridge, and author of 23 Things They Don't Tell You About Capitalism

Where you can order the book

Make It Stick: Challenging, informative book on learning. Really got me thinking about how I teach, learn, impart, etc.

CHECK OUT THE MOZAMBIQUE EXPERIENCE ON THE BUILDING STATE CAPABILITY WEBSITE

READ ABOUT PDIA IDEAS FOR UGANDA: Closing the gap between form and function: a new approach to institutional reform in Uganda

The World Bank doing a version of PDIA in Sierra Leone

This article draws on a variety of theories to discuss productive capabilities. I love the section on the firm, which builds on Penrose: " The firm is a collection of physical and human resources which can be deployed in a variety of ways to provide a variety of productive services. In fact, ‘the services yielded by resources are a function of the way in which they are used – exactly the same resource when used for different purposes or in different ways and in combination with different types or amounts of other resources provides a different service or set of services’ (Penrose 1959: 25). The growth process, in the Penrosian framework, is realized through the firm’s recognition and exploitation of productive opportunities, specifically of ‘all of the productive possibilities that its entrepreneurs see and can take advantage of’ (Penrose, 1959:31). As Best (1999:108) points out, ‘productive opportunities link the firm to the customer in an interactive relationship in which new product concepts are developed. The advances in productive services can extend the firm’s productive opportunities by enlarging the members’ capacity to recognize and respond to new product concept possibilities in the environment’."

Timely fault detection and diagnosis are critical matters for modern chemical plants and refineries. Traditional approaches to fault detection and diagnosis of those complex systems produce centralized models that are very difficult to maintain. In this article, we introduce a biologically inspired multi-agent model which exploits the concept of leadership; that is, when a fault is detected one agent emerges as leader and coordinates the fault classification process. The proposed model is flexible, modular, decentralized, and portable. Our experimental results show that even using simple detection and diagnosis methods, the model can achieve comparable results to those from sophisticated centralized approaches.

Albert O. Hirschman's Hiding Hand: Not new but a great and vital read for anyone in development.

This David Lewis article from 2012 is an interesting read, and emphasizes the fact that innovation is not always needed in development. PDIA is also not always needed. If we know what we are doing and the path to getting it done is clear, we just need to have a really good strategy and a disciplined bureaucratic process of doing it. I would be the first to agree with this. However, many many things in development are not in this category; either we don't really know what to do or the path to do it is unclear and we need to innovate in the politics and management systems to create space for doing even what seems mundane.

Innovation-vs-disciplined-execution-at-scale-r3-dynamic-linking1

I like this article by Steve Denning on 'the golden age of management' ... it discusses the appropriateness of different management modalities given different challenges, etc.

The Challenge of Policy Implementation: A paper on peer learning in the area of vocational training. Loads to think about. See page 13: 'Decades of research into policy making and policy implementation have placed in serious doubt both the presumed rationality as well as the linearity of the process. In the first instance, we learnt that policy making is far from the scientific and rational exercise it is touted to be. With his long experience in educational policy analysis, Ball (1998, p. 126) concludes that most policies are in fact ‘ramshackle, compromise, hit-and-miss affairs that are reworked, tinkered with, nuanced and inflected through complex processes of influence, text production, dissemination and ultimately, recreation in contexts of practice’. Policy making is often complex and messy – a Do-It-Yourself job – with those responsible ending up ‘borrowing and copying bits and pieces of ideas from elsewhere, drawing on and amending locally tried and tested approaches, combining theories, research trends and fashions, and not infrequently flailing around for anything at all that looks as though it might work’ (Ball, 1998, p. 126). The messiness of the policy-making process is heightened by its close, often tense relationship with politics, to the extent that, as Malen (2006) notes, education policies end up embodying ‘highly salient, value-laden issues that cannot be readily, fully or permanently resolved through rational deliberations or unanimous agreements. As a result, these policies tend to be adopted and implemented through political processes that reflect the relative power of contending groups more than the relative merits of policy options’.

Richard Bond's old paper on process approaches in Sri Lanka is one we should be reading as we ponder a world with PDIA and other approaches in play

Iterative and Incremental Development: A Brief History... Iteration is not new...but really important... a good read

I like this article on uncertainty and project management; note the discussion on learning..." In our simple model of projects, learning comes from signals that are incompatible with the project team’s predictions. As project teams monitor their projects, they must recognize that observed signals are incompatible with their model of the world and be willing to change their representation of the world either by updating the partition or the transition mapping." So learning is only possible if we know what our theory of change is (our expectations) and we have information about what is going on to see if reality departs from these expectations, and we actually adjust what we do as a result.

Diagnosing Deep Roots of Development: Johannes W. Fedderke, Robert E. Klitgaard, James P. MacMurray, Valerio Napolioni

"It is argued that in complex settings, performance management may benefit from new ways of carrying out performance management. It is suggested that performance management needs to be more agile, more decentralised and more political."

Imran Rasul and Daniel Rogger looking at infrastructure projects in Nigeria: A really interesting paper...36% of announced projects never start...

MY RECENT PAPER ON GOVERNANCE AS ENDS AND MEANS

This paper synthesizes the approach I take to looking at governance in nations states. The approach emphasizes ends as the starting point for any view of governance. (Asking about what governments do rather than how they do them). I also emphasize means; but in thinking about what it takes to produce ends, not as stand-alone factors.

MY PAST WORLD BANK MONOGRAPH ON LEADERSHIP IN DEVELOPMENT

The paper explores what it takes to make change happen in the context of development, and in particular, the role leadership plays in bringing about change. The analysis and findings conclude that leadership manifests itself in different ways in different contexts, depending on readiness, factors that shape change, and leadership opportunities. However, the key characteristics of plurality, functionality, problem orientation, and change space creation are likely to be common to all successful leadership-led change events.

I recently finished reading Bill Easterly's Tyranny of Experts. A very interesting read; provocative and entertaining. It got me thinking a lot abut the role of agency and power in development. Development initiatives give power to experts and assume that top-down leaders are vital to make things happen. What about the poor who are affected by decisions taken by experts and foisted on them by 'the leaders'? The poor are not the only ones left out in this kind of development discourse: what about the distributed agents in government, non profits and struggling small business who have to implement and live with the policies, reforms etc. introduced from above? A good read that still has me asking questions.

Bob Behn's latest book on performance stat was a quick and interesting read, but has details and insights that will keep me coming back for a long time. Bob writes about the performance state movement with first hand knowledge of examples. I was most taken by the views on how this management approach gets adapted to different contexts. A really good read.

Francis Fukuyama's 'Political Order and Political Decay' is a great read. It explores the emergence of states, and the role of states, and the challenges of thinking about state capability, democracy, and more across space and time.

Brian Levy's new book is full of practical and deep reflections on the real world of politics and development. He contributes to a growing literature stressing the importance of doing development within contexts and offers ways of thinking about the challenges of doing this. Inside these pages you will find Brian's views on 'what matters' in a context, and about how one can assess the space for action and change in a context, and what kinds of reforms might fit what kinds of contextual settings.

Rethinking Productive Investment, by Crespi, Fernandez-Arias, and Stein... I really like this volume. It blends theory and practical cases about the challenges involved in making economic transformation processes work. It is also available free!

A PAST MY BOOK OF THE WEEK: HELEN TILLEY...The Political Economy of Aid and Accountability. If you liked Ferguson's 'The Politics Machine' you should read Helen's book. Dynamic, deep, well researched and illuminating.

PLUG OF THE WEEK: READ 'THE GOLDEN HOUR' BY TODD MOSS...SERIOUS, FUN, AND ENTERTAINING (YES, DEVELOPMENT CAN BE ALL THOSE THINGS)

The Limits of Institutional Reform in Development

02/06/2013