MEASURES SHOULD CAPTURE THE SYSTEMS
BUT ONLY THOSE THAT HELP PRODUCE THE OUTCOMES.
A reader, David, asked an important question in response to my blogs about governance and outcomes. Essentially, he noted that the world of development has come a long way in recognizing that governance matters. Nowwe have a focus on administrative structures, rules of the game and the like, under the title 'Governance'. If we start saying that governance must be about outcomes, could we lose this focus...
A great question (which I hope I have represented fairly). Here are my thoughts on it.
While my past work may seem to suggest I see outcomes as measures of governance themselves, I do not. Outcomes come into the picture because governance must be about outcomes. So, governance is about the institutions, structures, systems etc. that shape authority mechanisms principals use to ensure that agents produce outcomes that lead to welfare gains for the principals.
Consider the figure below. The indicator should measure institutions, structures, etc. that we believe help achieve outcomes (or otherwise enhance the welfare of principals). In the case of governance in the child health arena, the indicator would measure whether a country has systems etc. that we believe facilitate the exercise of authority in such a way as to maximize the survival rate of under five children. It would aggregate many components for sure, to be a complex measure of the governance structures deemed necessary to keep kids alive. But it would not be a vague summary indicator of something that someone somewhere can postulate about.
Note that the outcome is central here, and the test is whether the indicator, and its components, reflect on the outcome. See the discussion on construct validity from a few days ago. You cannot assess construct validity unless you know what the governance focus is. And you cannot vouch for the construct validity of your indicator unless you have some way of testing whether its components are actually related to the outcome you care about.
If one thinks of governance indicators--and the relationship between governance, indicators and outcomes--in this way, it should alleviate concerns that the focus on institutions, systems, strcutures etc. will be lost in the future. They are still front and center. But they cannot be introduced in a stand-alone fashion. In this way of thinking they are chosen and included in the indicator and in our lexicon of reform because we believe that they help improve the way society produces outcomes.
I have used the word 'believe' above to be a little open and fluffy. I would love to see it replaced with 'know' or 'believe, on the basis of theory and evidence'. All the systems and structures and institutions we include when constructing an indicator should be things we believe, on the basis of theory and evidence, facilitate the exercise of authority by governments (and other entities) in such a way as to produce outcomes that advance the welfare of citizens. Sorry such a mouthful.
If we had this kind of theoretical, evidence based story, we would not weaken the governance agenda--or dilute it. We would surely strengthen it. Being able to say that having a multi-year budget in the health sector is empirically associated with having better child health outcomes is surely the best way of ensuring that these reforms gain traction. But we cannot say this. So, we depend upon the development community accepting that multi-year budgets have some intrinsic, stand-alone value...things to be done even if we don't know their value. I cannot really think of too many governance practices to put in this category.
I know I have not addressed the fullness of David's question, which asks about the impact of other factors on outcomes, potential interactions of these factors with governance reforms, etc. I hope the commentary following this--and future posts--will go some way to do this. In the mean-time, as usual, what do you think about the perspective?