Today was the 5th class of the semester, and we looked at one part of the New Public Management--the use of performance metrics to get things done.
This is one set of tools from NPM, and the question we asked was when it is a good tool, in what form, and with what design implications? The spirit of the conversation is the same as the discussion of bureaucracy: when do the tools matter and help us all move forward.
We looked at the use of performance targets in the NHS in the UK, and the mixed evidence that they (i) led to improved performance in some areas, (ii) may have led to some gaming, (iii) may have displaced some resources, and more...
So: the story in this case--as in most--is mixed.
We then looked at lessons from other reading on the topic about how performance metrics can be useful, but can also demotivate or foster risk aversion, and we looked at Christopher Hood's work on gaming... to see the way targets in particular can cause changes in behavior. I included a short blog post on motivation and the way stringent targets can undermine intrinsic motivations to do creative puzzle-solving in mission driven organizations (so, if puzzle solving is what you want...targets may not get you there).
We looked at articles that give some ideas about how to do use performance metrics, including a study from the World Bank that advises being realistic, careful about the complexity of the endeavor, politically relevant, and more. A paper we looked at would describe this as 'single loop learning' about doing performance management and advocated deeper 'double look learning' where one asks how to make performance management relevant in complex, ambiguous settings (perhaps where we face puzzles). The advice: be agile, decentralized and politically relevant. Finally, we visited Hood's work again and the difference between using targets, rankings and intelligence information.
My final points were that using performance information in management is no panacea...it can harm our chances of getting things done... but we have many variables to tinker with (the type of information, the way we structure incentives around it, how much we include it in the political discussion, etc.). The challenge is to work out if we need it and how we should use it. Put differently, the challenge is to see if we need to take this tool out of the toolbox and then to work out how best it should be used.
When results metrics, targets, and the like work
(and when they don’t)
Note: This is a team learning class, which means that the readings allotted for the class should be shared across the team as advised.
Class objectives and themes:
Overall: ‘Managing for results’, ‘performance management’ and ‘targeting’ and the like are very important ideas associated with the New Public Management (NPM). Most organizations now have some results metrics in place and tie these to organizational structure, pay processes, and more. The experience with using metrics to improve ‘getting things done’ has not always ben smooth, however, and there are many lessons to learn about when and why and how these tools can be used. This class will look to draw some of these lessons out.
We will discuss the following themes:
- What lessons can be learned about using performance metrics effectively?
- What lessons can be learned about the risks of using performance metrics?
- How can one ensure not to make mistakes with performance metrics?
You will prepare today’s class using the active learning mechanism where you are on the team advising the President of Messaria about restructuring her health sector. As part of the consultant team, you were previously asked to develop a summary of the approach a NPM theorist would take to reorganize the sector. Now, you have been asked to identify the risks or dangers of pushing for performance measurement or the use of results metrics in the sector (this is a key element in the NPM).
This is a vast topic, with many dimensions, so you have been asked to work as a team to do this exercise. Each team member must do some common preparation (read the summary of an article on the case of targeting in the UK National Health Service between 2002 and 2006, look at performance data from that case in 2010, and watch two short videos capturing critiques of the targets) and then do a summary of one particular article (with all six team members reading different articles).
- On the basis of the common case readings, each team member must come ready to participate in a short debate at the start of class, where we will ask the question: “Has the use of performance targets in the UK National Health Service been a success?”
- Beyond the case materials, each team member must produce a short summary of the article they focused on, using the following table. The team members should centralize all the summaries in one word file before class, and come ready to discuss what they read and learned. Each member will contribute to a discussion on “lessons from the NHS and beyond about when, how and why we use performance data to manage better…”
Article number and title
What are the key lessons (or warnings) about using metrics for better public management?
Are there any preconditions we should be aware of when using performance metrics?
Are there steps we can take to limit the dangers of using performance metrics to improve management?
Required case readings and video (for all students):
Bevan, G. and Hood, C. 2006.’Have targets improved performance in the English NHS?’ British Medical Journal, February 2006. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1370980/ (just read the summary on the web page…not the full article)
Have Targets Improved NHS Performance? http://www.kingsfund.org.uk/projects/general-election-2010/key-election-questions/performance-targets
NHS Too Target Driven, Complacent, and Unaccountable. https://www.youtube.com/watch?v=f6E04wfFop4
Get rid of NHS targets https://www.youtube.com/watch?v=RK_zwAT32W4
Required readings (to be allocated to different team members):
1. Deliverology: Using targets to promote delivery
McKinsey. Deliverology: From Idea to Implementation. (On the course page, about 10 pages) http://us.yhs4.search.yahoo.com/r/_ylt=A0oG7qccIPlRjzYAqc0PxQt.;_ylu=X3oDMTBzaDVoc25xBHNlYwNzcgRwb3MDMTEEY29sbwNhYzIEdnRpZAM-/SIG=15mak79sd/EXP=1375309980/**http%3a//www.mckinsey.com/~/media/McKinsey/dotcom/client_service/Public%2520Sector/PDFS/McK%2520on%2520Govt/Change%2520under%2520pressure/TG_MoG_6_Deliverology.ashx
Deliverology and all that (short blog post by Shanta Devarajan) http://blogs.worldbank.org/developmenttalk/deliverology-and-all
2. The limits of what can be measured; and how a results culture can be overly bureaucratic
Natsios, M. 2010. The clash of the counter-bureaucracy and development. (about 45 pages) http://www.cgdev.org/sites/default/files/1424271_file_Natsios_Counterbureaucracy.pdf
3. The dangers of gaming and interpreting performance information
Hood, C. 2006. ‘Gaming in Targetworld: The targets approach to managing British Public Services’ Public Administration Review July/August. (really short 6 pages) http://www.lse.ac.uk/study/executiveEducation/customisedExecutiveEducation/INAP/Targetworld.pdf
Motivation Revamped. Blog post. (http://checkside.wordpress.com/2012/01/20/motivation-revamped-a-summary-of-daniel-h-pinks-new-theory-of-what-motivates-us/) (a really short blog on motivation…which impacts how we use performance data)
4. Performance metrics, politics, and complexity
Van Dooren, W. and Thijs, N. 2010. Paradoxes of improving performance management (systems) in public administration. EIPASCOPE 2010/2. (17 pages)
5. Measures, targets, and lessons from past experience
Castro, F. 2011. Defining and using performance indicators and targets in government M&E systems. (about ten pages) http://siteresources.worldbank.org/INTPOVERTY/Resources/335642-1276521901256/ME12_v2.pdf
6. Types of performance metrics and their uses and potential influence
Chris Hood on ‘The use of measured performance indicators’ part 1 (https://www.youtube.com/watch?v=lbemM4icX-k) and part 2 (https://www.youtube.com/watch?v=sY_1DUEtgqw) (30 minutes of viewing in total)
Kamensky, J. 2013. ‘3 Performance Measures That Don’t Work (But Sometimes Do)’ The Government Executive (really short reading).http://www.govexec.com/excellence/promising-practices/2013/01/3-data-driven-performance-measures-dont-work-sometimes-do/60960/
Additional reading and resources (I will refer to these in class):
Tomono, K. 2008. What money can’t buy: Health care reform in the UK since 2000. Reuters Fellowship Paper, Oxford University. https://reutersinstitute.politics.ox.ac.uk/sites/default/files/What%20Money%20Can't%20Buy%20-%20Healthcare%20reform%20in%20the%20UK%20since%202000.pdf
Targets don’t work. https://www.youtube.com/watch?v=nPvAhy9LWYc
Buntaine, M., Buch, B., and Parks, B. 2014. Why the “Results Agenda” Produces Few Results: An Evaluation of the LongRun Institutional Development Impacts of World Bank Environmental Projects. http://www.uni-heidelberg.de/md/awi/peio/buntaine__buch__parks_23.08.2013.pdf
Smith, P. and Busse, R. Undated. ‘Targets and performance measurement’ Chapter 5.1 in Performance Measurement for Health System Improvement. (only chapter 5.1) http://www.euro.who.int/__data/assets/pdf_file/0003/135975/E94887_Part_V.pdf
Bob Behn’s Performance Leadership Report. See his interesting blog entries. http://www.hks.harvard.edu/thebehnreport/ Interview with Bob Behn (podcast). http://govinnovator.com/wp-content/uploads/2013/06/Bob-Behn.mp3