The question that generated the most interest was:

Q. What would you say are the most common challenges that M&E practitioners face in developing the M&E arrangements for projects?

Paul's Answer:

There are many challenges, but three big ones that come to mind are discussed below:

First, there are the perennial challenges of ‘measurement’ and ‘attribution’. How do we ‘measure’ unmeasurable or amorphous social changes such as ‘empowerment’, ‘wellbeing’ and ‘human development’? And even if we can come up with meaningful proxy measures for these, how can we plausibly attribute any measurable changes to particular interventions? Projects take place within ‘open systems’ within which there are multiple endogenous factors that can enhance or impede success. And so how can we draw out the contribution of a single intervention to any (un)desirable change?

Second, although some stakeholders may have ambitious performance information expectations, these are limited by the pragmatics of time and budget constraints. There are both business and ethical reasons why M&E should be conducted economically. The business reality is that M&E budget allocations typically hover around 4% of aid project budgets (range between 1% and 10%). This pragmatically limits the extent and rigor of M&E processes. The ethical reality is that any resources invested in M&E, by definition erodes the resources that can be dedicated to the substantive work of the project (NB this is akin to a football game that invests in umpires and score-keepers to the detriment of players on the pitch).

Third, M&E practitioners are frequently faced with a range of practical dilemmas that they must balance in designing and implementing M&E arrangements:

- There is need for project-specific performance information, and yet there is a need for generic/scalable information to enable interpretation and aggregation from the program-wide perspective.

- There is need for brevity and simplification of performance issues, and yet the program performance issues are inevitably complex and require elaboration.

- There is need for quantitative information that can be aggregated and disaggregated to meet specific requirements, and yet this form of data frequently lacks meaning.

- There is need for qualitative information to shed light on beneficiary changes, and yet this form of data can not be readily aggregated or manipulated. - There are precise and anticipated information needs, and yet there are ad hoc or emergent information needs likely to arise.