A Simple Guide to ERDF Project Evaluation and Summative Assessment

DCLG has now published guidance on the preparation of Summative Assessments for ERDF funded projects across England. Regeneris Consulting helped DCLG to draft the guidance and we have a clear understanding of these requirements.

Since the guidance has been published, we have been approached by several ERDF projects wanting to know how it applies to them. We thought it would be helpful to set out the common questions that have cropped up and provide some quick pointers.

1 – Does the new guidance apply to my project?
Some grant recipients have asked whether they need to prepare a Summative Assessment if they have already submitted their project logic model to DCLG.
The simple answer is yes. All ERDF grant recipients need to undertake a Summative Assessment for their project, no matter how big or small. The logic model is just part of this assessment process.

2 – Is Summative Assessment the same as project evaluation?
Again, the simple answer is yes, it is. DCLG has called it a Summative Assessment rather than an evaluation as not all impacts will have arisen, or be measurable, by the time the assessment is undertaken. While this could be a challenge for some project types, all grant recipients will need to provide an analysis of the scale and nature of impacts they expect their projects to ultimately achieve as part of the Summative Assessment.

The guidance sets out that the Summative Assessment must cover all the usual strands of project evaluation:

• review of project design
• evaluation of delivery performance including performance against targets
• process evaluation including approach to delivery and management
• economic impact evaluation including achievement of outcomes and impacts for beneficiaries and the local economy targeted
• economic evaluation focusing on value for money to the funders.

In practice, the weight placed upon these strands may vary according to the nature and size of the project under review.

3 – How much resource should we devote to a Summative Assessment?
DCLG’s guidance is clear that the resources devoted to Summative Assessments should be proportionate and that up to 1% of eligible project expenditure can be spent on the exercise.
Although this is a useful rule of thumb, resource requirements will vary for different types of project. We would encourage all ERDF grant recipient s to invest enough of their resources to ensure robust and insightful evaluation evidence is generated.

After all, whilst we’re not sure what will replace the Structural Funds, it is highly likely that competition for resources will be stiff and evidence about the effectiveness of your project will improve its chances of securing a slice of the cake in future years.

4 – What type of methods are most appropriate to measure the impact of my project?
The new guidance reflects the increased emphasis being placed by government on robust evidence of economic impacts. This is most clear in the focus on Counterfactual Impact Evaluation (CIE) methods. In simple terms, CIE methods seek to use comparison areas or groups to isolate a project’s contribution to achieving economic and other impacts from all the other factors affecting local conditions.

CIE methods can be complex, particularly for projects with no direct beneficiaries such as those providing new infrastructure. Grant recipients should however be open minded about the possibility of using CIE methods as part of their Summative Assessments and should work with expert evaluators to explore the feasibility and appropriateness of these methods.

It is important to note that CIE methods are not always appropriate and can, if implemented poorly, lead to very misleading findings and waste considerable time, effort and resources. Grant recipients may decide that other impact assessment methods are more appropriate. These could include various research methods as part of a theory of change framework, although evaluators need to be careful about the weight placed on subjective opinions and recognise that, unlike a robust CIE approach, these methods do not match CIE methods’ robust attribution of impacts.

Next steps for your project evaluation…
It is important that to understand what has worked in the ERDF programme (and what didn’t) if the UK is to help develop more effective local growth programmes following Brexit.

Project evaluation is central to this, but it can be complex, and often requires experienced practitioners to help design and implement appropriate methods. This takes careful planning and ERDF grant recipients should plan ahead for their Summative Assessment In particular, we recommend that applicants carefully consider what data needs to be collected now to support a robust and insightful impact assessment in future.

The guidance is a good starting point for a grant recipient wishing to determine their evaluation requirements and procure their evaluators. Whilst the guidance also points the reader to more detailed evaluation guidance, we strongly recommend grant recipients work through these issues sooner rather than later and explore your ideas with experienced evaluators before procuring outside help.

If you want to know more about the Summative Assessment requirement and how it applies to your ERDF project please contact Neil Evans or Stuart Merali-Younger.