Incentives for performance in cohesion policy — reality or wishful thinking?
By Cristina Jianu, Investment for Cohesion, Growth and Inclusion Directorate, and Bernard Witkos, cabinet of Marek Opiola, ECA member
Cohesion policy is the EU’s main investment tool for promoting economic, social and territorial development. Increasing focus on the performance of cohesion policy has been on the agenda for many years, but it has gathered pace since the financial crisis in 2008, when shrinking public finances in many Member States called for increased attention to the impact of EU spending. The legislative package for 2014‑2020 introduced a series of developments as regards the performance orientation of cohesion policy, including a set of financial incentives to reward good performance and sanction under‑performance. Bernard Witkos, Attaché, and Cristina Jianu, Senior Auditor, respectively head of task and deputy head of task for the recent audit in this area, report on the audit that was published as ECA special report 24/2021, and consider the effectiveness of these incentives.
The long quest for incentives to improve performance in cohesion policy
Cohesion policy has undergone reforms aiming to strengthen result orientation over the last two decades. The most relevant changes concern two interlinked elements:
- the framework for measuring performance with output and result indicators to articulate what achievements are expected from investments and to enable monitoring; and
- mechanisms to incentivise performance and encourage authorities to pursue the priorities of cohesion policy.
Performance incentives have been used in cohesion policy since 2000. The legislative package for the 2000‑2006 period introduced a mandatory ‘performance reserve’ of 4% at OP level1 for the very first time. The idea was that such a mandatory performance reserve would retain a share of allocated funding until a satisfactory performance was demonstrated based on indicators. The instrument did not meet expectations, as it placed emphasis on spending rather than on other dimensions of performance, and was therefore abandoned in the 2007‑2013 period.
A major shift occurred in the 2014‑2020 programme period. This was triggered by criticism of insufficient alignment between cohesion policy and the Lisbon strategy, by mixed evidence of achievements in previous periods, and by the financial crisis of 2008. In 2009, the European Commission asked Fabrizio Barca, the former Director‑General of Italy’s Ministry of the Economy, to prepare an independent assessment of the EU’s cohesion policy and to submit proposals for reform for the post‑2013 period. The Barca report concluded that ‘orienting grants to results’ should be a reform priority, and suggested, among other things, that objectives and results should be clearly identified. The report recommended creating adequate incentives to link financing to verifiable results. The Commission further expanded these considerations in its budget review of 2010 and the new 10‑year Europe 2020 strategy for smart, sustainable and inclusive growth.
The 2014‑2020 legislative framework required Member States to set up a performance framework and identify objectives and indicators for output and results. In addition, the following specific instruments were introduced to provide Member States with financial incentives, and to optimise their use of EU funding:
- ex ante conditionalities (EACs);
- a performance reserve; and
- performance‑based funding, in particular Financing Not Linked to Costs (FNLTC).
All except the performance reserve were retained in the 2021‑2027 programme period, as shown in Table 1 below.
Table 1 — Performance incentivisation in cohesion policy
Carrots and sticks to incentivise better performance in 2014‑2020 cohesion policy
The three instruments examined all involve ‘positive’ financial incentives or rewards for meeting the required conditions and for satisfactory performance, but also the possibility of imposing financial sanctions in the form of payment suspensions, i.e. ‘negative incentives’, as described in Table 2 below.
Table 2 — Performance incentives in 2014‑2020 cohesion policy
Auditing these instruments
The preparatory work for the audit published as ECA special report 24/2021 Performance‑based financing in Cohesion policy: worthy ambitions, but obstacles remained in the 2014‑2020 period started in April 2020, at a time when lockdowns and restrictions had just begun in the Member States. From mid‑March 2020, the ECA introduced teleworking as a default working mode. While none of us had the slightest idea how long and how far‑reaching the COVID‑19 pandemic would be, we decided to adapt the audit to the new working conditions. Based on our previous audit work, and not least because of the Commission’s significant progress in digitalising cohesion policy, we were relatively confident that we would be able to obtain the necessary audit evidence without on‑the‑spot visits.
We should probably also mention that we were lucky enough not to have to start this audit from scratch. Previous horizontal audits covered performance orientation in cohesion policy at different stages of the 2014‑2020 programme period: ECA special report 2/2017 and ECA special report 15/2017, review 08/2019, and our annual reports for 2013 (chapter 10) and 2014 (chapter 3).
Our audit assessed whether:
- the performance-based instruments were well designed to incentivise performance and shift the focus towards achieving results;
- the Commission and the Member States used them effectively; and
- their use made a difference in the way cohesion funding was allocated and disbursed.
For this audit, we analysed data provided by the Commission and 14 operational programmes (OPs) from four Member States (Germany, Italy, Poland and Romania).
An attractive idea hampered by design flaws
Overall, we concluded that the Commission and Member States have been only partially successful in using the three instruments to make the financing of cohesion policy more performance‑oriented. Ex ante conditionalities (EACs) were designed to set the conditions for effective spending, but their assessment was a one‑off exercise without subsequent monitoring. The performance reserve was released almost exclusively on the basis of progress in spending and outputs. Financing not linked to costs (FNLTC) was hardly used.
These mixed results were largely the consequence of design weaknesses, in particular a lack of clarity about the rules. This led to difficulties in implementation and repeated revision of the rules throughout the programme period. These were less‑than‑ideal conditions for the uptake of new instruments. Performance‑based funding is an attractive idea in principle, but it needs to be codified in the rules that apply to cohesion policy funds. Robust design is critical for the success of such instruments.
Working on this audit allowed us to develop five common non‑exhaustive criteria that we consider critical to the effective functioning of incentives. For each of these criteria, we assessed the degree of fulfilment for the three instruments using qualitative scores: from ‘low’ (representing the poorest degree of fulfilment) to ‘high’ (representing the most satisfactory; see Table 3 below). These scores are only proxies that allow for a simpler presentation of the findings from the report.
Table 3 — Qualitative assessment of the functioning of incentives
a. Predictability of rules, including audit arrangements
The stability of rules affects medium‑ and long‑term planning. Modifying the initial rules should be possible to ensure flexibility, but without softening them to such an extent that it undermines their incentive effect; it should also be limited to exceptional cases. The basic rules for EACs were set out in the legislation, but the guidance clarifying how to assess progress was developed late, in parallel to programming. For the performance reserve, we pointed out that the Commission progressively modified the underlying rules, changing the conditions for assessment and adding flexibility for Member States. The first dedicated regulation specifying FNLTC for cohesion policy funds also left several key elements to the discretion of programme authorities, such as the amount of funding per unit of energy savings. Audit arrangements were also unclear. As programme authorities did not have experience of using this new instrument, and the Commission had not provided guidance or replicable examples, uptake was extremely low: only one pilot project was implemented in Austria by the end of the 2014‑2020 period.
b. Assessment criteria should be clear and known upfront
What constitutes good performance should be agreed at the start of a programme period. Uncertain and ambiguous criteria may lead to inconsistent assessments that can transform into a box‑ticking exercise. For EACs, we found that assessment criteria were broad, leaving considerable room for interpretation. There were no specific quantifiable targets even where such targets could be derived from EU legislation, and this led to instances of inconsistent assessment. Unlike in the 2000‑2006 period when the criteria for assessing performance were not set out in the OPs, the milestones for releasing the performance reserve in 2014‑2020 were defined for each priority axis at the beginning of the period in the adopted OPs.
c. Performance data need to be reliable and comparable
The foundation of a performance‑based system is the quality of underlying data. For EACs, we found that Member States had no obligation to provide information to the Commission that would allow a systematic and consistent assessment. For the allocation of the performance reserve, we were able to confirm the reliability of 89% of the indicators we examined. At the same time, we found that data quality varied between programmes. We have not examined the quality of data reported for the pilot FNLTC.
d. Enforcement should be credible
Changing the criteria that were initially set for the performance assessment, combined with sanctions that are unlikely to be applied, weaken the incentivising effect of performance‑based instruments. The Commission had the power to suspend payments if Member States failed to fulfil relevant EACs, if it became aware of serious deficiencies in the quality and reliability of the monitoring system or of the data on indicators, or for ‘serious failure’ to achieve at least 65% of the milestone value by 2018. For EACs, the Commission did not suspend any payments upon OP adoption, and subsequently applied payment suspensions for two OPs that had not completed their action plans. In the other two possible scenarios, the Commission launched several pre‑suspension procedures, but ultimately did not suspend any payments. We also concluded that out of the 5 802 indicators and related milestones used for the allocation of the performance reserve, Member States had modified more than half of them. Consequently, the Commission allocated 82% of the performance reserve, i.e. significantly more than the 56% of the reserve that would have been released if the indicators and milestones had not been amended.
e. Performance measurement should go beyond input and outputs (result orientation)
Pre‑defining output and result indicators to measure the contribution to EU objectives and to serve as a basis for payment can be challenging, but allocating EU funds based on absorption and progress on implementation is difficult to justify. EACs aim to create favourable conditions for investments from cohesion policy funds that contribute to generating results, although there is no direct link between the fulfilment of EACs and results. For the performance reserve, we established that it was primarily released on the basis of Member States’ success in spending cohesion policy funds and progress on implementation, as reflected by the delivery of key implementation steps and outputs rather than results (see Figure 1). For FNLTC, the dedicated regulation provided that funding be offered for results (kWh of energy savings or tonne of CO2 emission reductions). However, in practice it may also be disbursed for interim procedural steps (such as project selection).
Figure 1 — Indicators used in the 2019 performance review per fund
What can we learn for the future?
With the introduction of performance‑based funding instruments in the 2014‑2020 period, the EU took a new path that shifted the focus from inputs to results in cohesion policy. The outcomes of these efforts were mixed. Nevertheless, performance‑based funding has now become a dominant form of EU financing, as it is the default for the Recovery and Resilience Facility and is optional under cohesion policy. Our examination offers a number of useful lessons to strengthen these instruments in the future and make them more effective.
First, performance orientation needs to be more than just a concept. It requires careful preparation and operational models, including robust methods for estimating costs, and meaningful criteria for funding and for partial payments. It should not be forgotten that this innovative form of funding might not be suitable for all areas and investments. This is especially the case when there is a time lag between investment and the achievement of results, or when the delivery of results can be significantly influenced by external factors. Programme authorities or beneficiaries also bear higher financial risks, as payment is dependent on the delivery of results. Performance orientation also requires the selection of meaningful indicators that genuinely reward concrete achievements and ensure good value for money for the EU.
Second, control arrangements regarding compliance with internal market rules (public procurement and state aid) need to be clarified. While the control of performance‑based instruments should be limited to the delivery of agreed conditions or results, national authorities must ensure that internal market rules are applied correctly. An alternative system of checks needs to be developed in order to guarantee this.
Third, for such instruments to be attractive, they need to offer a lighter administrative burden and simpler implementation. However, setting up performance‑based instruments requires an initial investment in terms of administrative resources, and this should not be underestimated.
Overall, however, as already recognised by the OECD in 2017 (2), cohesion policy compares favourably with most OECD countries where financial incentives are rarely used in practice. Its performance orientation has been gradually increasing from one programming period to the next, through innovation and experimenting with new elements, some of which were subsequently even mainstreamed in other areas of the EU budget. As the EU’s external auditor, the ECA has contributed to and supported this development in the past, and we will undoubtedly continue to do so in the years to come.
(1) See Article 7(5) of Council Regulation (EC) No 1260/1999 of 21 June 1999 laying down general provisions on the Structural Funds: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A01999R1260-20050222.
(2) In 2017, in the OECD Journal on Budgeting, the OECD published a review entitled ‘Budgeting and performance in the European Union: A review in the context of EU budget focused on results’ as part of a comparative review on incentivising performance in public investment policies delivered at national and subnational levels.
This article was first published on the 1/2022 issue of the ECA Journal. The contents of the interviews and the articles are the sole responsibility of the interviewees and authors and do not necessarily reflect the opinion of the European Court of Auditors.