Performance Assurance:ERROR AND FAILURE Resolution(EFR) and THE EFR Escalation Process |
Guidance Note |
What this document covers
What the Error and Failure Resolution (EFR) Performance Assurance Technique (PAT) is and what it is intended to achieve;
How and when the technique is applied;
The actions Parties and Party Agents should take when they are being considered for EFR
The timescales for producing and reviewing an EFR plan once the technique has been deployed;
How EFR plans are monitored and how and when a plan can be closed;
When and how EFR plans can be escalated to the Performance Assurance Board (PAB) or BSC Panel, and the possible consequences should this happen.
What this document does not cover
Detailed advice on producing an EFR plan, which is included in a separate Guidance Note
Details of how to achieve compliance with the BSC, which can be found in the Code itself as well as the BSC Procedures (BSCPs)
If you have a more general question about EFR, please raise a ticket through the online
Elexon Support portal.
EFR is a
Remedial PAT used to assure Elexon, the PAB and the rest of the industry that Performance Assurance Parties (PAPs) understand performance issues and have robust plans in place to correct them in a timely manner. It is formally documented in
BSCP538: Error and Failure Resolution.
In particular, EFR is used:
In cases where Elexon and/or the PAB judge that a Party or Party Agent requires Elexon’s support in order to meet its obligations under the Code; and/or
To provide additional visibility and accountability around a Party or Party Agent’s plans where Elexon and/or the PAB judge that the continuing non-compliance poses a significant risk to the accuracy of Settlement, and therefore to the interests of all participants in the energy market.
PAPs in EFR are required to provide Elexon with a detailed plan showing the actions they will take to address the performance issue. This plan must include milestone dates for completing these actions, and forecasts of any associated improvement in performance metrics.
Failure to meet these milestones, or to rectify the performance issue within a reasonable timeframe, may result in escalation to the PAB or the BSC Panel.
For BSC Parties, escalation can lead to being placed in Default. This can result in the BSC Panel removing the Party’s right to register Metering Systems and, ultimately, in expulsion from the BSC.
For Party Agents, escalation can lead to the Removal of Qualification. This prevents an Agent from being appointed to operate in that role.
The guidance below is for PAPs that may have or have already had EFR applied. It sets out the steps within each level of the EFR process.
All Parties and Party Agents that are not compliant with the provisions of the Code should be taking action to address this and must provide regular updates to their OSM on the work they are undertaking. The most desirable outcome for PAPs is to address any issues prior to the deployment of EFR.
Settlement Performance EFR Plans
Elexon and the PAB deploy EFR for
Suppliers based on their Settlement Performance against the standards set out in
Annex S-1.2 of the BSC.
Typically, Elexon and the PAB consider Suppliers’ monthly performance in the following Market Segments:
Non Half Hourly (NHH) Measurement Class (MC) A and Half Hourly (HH) MC F Import (together as “NHH”);
HH MCs E & G Import (“Sub 100kW”);
HH MC C Import (“HH”); and
NHH MC A Export (“NHH Export”).
Each quarter, Elexon and the PAB agree a “Focus Threshold” for each of these Market Segments to ensure that EFR remains focussed on the
Suppliers contributing the greatest negative impact to overall Settlement Performance. These reviews are then made public through the
Elexon website.
Elexon and the PAB consider all Suppliers whose volume of monthly non-compliant estimation breaches these levels.
However, any Supplier with performance below the standards set out in the Code will be considered for EFR where Elexon or the PAB judge this appropriate.
Such considerations may include, but are not limited to:
Performance that is persistently below the standards set out in the Code, with little or no evidence that a Supplier is making sustained improvement towards those standards;
An increase in total volumes at earlier Settlement Runs, such that the current level of Settlement Performance is likely to result in a Supplier coming into focus for EFR if improvements are not made; and/or
An apparent or perceived lack of understanding of Settlement obligations and processes, including those of Party Agents, and of the potential impact of poor Settlement performance on other Parties.
Under MHHS, Elexon and the PAB will typically consider Supplier’s monthly Import and Export performance across three new Market Segments:
The PAB will continue to set quarterly “Focus Thresholds”, but the method by which Supplier’s Settlement Performance is calculated will change to take into account method of estimation in order to emphasise the importance of Suppliers’ using the best estimation method possible where actual data is not available.
The exact formula for this method is still under development but will be made available to all Suppliers at the earliest possible opportunity.
BSC Audit Issue EFR Plans
Elexon can deploy EFR for any BSC Party or Party Agent where the BSC Auditor has identified a material non-compliance with the Code.
In general, Parties and Party Agents should expect that Elexon will deploy EFR for all High or Medium BSC Audit Issues, and for any Low BSC Audit Issues that have:
Again, any Party or Party Agent with open Material BSC Audit Issues may be considered for EFR where Elexon or the PAB judge this appropriate.
Such considerations may include, but are not limited to:
Persistent non-Compliance with the provisions of the Code, with little or no evidence of sustained improvement towards Compliance; and/or
An apparent or perceived lack of understanding of Settlement obligations and processes, and of the potential impact of poor Settlement performance on other Parties.
Elexon and the PAB may choose to deploy EFR for any PAP based on the outcome of any other Performance Assurance Technique or on relevant information received from any other source or process.
In such cases, EFR will be deployed in a manner broadly consistent with the considerations outlined above.
What should I do if I am being considered for EFR?
As mentioned above, all Parties and Party Agents that are not compliant with the requirements of the Code should already be working to resolve this.
This means that – in most cases – you should already have a plan, or be working on one.
If you are being considered for EFR your OSM will contact you to discuss the situation. If you are already aware of an issue – for example, if you are a Supplier whose volume of non-Compliant estimation has fallen below the standard, or if you have multiple High or Medium BSC Audit Issues – then you may consider initiating this contact yourself.
Your OSM will review the actions you are already taking and provide you with any additional support or education you may require.
In general, EFR is less likely to be deployed in cases where:
The issue results from a small number of root causes that you have already identified; and
You already have a plan to take specific, time-bound actions to address these; and
These actions will be completed and the issue resolved as soon as possible, and within the next three months
Please remember that EFR is not a punishment or sanction. Elexon and the PAB may still deploy EFR even if they are confident that your existing plans are likely to address an issue in a timely manner if they think it necessary in order to provide additional accountability, visibility, and support if the plan runs into difficulties.
What should I do when EFR has been turned on?
When EFR is deployed, you will receive a formal letter from the PAB Chair asking you to provide your OSM with an action plan detailing the steps you will take to resolve the issue and the timescales for completion.
What are the timescales for agreeing a plan?
You have 20 Working Days to provide your OSM with a plan. Your OSM will then review your plan within 20 Working Days of receiving it to ensure that it is appropriate, robust and timely.
If your OSM agrees that your plan will address the issue within a reasonable timeframe, they will accept or “baseline” the plan. If not, they will return it to you with specific details of what they will require from you in order to accept it. In such cases, you have 10 Working Days to make these changes and provide an updated plan.
If you do not keep to these timescales, of if your OSM is not satisfied that that you have produced a credible, adequately resourced plan to return to Compliance by the end of this process, they will consider escalating you to the PAB. This process is covered in more detail under “When is an EFR plan escalated?” below.
Fig. 1, below, maps out this part of the process.
What should I include in my EFR Plan?
BSCP538 includes an EFR Plan template that can help you structure your plan, although you are not required to use it. Your OSM will also be available throughout the process to offer advice and guidance should you require it.
Your EFR Plan should demonstrate that:
You fully understand the cause or causes of the issues and its impact; and
You have identified specific actions required to address this cause or causes and prevent their recurrence; and
You have set clear timescales by when these actions will be completed; and
You have identified the improvements to performance that will result from these actions
Your plan must include the date by which you will return to compliance with the Code, and milestones that will allow your OSM to check that the plan is on track.
These milestones should be a mixture of:
“Action” milestones, showing improvements to systems or processes (e.g. ‘root cause analysis completed’ or ‘software fix implemented’); and
“Performance” milestones, showing improvements against a particular performance metric (e.g. ‘the backlog of exception dataflows will reduce to 1,000’ or ‘the percentage of energy Settled on Annualised Advances will improve to 97%’).
You plan should clearly set out how the performance milestones will result from the action milestones (e.g. ‘3% of our energy is Settled on Estimated Annual Consumption from sites that are scheduled to have a Smart meter installed by the end of January. Our success rate for completing scheduled installations is 50%. We are therefore forecasting a 1.5% increase from this workstream in the percentage of our energy Settled on Annualised Advances by the end of January.’).
Milestones should be no more than three months apart, and Settlement Performance plans must include monthly performance milestones showing the forecast improvement up until the point at which you will achieve Compliance.
Your OSM will agree with you the best way to provide evidence that you have completed action milestones. You will need to provide the agreed evidence within five working days of the milestone date.
Your EFR plan must be signed off by someone within your organisation who is able to allocate the necessary resource and budget for the plan to succeed.
Fig. 1: Agreeing an EFR plan
How will my EFR Plan be monitored, and how do I exit EFR?
If you don’t already meet with your OSM on a regular basis, you should set up meetings to provide regular updates on the progress of your plan. Your OSM will work with you to agree how often these should take place.
If you already have regular meetings with your OSM, you should review the frequency and length of these meetings to make sure you have enough time to discuss the progress of your plan as well as any other issues you wish to raise.
During these meetings, you should provide your OSM with an update on the progress of the plan. These updates should be as specific as possible and, where appropriate, include evidence of the progress you are making.
For example, if you have set a target of reducing a backlog to 1,000 items by the end of the quarter, you should provide details of:
The number of items currently in the backlog;
How much it has reduced since the last meeting; and
how much it will need to have reduced by the next meeting if your plan is to remain on track.
Your OSM may raise additional questions and concerns. It’s important that you address these as quickly as possible, ideally in time for your next meeting.
Within 5 working days of each milestone, you must provide your OSM with evidence that you achieved it. Your OSM will also check any available monitoring information (such as Settlement Performance reports) to confirm that a milestone has been achieved.
Failing to provide the agreed evidence within this timescale will always count as a missed milestone, even if it Elexon are subsequently able to confirm that you had carried out the action or achieved the required level of performance.
If you miss a milestone, or if it becomes apparent that you are not going to be able to meet your next milestone, you will be considered for escalation to the PAB.
Fig. 2, below, maps out this part of the process.
Once you have completed all of the milestones in your plan and returned to compliance, Elexon and the PAB will consider turning off the EFR technique. Where EFR has been deployed for a BSC Audit Issue, we will normally wait for the outcome of the subsequent BSC Audit to provide additional assurance that the issue has been addressed.
Once the PAB are satisfied that you have achieved a sustainable level of Compliance with your obligations, you will receive a formal letter from the PAB Chair advising you that EFR has been turned off.
This letter may also contain additional observations or actions for you to complete. If so, your OSM will discuss next steps.
Fig 2: Monitoring and closing an EFR plan
When is an EFR plan escalated?
If you show insufficient engagement with the EFR process, or make insufficient progress towards addressing the issue, you will be considered for escalation to the PAB.
Examples of insufficient engagement or progress can include, but are not limited to
Failing to agree a credible, adequately resourced plan with your OSM within the timescales set out above;
Missing a milestone within your plan;
Missing the deadline to provide evidence showing you have met a milestone
If you are being considered for escalation your OSM will contact you to discuss this. You should provide your OSM with as much detail as possible setting out why you have missed the milestone or have not been able to produce a plan or evidence on time. Elexon and/or the PAB will then make a judgement as to whether a formal escalation is required.
If a formal escalation is required, you will receive a letter from the PAB Chair inviting you to a meeting in person at the Elexon offices.
In general, PAB delegates the first escalation for a missed deadline or milestone to the EFR PAT Owner within Elexon’s Assurance Product, and the second escalation for a missed deadline or milestone to Elexon’s Head of Assurance. After this, the invitation will be to a meeting with the PAB itself, or to the BSC Panel at the PAB’s discretion.
Elexon may suggest holding delegated escalation meetings at your premises in order to give you an opportunity to demonstrate first-hand the work you are doing to meet the objectives of you plan.
The decision to delegate some escalations is at the PAB’s discretion, and you may be escalated directly to the PAB if Elexon or the PAB judge this more appropriate. This is particularly likely if your OSM finds that you are insufficiently engaged with the process or in cases where your non-compliance is likely to cause significant impact to the rest of the market.
In particular, escalation of Settlement Performance EFR plans will always be fast-tracked to the PAB where the most recent monthly Settlement Performance data for the relevant Market Segment shows the Supplier is responsible for:
More than 20% of overall non-compliant estimation; and/or
A proportion of overall non-compliant estimation more than 10% greater than their share of overall volume; and/or
More than 20,000 MWh of non-compliant estimation.
Escalations that the PAB has delegated to Elexon are still part of the formal PAB escalation process, and you should prepare for them as though you were attending a meeting of the PAB.
In particular, the person who signed off the plan must attend all escalations. Remember that this person should be someone with sufficient authority to sign off any resources and budgets required for the plan to succeed.
How should I Prepare for a PAB Escalation?
In advance of the escalation meeting, you should produce a revised EFR plan and a short presentation summarising any key points you wish to highlight to the PAB or its delegated representatives.
In addition to updates on the initiatives from your original plan and any new workstreams you have identified, you should ensure this revised plan includes:
Details of the underlying root cause(s) for your failure to reach the expected milestones set out in the original Plan;
Details of the actions you will take to address these and prevent their re-occurrence; and
Either confirmation that you will meet the milestones in the original Plan or a revised set of milestones up to the date by which you will become Compliant.
If your escalation is directly to the PAB, you should provide your OSM with both the revised plan and any slides you wish to present no later than 7 Working Days before the meeting you will attend. This is to ensure that the PAB Members have ample time to review the plan before you present it to them.
If you would like Elexon to provide feedback on the plan and presentation before circulating them to PAB Members, please provide them to your OSM no later than twelve Working Days before the meeting.
What will happen during and after a PAB Escalation?
If you are attending a PAB Meeting in person, your OSM will arrange a room for you to work in ahead of your presentation on the day. The PAB often has a busy agenda and items can overrun, so it’s best to allow additional time for any delays when booking your return journey.
When it is time for your presentation, your OSM will provide the PAB with a short overview of the progress of your EFR plan to date. You will then be invited in to the meeting to present your slides.
Once the PAB has asked any questions about your revised plan and presentation, they will take some time to deliberate before calling you back in to inform you of their decision.
Where the PAB has delegated an escalation to the EFR PAT Owner or Head of Assurance the meeting will usually take place remotely or, less frequently, in your own offices. In these cases, the PAB’s delegate will either inform you of their decision during the call, or – if they need more time to deliberate – ask your OSM to contact you with their decision.
In either case, you will receive a formal letter documenting the outcome within 5 Working Days of the escalation meeting.
Having considered your revised Plan, the PAB (or its delegate) will either:
When rejecting a plan, the PAB may also decide to escalate a Party to the BSC Panel.
The PAB may ask you to return to future meetings to provide further updates. If the PAB is unsatisfied that the person who signed off on the plan is of sufficient seniority to ensure its success, it may also require that a Director, CEO or CFO or other senior executive attend a meeting in order to provide assurance that the plan has appropriate support within your organisation.
Fig. 3, below, maps out this part of the process.
Escalation to the BSC Panel
If the PAB decides that you should be escalated to the BSC Panel, you will receive a letter from the PAB Chair informing you of this.
As with a PAB escalation, you will be required to prepare a revised plan and presentation. You should provide your OSM with both the revised plan and any slides you wish to present no later than seven Working Days before next the PAB meeting (12 Working Days if you would like Elexon to provide feedback on the plan and presentation before circulating them to PAB Members).
At its next meeting, the PAB will decide whether to recommend that the BSC
Panel approve your revised plan. It will also decide whether to recommend that the BSC
Panel issue a Public notification of your failure to make sufficient progress with your EFR plan under the provisions of
Modification P427. You will not usually be required to attend this meeting.
On the day of the meeting, your OSM will arrange a room for you to work in ahead of your presentation. Like the PAB, the Panel often has a busy agenda and items can overrun, so it’s best to allow additional time for any delays when booking your return journey.
When it is time for your presentation, your OSM will provide the Panel with a short overview of the progress of your EFR plan to date. You will then be invited in to the meeting to present your slides.
Once the Panel has asked any questions about your revised plan and presentation, they will take some time to deliberate before calling you back in to inform you of their decision.
Having considered your revised Plan and the PAB’s recommendation, the Panel will either:
When accepting a plan, in whole or part, the Panel will also decide whether any subsequent escalations will be to the PAB or to the Panel itself.
If you are a BSC
Party, the
Panel may also decide to begin the
Breach and Default process when rejecting a plan. This can result in the
Panel removing your right to register
Metering Systems and, ultimately, in expulsion from the BSC.
If you are a
Party Agent, the
Panel may decide to begin the
Removal of Qualification process when rejecting a plan. This will prevent
Suppliers and Registrants from appointing you to operate in that role.
Regardless of whether the plan is accepted or rejected, the Panel may decide to issue a Public notification of your failure to make sufficient progress with your EFR plan.
Fig. 4, below, maps out this part of the process.
Do you need more information?
Your
OSM is your first point of contact for EFR and is available to support you through the process. If you’re not sure who they are, check the
BSC Signatories and Qualified Persons page of our website.
For any other information please raise a ticket through the online
Elexon Support portal.
Intellectual Property Rights, Copyright and Disclaimer The copyright and other intellectual property rights in this document are vested in Elexon or appear with the consent of the copyright owner. These materials are made available for you for the purposes of your participation in the electricity industry. If you have an interest in the electricity industry, you may view, download, copy, distribute, modify, transmit, publish, sell or create derivative works (in whatever format) from this document or in other cases use for personal academic or other non-commercial purposes. All copyright and other proprietary notices contained in the document must be retained on any copy you make. All other rights of the copyright owner not expressly dealt with above are reserved. No representation, warranty or guarantee is made that the information in this document is accurate or complete. While care is taken in the collection and provision of this information, Elexon Limited shall not be liable for any errors, omissions, misstatements or mistakes in any information or damages resulting from the use of this information or action taken in reliance on it. |
