Performance Assurance: Menu of Performance Assurance Techniques |
Guidance Note |
What this document covers
What this document does not cover
Detailed advice on the operation and application of the Techniques themselves, which can be found in the dedicated Guidance Notes for each of the PATs
Details of the framework of Settlement Risks that Elexon and the Performance Assurance Board (PAB) use to inform the scope and deployment of Techniques during each Performance Assurance Operating Period (PAOP), which is included in a separate Guidance Note
Your Operational Support Manager (OSM) is your first point of contact for Performance Assurance queries and is available to support you.
If you have a more general question about Performance Assurance, please raise a ticket through the online
Elexon Support portal.
What are Elexon’s Performance Assurance Techniques (PATs)?
Elexon’s PATs are a set of specific provisions and processes set out in the
Code and the
BSC Procedures (
BSCPs) to mitigate the impact of
Settlement Risk.
They are broadly divided into four categories:
Preventative Techniques, that lessen the chances of Settlement Risk materialising;
Detective Techniques, that monitor for instances of material Settlement Error and check that Parties and Party Agents have appropriate controls in place as set out in the Code;
Remedial Techniques, that correct Settlement Errors that have already occurred and address failures or gaps in processes; and
Incentive Techniques, that motivate Parties and Party Agents to take steps to prevent, detect and/or correct the impacts of Settlement Risk
The PATs are designed to work together as part of Elexon’s Performance Assurance Framework (PAF). Some specific examples of the ways PATs can interact with each other are given below.
All PATs are overseen by the Performance Assurance Board (PAB). The majority are operated by Elexon’s Performance Assurance product. Where this is not the case, the responsible team within Elexon is identified in the description below.
How and when are PATs applied?
The PATs are deployed in line with the Risk Operating Plan (ROP) for the Performance Assurance Operating Period (PAOP) covering each financial year from April to March.
Some PATs are deployed every year on an annual cycle, although their scope may vary from year to year. Other PATs are deployed in response to specific events, including the outcomes of other PATs.
Qualification and Re-Qualification
All new BSC Parties and Party Agents are required to Qualify before they can operate in the Market. During this process, the Party or Agent is required to demonstrate to the PAB that its systems and processes are sufficient for it to meet its BSC Obligations.
Parties (other than Suppliers) and Agents may also have to re-Qualify on an ad hoc basis if they make material changes to their systems and processes. Parties and Agents who are making changes to their systems and processes are required to carry out a risk and impact assessment of the change to see if this is necessary.
The
Qualification process is set out in
BSCP537 and managed by Elexon’s
Qualification Team and KPMG as the
Qualification Service Provider (QSP).
When Qualifying a Party, the PAB may recommend additional follow-up checks of the Party or Agent’s live operations are carried out during the BSC Audit or through the Assurance Information Request (AIR) or Technical Assurance of Performance Assurance Parties (TAPAP) PATs.
The PAB can also determine that a Party (other than a Supplier) or Agent must re-Qualify based on information obtained through a Detective Technique such as the BSC Audit or disclosed as part of an Error and Failure Resolution (EFR) Plan.
Following the transition to Market Wide Half Hourly Settlement (MHHS), all Parties and Agents (including Suppliers) will be required to undergo Renewal of Qualification on a periodic basis. This will:
avoid “Trigger’s Broom” or “Ship of Theseus” scenarios in which a Party or Agent makes multiple piecemeal changes that add up to a material change, but are never large enough by themselves to warrant re-Qualification;
hold Parties accountable for a lack of disclosure of large changes that they may have implemented since their Qualification was first awarded; and
make the Re-Qualification process more frequent, yet less onerous, to encourage parties to implement necessary system changes when required
Elexon provides education and training on any aspect of
Settlement free-of-charge to any BSC
Party or Agent and organises regular webinars and workshops where Parties and Agents can learn more about their BSC obligations and share best practice. Elexon also maintains a library of
Guidance Notes as part of the Digital
Code.
Elexon may sometimes mandate Education as a Remedial Technique for non-Compliances with the Code detected through other PATs such as the BSC Audit.
Education is delivered by Elexon’s OSM team, in conjunction with appropriate Subject Matter Experts (SMEs) from across the business. Parties or Agents wishing to arrange training should contact their OSM.
Elexon is reviewing its Guidance Notes and other training materials in preparation for the transition to MHHS.
The Bulk Change of Agent Technique ensures that traffic to Supplier Meter Registration Service (SMRS) systems remains at manageable volumes when Suppliers make changes to their Appointed Agents across multiple Metering System ID (MSIDs).
Suppliers intending to carry out changes to more than 20,000
MSIDs per day – either in a single
GSP Group or in aggregate across two or more
GSP Groups – are first required to obtain permission from the PAB by following the process set out in
BSCP513.
Suppliers who exceed this limit without first obtaining permission from the PAB may be subject to further Performance Assurance action, such as the EFR or Public Notification of Non-compliance Techniques.
The basic principles of Bulk Change of Agent will remain the same during and after the transition to MHHS.
Performance Assurance Reporting and Monitoring System (PARMS) & Material Error Monitoring (MEM)
PARMS aggregates quantitative data submitted each month by Supplier Meter Registration Agents (SMRAs), the Supplier Volume Allocation Agent (SVAA) and the Central Data Collection Agent (CDCA) into information about specific Settlement Risks and BSC Non-Compliances (the “PARMS Serials”).
In the past, industry Participants were required to submit significant amounts of
PARMS data. These “participant-reported”
Serials were removed under
Modification P429 in February 2022.
The remaining PARMs Serials are:
SP07 (submitted by SMRAs and the SVAA)
TA01, TA02, SP07, SP08 and SP09 (submitted by the SVAA)
CM01 and CM02 (submitted by the CDCA)
PARMS requirements, including the information aggregated in each
Serial, are set out in
BSCP533.
PARMS data is used in the calculation of Supplier Charges and can be made Public through the Peer Comparison Technique.
Poor Settlement Performance, reported through Serial SP08, is one of the most common reasons Suppliers are placed in EFR. Less frequently, PARMS data can be the trigger for the deployment of the AIR and TAPAP Techniques.
Where PARMS does not provide sufficient quantitative data on a Settlement Risk with a significant material impact, Elexon and the PAB can mandate the provision of additional data under the Material Error Monitoring (MEM) PAT.
MEM data quantifies specific indicators of Settlement Error which Parties and Party Agents are required to work to reduce. Failure to do so can highlight where Parties or Agent may benefit from further education or training and/or lead to a Party or Agent entering EFR.
Under MHHS, PARMS and MEM will be transitioned into a single PAT, Performance Assurance Monitoring (PAM).
The BSC Audit is the largest non-financial Audit in Europe, and checks BSC Parties’ and Party Agents’ compliance with their obligations under the Code by through a series of workpapers that review statistical samples of Parties’ and Agents work across a range of Settlement business processes.
The scope of the Audit is set for each PAOP by the BSC Panel on the advice of Elexon and the PAB. When setting the Scope, Elexon and the PAB consider a range of factors including the findings of the previous year’s Audit, changes to the BSC, and issues identified by other PATs.
Any non-compliances with the BSC or BSCPs found through the Audit are recorded as “BSC Audit Issues” and classified by their impact to Settlement as “High”, “Medium” or “Low”. Weaknesses in controls or failures to follow best practices that do not result in a material impact to Settlement are recorded as “Management Letter Points” or MLPs.
All Parties and Party Agents with open BSC Audit Issues are required to take action to close them and must provide regular updates to their OSM on the work they are undertaking to do so. Where necessary, Elexon will offer support through the Education Technique. Elexon may also deploy the EFR Technique to provide additional transparency and accountability around a Party or Party Agent’s plans.
Where the BSC Audit suggests that Elexon may need to know more about a particular process or Settlement Risk, it may be the trigger for the deployment of the AIR or TAPAP PATs.
The incidence of Audit Issues associated with particular Settlement Risks provides Elexon with quantitative data for the Risk Evaluation Register (RER).
The BSC Audit is carried out by KPMG as the BSC Auditor, with the support of Elexon’s Assurance and OSM teams.
Elexon and the BSC Auditor are currently reviewing all BSC Audit workpapers in preparation for the transition to MHHS.
Technical Assurance of Metering Systems (TAM)
The TAM Technique Audits a selection of Metering Systems through a combination of site visits and desktop Audits of the records held for Metering Systems by the Parties and Party Agents responsible for them.
The scope of the Audit is set for each PAOP by the Panel on the advice of Elexon and the PAB. When setting the Scope, Elexon and the PAB consider a range of factors including the findings of the previous year’s Audit, changes to the BSC, and issues identified by other PATs.
At present, the Audit typically consists of:
A Supplier Volume Allocation (SVA) Main Sample of Half Hourly (HH) metering systems in Measurement Class C (MC C). The size of the sample is calculated to provide a reasonable chance that the instance of non-compliance within the sample is representative of the whole MC C metering population. Typically, this equates to ~1% of the total MC C Metering Systems. Metering Systems are chosen at random, and the sample cannot intentionally be weighted towards any one Party or Party Agent.
One or more Specific Samples, at least one of which covers metering systems in the Central Volume Allocation (CVA) market. The CVA metering population is too small for a representative sample to be selected for inspection using the statistical methodology used for the SVA Main Sample. Instead, inspections focus on the detection of high-volume Settlement errors through targeting metering systems most at risk, as set out in the Scope.
These samples can be supplemented by Targeted Visits of any Metering System where Elexon or the PAB have reason to suspect a metering error is causing a material impact to Settlement.
Any non-compliances with the BSC or BSCPs found through the Audit that have a material impact on Settlement are recorded as “Category 1” (or “A”, for desktop Audits). Weaknesses in controls that do not result in a material impact to Settlement are recorded as “Category 2” or “B”. The Auditor may also note where best practices have not been followed.
All Parties and Party Agents with open Category 1 or A non-compliances are required to take action to close them as soon as possible and must provide regular updates to their OSM on the work they are undertaking to do so. Where necessary, Elexon will offer support through the Education Technique.
TAM and the associated Audit checks are set out in detail in
BSCP27.
Where the TAM Audit suggests that Elexon may need to know more about a particular process or Settlement Risk, it may be the trigger for the deployment of the AIR or TAPAP PATs. Where the TAM Audit shows a widespread incidence of non-compliance with a particular BSC Process, it may be the trigger for Education or a BSC Change. The material impact of a Category 1 or A non-compliance may sometimes require correction through a Trading Dispute.
As the SVA Sample is chosen at random, and because the sample is sized to be representative of the industry as a whole, it does not provide a view of the performance of particular Parties or Party Agents. For this reason, TAM Audit outcomes are seldom a trigger for the deployment of EFR.
The incidence of Audit non-compliances associated with particular Settlement Risks provides Elexon with quantitative data for the Risk Evaluation Register (RER).
The TAM Audit is carried out by C&C Group as the Technical Assurance Agent (TAA), with the support of Elexon’s Assurance, Metering and OSM teams.
Upcoming changes to BSCP27 will remove references to “HH” and “MC C” as these will no longer be applicable categories following the transition to MHHS. However, the SVA Main Sample will continue to focus on the largest supplies by volume, with the sample population defined each year in the Audit Scope.
Assurance Information Request (AIR) & Technical Assurance of Performance Assurance Parties (TAPAP)
The AIR and TAPAP are supplementary Techniques that Elexon and the PAB can deploy to gather additional information to understand how and to what extent Parties and Party Agents are able to meet their Code obligations.
An AIR requires Parties and Party Agents to provide Elexon with quantitative data on a specific business process or Settlement Risk. A TAPAP is a remote or in-person Audit through which Elexon carries out a “deep dive” into a particular Risk or process area.
The process for raising an AIR is set out in
BSCP605; the process for conducting a TAPAP in
BSCP535.
Both Techniques are typically deployed to investigate areas of concern identified through the PARMS, MEM, BSC Audit and TAM Techniques, or in anticipation of a foreseeable market event likely to impact specific Settlement Risks or business processes.
Deployment of AIR or TAPAP should always be followed by the deployment of at least one other PAT in response to the findings. Common examples include Education, EFR, or BSC Change.
The basic principles of both Techniques will remain the same during and after the transition to MHHS.
Error and Failure Resolution (EFR)
The EFR Technique is used to assure Elexon, the PAB and the rest of the industry that Parties and Party Agents have robust plans in place to correct performance issues in a timely manner.
In particular, EFR is used:
In cases where Elexon and/or the PAB judge that a Party or Party Agent requires Elexon’s support in order to meet its obligations under the Code; and/or
To provide additional visibility and accountability around a Party or Party Agent’s plans where Elexon and/or the PAB judge that the continuing non-compliance poses a significant risk to the integrity of Settlement, and therefore to the interests of all participants in the energy market.
Parties and Party Agents in EFR are required to provide Elexon with a detailed plan showing the actions they will take to address the performance issue. This plan must include milestone dates for completing these actions, and forecasts of any associated improvement in performance metrics.
Failure to meet these milestones, or to rectify the performance issue within a reasonable timeframe, may result in escalation to the PAB or the Panel.
The EFR and EFR Escalation Process is formally documented in
BSCP538.
EFR is commonly deployed to rectify Settlement Performance identified through PARMS or MEM data. The criteria for EFR deployment for poor Settlement Performance are reviewed on a quarterly basis by the PAB and made available on the Elexon website.
EFR is also deployed to ensure Parties and Agents rectify their BSC Audit Issues. It is often used in conjunction with the Education Technique.
Where a Party or Party Agent is repeatedly escalated to the PAB or Panel, EFR can lead to Breach & Default (for Parties) or Removal of Qualification (for Party Agents).
The detail provided by Parties and Party Agents in their EFR Plans can provide qualitative and quantitative data for the RER.
The basic principles of EFR will remain the same during and after the transition to MHHS.
Trading Disputes are used to correct the material impact of Settlement Error outside of the timetabled Reconciliation Runs.
In most cases, this is because correcting the error requires amending historic data for Settlement Days that have passed the Final Reconciliation Run (“RF”) and involves scheduling an additional (“DF”) Reconciliation Run to process the amendments.
Rarely, the Technique can also be used to make an immediate correction to Parties’ invoices (an “Extra Settlement Determination” or “ESD”), for example if the financial impact of an Error to one or more BSC Parties were so severe that waiting for the next Reconciliation Run to correct it would endanger their ability to continue trading.
Trading Disputes must meet certain criteria to progress. They are reviewed and authorised by the
Trading Disputes Committee (TDC) under the provisions of
BSC Section W and
BSCP11.
Trading Disputes are usually raised by BSC Parties, but Elexon can also raise Trading Disputes and will sometimes do so as a precaution where a significant Settlement Error is suspected (for example when the TAA records a Category 1 or A non-compliance).
Elexon maps the underlying causes of Settlement Errors resolved through Trading Disputes to specific Settlement Risks to provide quantitative data for the RER.
The basic principles of Trading Disputes will remain the same during and after the transition to MHHS.
Supplier Charges are liquidated damages that Suppliers incur if they fail to meet the Performance Standards set out in the Code and measured through the SP08 PARMS Serials (SP08a, SP08b, and SP08c).
Should Suppliers fail to meet the Performance Standards measured by the SP08 Serials, Elexon applies a fixed charge per Serial and Settlement Run to each MWh of underperformance. The charges then are redistributed to other Parties disadvantaged by those who are not meeting the standards, with 90% redistributed to NHH Suppliers and 10% to Trading Parties (that is, Generators).
The full methodology for the calculation of Supplier Charges is set out in BSCP536.
In the past, Supplier Charges made use of the participant-reported PARMS serials discontinued under Modification P429.
Aside from the relevant PARMS Serials, Supplier Charges are not directly linked to any other PAT, although there is a natural correlation between Suppliers’ underperformance against the Performance Standards, the Supplier Charges they incur, and the likelihood of EFR and Education being deployed to help improve their Settlement Performance.
The basic principles of Supplier Charges will the same during and after the transition to MHHS, although the methodology used to calculate them will undergo significant changes, in particular:
Supplier Charges will no longer be linked to the Settlement Performance standards or PARMS Serial SP08;
Charges will be dynamic rather than fixed, meaning that better overall Industry performance will result in a smaller “pot” of monies to be redistributed; and
All monies collected will be redistributed to Suppliers, not Trading Parties.
The methodology underpinning MHHS Supplier Charges will be made available to the industry in advance of the changes taking effect.
Elexon, the PAB, and BSC Parties can propose changes to the BSC and
Code Subsidiary Documents (CSDs) such as the
BSCPs at any time through the
BSC Change process.
For this reason, BSC Change is considered a Performance Assurance Technique in cases where it is required because the current provisions of the Code and CSDs are insufficient to mitigate a particular Settlement Risk.
A proposal for BSC Change is one of the possible outcomes of the AIR or TAPAP Techniques.
The basic principles of the BSC Change process will remain the same during and after the transition to MHHS.
Peer Comparison & Public Notification of Accountability
Under
Modification P427, implemented in February 2023, Elexon and the PAB may recommend that the
Panel authorise the publication of data or information relating to a
Party or
Party Agent’s contribution to a
Settlement Risk.
Elexon currently publishes monthly Peer Comparison data on
Supplier’s compliance with the
Settlement Performance Standards set out in
BSC Annex S-1, and on the progress
Supplier and
Unmetered Supplies Operators (
UMSOs) are making in migrating Unmetered Supplies to Half Hourly
Settlement under
Modification P434.
BSCP533 allows for the publication of monthly and quarterly Peer Comparison data from
PARMS serials. Elexon currently publishes information from
Serials SP08 and SP09.
Modification P427 also allows Elexon and the PAB to recommend that the Panel notify the industry when a Party is accountable for a Settlement Error above de minimis thresholds agreed by the Panel.
These thresholds are currently set at:
In most cases, the Techniques are deployed where Parties and Party Agents have failed to improve Settlement Performance or address non-Compliances with the Code following the deployment of Education and/or EFR.
The basic principles of both Techniques will remain the same during and after the transition to MHHS.
Breach and Default & Removal of Qualification
Where Parties or Party Agents demonstrate that they are unable or unwilling to meet their obligations under the Code, the PAB and Panel may invoke either the Breach and Default or Removal of Qualification processes to protect other Parties and Party Agents and the overall integrity of Settlement.
The Breach and Default process applies to BSC Parties and Licensed Distribution System Operators (LDSOs), whilst Removal of Qualification applies to Party Agents.
The Panel will issue a Public Notice that the Party or Agent is in Breach of the Code or has entered the Removal of Qualification process (as appropriate) and may take further actions that restrict its ability to operate in the market.
Except in the most extreme cases, these Techniques will only be deployed where a Party or Party Agent has failed to address a non-Compliance with the Code following the deployment of EFR and repeated escalations to the PAB and/or Panel.
The basic principles of both Techniques will remain the same during and after the transition to MHHS.
Do you need more information?
Your
OSM is your first point of contact for Performance Assurance queries and is available to support you. If you’re not sure who they are, check the
BSC Signatories and Qualified Personspage of our website.
For any other information please raise a ticket through the online
Elexon Supportportal.
Intellectual Property Rights, Copyright and Disclaimer The copyright and other intellectual property rights in this document are vested in Elexon or appear with the consent of the copyright owner. These materials are made available for you for the purposes of your participation in the electricity industry. If you have an interest in the electricity industry, you may view, download, copy, distribute, modify, transmit, publish, sell or create derivative works (in whatever format) from this document or in other cases use for personal academic or other non-commercial purposes. All copyright and other proprietary notices contained in the document must be retained on any copy you make. All other rights of the copyright owner not expressly dealt with above are reserved. No representation, warranty or guarantee is made that the information in this document is accurate or complete. While care is taken in the collection and provision of this information, Elexon Limited shall not be liable for any errors, omissions, misstatements or mistakes in any information or damages resulting from the use of this information or action taken in reliance on it. |
