Understanding Settlement Risk 28 and How to Improve My Performance |
Guidance Note |
This guidance note is for Suppliers, Half Hourly Meter Operator Agents (HHMOAs) and Half Hourly Data Collector (HHDCs) to help analyse and address the root causes of the non-compliances (NCs) raised by the Technical Assurance Agent (TAA) to measure Settlement Risk 28.
1. What is Settlement Risk 28?
"The risk that HHMOAs make changes to the Metering System and do not inform the HHDCs resulting in Meter readings being misinterpreted or not collected."
SR0028 is currently assigned a severity rating which puts it in the category of “top” Settlement Risks, which the PAB monitors monthly. It has a high net significance (12) with low levels of controls and is one of the top risks that Elexon and the industry is focusing on to improve.
We look for relevant performance metrics for all top risks. Some of the Technical Assurance Agent’s findings are deemed relevant to SR0028. This guidance note explains how the metrics are used to measure Supplier and Meter Operator (MOA) performance and what you can do to improve your performance.
The Performance Assurance Board (PAB) has directed Elexon to work with Suppliers and HHMOAs to ensure that both are addressing this risk effectively. Elexon currently report to the PAB each month on each Supplier’s and HHMOA’s performance against this risk through the ‘Settlement Risk Report’. Since June 2010, Elexon has been requesting corrective action plans from Suppliers and HHMOAs highlighted as the worst performers against this risk in line with the Business Unit Settlement Risk Rating (BUSRR) criteria for SR28, used in the Settlement Risk Report.
In order for HHMOAs and Suppliers to address SR28 effectively, we are providing this guidance. It tells you what has been identified as non-compliant by the TAA and possible root causes. This will allow you to focus your actions so that you can reduce the probability and severity of the risk occurring, improving your SR28 performance and industry processes.
Root cause analysis is not aimed at closing the non-compliances the TAA has raised. Instead, it aims to identify the causes of problems or events. It works on the basis that problems are best solved by attempting to correct or eliminate those causes, rather than addressing the immediately obvious symptoms.
2. Where does the data that measures Settlement Risk 28 gathered come from?
The process supporting the Technical Assurance of Metering technique and the Technical Assurance Agent (TAA) provides us with the raw data.
The TAA inspects a 1% sample of the Supplier Volume Allocation (SVA) market; this is around 1200 inspections a year. The audit year runs from April to March. All the necessary data (Metering System records and information, visit schedules, inspection results and rectification details) is held on a database that all customers can access and update as required, if you are required to. The data for measuring SR0028 is extracted from that database.
3. What types of Market Participants are measured on SR0028
Suppliers and Meter Operator Agents (MOAs).
Non-compliances are assigned by the TAA to the responsible party. For example, a mismatch in a Key field between the HHDC MTDs and the TAA will result in a Category 1.01 non-compliance. This failure is assigned to the HHDC (because it was highlighted in the DC MTDs), though will require collaboration with the MOA to resolve the non-compliance (because it is the MOA who holds the source MTDs).
It is the Supplier’s obligation to ensure that the data entering Settlement is complete and accurate. It is the Supplier’s responsibility to ensure its agents perform to meet the obligation.
It is the MOA’s responsibility to ensure that the Metering System is recording data accurately and in a timely manner. The MOA is also responsible for ensuring that the MTDs are up to date and accurate and have been delivered to all appropriate parties, as set out the BSC.
4. How do we turn the data into the Settlement Risk 28 measure?
To turn TAA non-compliance data into the Settlement Risk 28 measure we use statistics over a rolling 12 month period. The report dated May 2014 will use data from the period 1 May 2014 to 30 April 2013.
Why? 12 months of data allows us to monitor longer term trends in the data. We use a rolling period so that we can see any improvement or deterioration being made in those trends.
Additionally the TAA does not perform the inspections consistently (in number and by Supplier/MOA) over the audit year, so a single month or quarter will not give an appropriate view of performance.
We extract the numbers of non-compliances per Supplier and per MOA that fall into the following categories:
Category 1 NCs are deemed to be impacting the quality of data entering Settlement e.g. 1.01.
Category 2 NCs are deemed to have the potential to impact the quality of data entering Settlement e.g. 2.01.
Non-compliance Ref raised by the TAA | Description |
1.01 | Inaccuracy of Standing Data (Key MTD fields) held by Data Collector Outstation serial number Meter ID (serial number) Outstation number of channels Measurement Quantity ID Pulse multiplier Channel configuration Outstation multiplier / Outstation channel multiplier Complex Site Supplementary Information Form (SVA only) |
2.01 | Inaccuracy of Standing Data held by Meter Operator Outstation serial number Meter ID (serial number) Outstation number of channels Measurement Quantity Id Meter Register Multiplier Pulse multiplier Channel configuration Outstation multiplier / Outstation channel multiplier Measurement Transformer Ratios Complex Site Supplementary Information Form (SVA only) |
2.02 | Inaccuracy of Standing Data (non-Key MTD fields) held by Data Collector Other non-Key fields (e.g. Measurement Transformer Ratios) Meter Register Multiplier |
2.03 | Non-provision of Standing Data Meter Technical Details not provided – Meter Operator Agent and Data Collector Complex Site Supplementary Information Form not provided (SVA only) |
Why? There are many non-compliance types that are used to classify a range of issues found at the Metering System by the TAA. These particular types of non-compliances are directly related to the MTDs being missing or inaccurate and correlate best with SR0028.
We count the number of visits made by the TAA that have resulted in one or more of the categories of non-compliance described above.
MPID | Number of visits? | Cat 1.01 | Cat 2.01 | Cat 2.02 | Cat 2.03 | No of non-compliant visits? |
ABCD | 1 | 2 | | 1 | | 1 |
EFGH | 1 | | 3 | 2 | | 1 |
IJKL | 1 | | | | | 0 |
For example - ABCD had 1 visit resulting in 3 non-compliances. For the Settlement Risk 28 purposes, this is classed as 1 non-compliant visit.
We only count the number of generated non-compliances in the particular 12 month period. This means that even when a non-compliance has been resolved it will remain in the data set used to measure Settlement Risk 28.
Why? The risk refers to changes being made to the MTDs by the HHMOA and the HHDC not being aware of those changes. We are measuring how often this happens using the data we have available through the TAM process. For this Settlement Risk we are only interested in where these problems have been found in the first instance, because the TAA inspects only a small sample of the total HH population. However all parties should ensure that they take all action necessary to resolve any non-compliances identified by the TAA.
We compare non-compliant visits to the total number of TAA visits made per Supplier ID and per MOA ID.
This is a count of all visits made by the TAA for the corresponding Market Participant IDs – excluding any cancelled visits and including any no access visits.
e.g. The TAA planned to make 100 visits for MPID ABCD in 12 month period. Ten of those were cancelled. Only 90 visits were actually made.
5 of those 90 visits resulted in no access to the metering system. No access inspection visits have been included in the Settlement Risk 28 report.
The total number of visits made for MPID ABCD is 90.
If there were 30 non-compliant visits (after calculating as detailed in step 3), the percentage of non-compliance is 33%.
Why? We count this so that we can calculate a percentage per MPID. We use the data to compare and rank performance.
The number of visits that the TAA completes for any particular MPID is dependent upon the volume of appointments per MPID in each Grid Supply Point Group (GSPG). Therefore the number of visits is proportional to the number of appointments a MPID has in any GSPG.
In order to fairly compare the performance of each Supplier and MOA we calculate the number of non-compliant visits / the number of visits made, to arrive at a percentage of visits that are non-compliant because the MTDs are inaccurate or missing.
5. How do we report the performance of parties against SR28?
On a monthly basis, a report is given to the PAB on the performance of Suppliers and HHMOAs against SR28. It shows the proportion of non-compliant visits found by the TAA. Suppliers and HHMOAs are given a RED, AMBER or GREEN BUSRR based on criteria agreed by the PAB (see the table below).
6. How can I improve my SR28 performance?
In order to improve your performance against SR28, you will need to do some analysis on the types of non-compliance and perform root cause analysis. We recognise that you may be driven to focus only on the non-compliances that go into the SR28 report. However, both Suppliers and MOAs have a responsibility to ensure that ALL Metering Systems and associated data are accurate and complete.
If we are seeing these types of non-compliance arising in the sample inspected by the TAA (remember that it is a proportional sample of the Metering System population), then we could extrapolate that these non-compliances are happening throughout your portfolios. In section eight you can see an overview of the types of common non-compliances found.
Root cause analysis is based on minimising reoccurrence and can be used to continuously improve processes.
7. Managing Supplier and Supplier Agent performance
We will contact Suppliers and MOAs where the SR28 BUSRR indicates performance issues and may request action plans under the Error and Failure Resolution process (BSCP538) (we may also use additional drill down information to support the SR28 measures). We hope that this guidance notes assists ALL Suppliers and MOAs to address root causes of non-compliances found by the TAA
8. What common issues do we typically see?
In this section we’ve provided an overview of the common issues we have seen contribute to non-compliances identified by the TAA. We’ve also provided some ideas per party type (Supplier, MOA and DC) that you can use to work towards an holistic approach in tackling non-compliances, by tackling the root causes of the problem.
Common Issue | Why? |
Poor commissioning processes and communication between Licensed Distribution System Operators (LDSOs) and MOAs. | We have found that MOAs can sometimes struggle to commission the Metering System. We know that there are communication problems between LDSOs and MOAs and we’re looking at ways to resolve this. We also know that there are issues in retaining and storing the commissioning records. There is a lack of understanding of the importance of commissioning and the impact on yourselves or other parties if it doesn’t happen or if it is in accurate or incomplete. The BSC Panel and the PAB have asked Elexon to focus on addressing these issues and we will be looking at ways to both incentivise LDSOs and MOAs to communicate to ensure that a full commissioning test takes place and those records are retained and passed on accordingly. |
Breaks in the process where human / manual intervention is introduced. | Through various routes (TAPAP, BSC Audit, TAMEG) we have found that where MOAs have electronic data transfer between the Metering System on site and the back office data processing systems and the Data Transfer Network (DTN), there was less opportunity for non-compliance. |
A lack of proactive exception management for Supplier Agent appointments and fault investigation requests. | Likely this occurs because there is a lack of understanding of how to improve the processes and also what impact the levels of reactive and manual exception management can have both procedurally and commercially . Including proactive management of the appointments and associated data flows (D0155 and D0148) will benefit you by reducing the cost of poor quality in data that you provide to others and that is provided to you. This failure will also impact on other Settlement Risk performance. |
DCs are not always raising potential faults with the MOA, for it to investigate. | We have seen DCs ‘fix’ aspects of the MTDs to help them obtain the data from the Metering Systems. The PAB is keen that Elexon work with DCs to educate them and make them aware that any problems with MTDs MUST be highlighted to the MOA, using the correct procedures, such as the investigate inconsistencies procedure in BSCP514 and BSCP502. Failure to comply in this manner is likely to be highlighted by the BSC Auditor and managed accordingly. |
Ineffective training of staff and supporting working instructions & procedures in place. | In every TAPAP check that we have performed since 2009 we have highlighted that there is a decreasing pool of expertise in both the field and in the back offices to support the creation, maintenance and update to MTDs. In all of those TAPAP checks we have seen a varying degree of documented procedures and a varying level of skill required to do the roles. It is important that you ensure that you recruit the correct skill level to understand the role and the work. In addition you also need to ensure that you educate and train your staff to a level that helps them understand where they sit in the greater scheme of things, providing them with an holistic perspective of the work that they input into. |
Poor delivery and accuracy of information from Supplier. | Where effective exception management takes place by the MOA and DC, this is less of an issue. However, Suppliers should ensure that they are providing their Supplier Agents with accurate data by validating and checking it before issuing it. |
Poor delivery and accuracy of data from other third parties – MOA and LDSO. | MOAs should have effective communication channels in place with the LDSOs and vice versa. LDSOs need to ensure that they are validating the data that they provide to the MOA, ensuring that it is accurate and that all relevant data for any particular Metering System is passed to the MOA in an effective and efficient manner. |
Intellectual Property Rights, Copyright and Disclaimer The copyright and other intellectual property rights in this document are vested in Elexon or appear with the consent of the copyright owner. These materials are made available for you for the purposes of your participation in the electricity industry. If you have an interest in the electricity industry, you may view, download, copy, distribute, modify, transmit, publish, sell or create derivative works (in whatever format) from this document or in other cases use for personal academic or other non-commercial purposes. All copyright and other proprietary notices contained in the document must be retained on any copy you make. All other rights of the copyright owner not expressly dealt with above are reserved. No representation, warranty or guarantee is made that the information in this document is accurate or complete. While care is taken in the collection and provision of this information, Elexon Limited shall not be liable for any errors, omissions, misstatements or mistakes in any information or damages resulting from the use of this information or action taken in reliance on it. |
