The mission of this group is to bring together utility professionals in the power industry who are in the thick of the digital utility transformation. 


PG&E Customer Mis-assignment Correction Initiative

image credit: PG&E

This item is part of the Special Issue - 2020-12 - Data Analytics & Intelligence, click here for more

As is the case in many utilities, the association between customer meters and transformers is one area where there are known inaccuracies. Public Safety Power Shutoffs (PSPS) have highlighted the need to correct this association so that the correct set of customers can be notified in advance of a safety shutoff. Manual inspection of all of PG&E’s 5.4 million meters is infeasible, so an alternative approach was needed.  A project to automatically identify misassignments, recommend corrections, and prioritize workflows to review and correct errors has been implemented.

This project has been informed by the initial work that was done as part of an Electric Program Investment Charge project in which analysts from PG&E and several other vendors evaluated the potential for a fully automated solution[1]. Unfortunately, this evaluation determined that though the algorithms were able to find and accurately correct some mis-assignments, the number of incorrect outcomes outweighed the correct ones. Without a fully automated solution, a human in the loop solution was implemented to enable the issue to be mitigated prior to the availability of a sufficiently accurate algorithm.

Your access to Member Features is limited.

Please or apply for membership to continue reading this post.

 Flag Misassignments

The first stage of the process is to attempt to flag anomalies. This is a modular process which allows the creation of flags from different sources. The primary source of flags are inconsistencies within the voltage for service points on the same transformer. However, there are a variety of other mechanisms that can raise a flag, including manual reporting of customer complaints, service points with large distances to their transformer, or voltage profiles that are very different from the others on the transformer. In addition, other smart meter, electrical, geospatial and asset characteristic information can be used to drive investigations.

Recommend Correction

The process to recommend corrections is a result of an advanced data processing and clustering algorithm. Interval readings of smart meter voltage data along with geospatial features are clustered to create groups of service points which are similar to each other composed of individual or multiple transformers.  An example visualization of this process is shown in Figure 1. This visualization uses the UMAP[2] dimensional reduction to visualize the multi-dimensional features used in the analysis. The chart on the left shows different colors for individual transformers, and the one on the right shows the cluster groups. Though there are fewer groups than there are transformers, they are effective for identifying significant anomalies. The grouping of service points in the clusters is also used to drive the process to identify recommended corrections for misassignments.

Figure 1: UMAP projection of voltage data comparing actual labels with cluster groups

PG&E has more than 5 million smart meters, reporting over 300 million voltage readings per day, so creating a process which was scalable was required. The clustering algorithm was applied at the circuit level, however service points and transformers in proximity to the circuits being processed were included in the group to enable reassignment.


Once the set of flagged transformers is generated, it is important to prioritize the work.  Considerations such as location, customer characteristics and the potential for mis-notification during outages are considered to enable focused reduction of customer notification impacts.

Review & Correct

The prioritized misassignments are manually reviewed by the PG&E mapping team, and where appropriate reassignments are processed and corrected. A user interface has been developed to support investigation and review of the proposed correction and to override the flag or suggestion.  The outcome of the investigations is captured, and can be used to support future algorithm improvement. Since deployment, over 2,600 flags have been evaluated, resulting in over 1,500 corrections.  PG&E plans to continue to refine this algorithm and correct these data quality issues as we continue to strive towards safe, reliable and affordable grid.


[2] [1].McInnes, L., Healy, J., and Melville, J., “UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction”,,  2020.

Devon Yates's picture
Thank Devon for the Post!
Energy Central contributors share their experience and insights for the benefit of other Members (like you). Please show them your appreciation by leaving a comment, 'liking' this post, or following this Member.
More posts from this member


Spell checking: Press the CTRL or COMMAND key then click on the underlined misspelled word.
Matt Chester's picture
Matt Chester on Dec 2, 2020

PG&E has more than 5 million smart meters, reporting over 300 million voltage readings per day, so creating a process which was scalable was required

That's enough to make any data scientist's head spin. How do you go about assuring quality and accuracy of such a vast swath of data? 

Devon Yates's picture
Devon Yates on Dec 3, 2020

For large datasets like these spark is an important piece of technology to enable distributed computing.  In regards to data quality, it is a challenging problem, but when smart meter or other sensor data is available it can be an important source of truth to try to distinguish the sources of data quality inconsistencies, and we try to leverage that as much as possible.

Get Published - Build a Following

The Energy Central Power Industry Network is based on one core idea - power industry professionals helping each other and advancing the industry by sharing and learning from each other.

If you have an experience or insight to share or have learned something from a conference or seminar, your peers and colleagues on Energy Central want to hear about it. It's also easy to share a link to an article you've liked or an industry resource that you think would be helpful.

                 Learn more about posting on Energy Central »