PSEG Long Island's Isaias experience shows how storms can stress customer communication systems
image credit: Photo 192991758 © David Wood | Dreamstime.com
- Sep 25, 2020 9:11 pm GMTSep 25, 2020 4:33 pm GMT
- 744 views
Electric utilities are responding to climate change by doing everything from reducing their carbon dioxide emissions to making their physical infrastructures better able to deal with more severe and frequent storms, heatwaves and fires.
Tropical storm Isaias showed they also need to make sure their systems for communicating accurate outage information to their customers can handle heavy loads.
At hearings in New York and Connecticut last month, utilities came under heavy fire from lawmakers and others not just for prolonged outages after Isaias rolled through their territories but for failing to provide their customers with accurate — and at times any — information about the outages.
The New York Post reported that New York Public Service Commission Chairman John Rhodes told Empire State lawmakers that “ … some utilities provided inaccurate information, making it challenging for families and businesses to plan. This was more than inconvenience. It was completely unacceptable and a public health and safety concern.”
One of those utilities was PSEG Long Island, the PSEG subsidiary that operates the transmission and distribution infrastructure in the Rockaways sections of Queens and most of Long Island outside New York City on behalf of the Long Island Power Authority (LIPA), which owns the infrastructure.
Isaias hit Long Island with sustained winds of 45 to 50 miles per hour and gusts of 70 mph, causing more than 645,000 of PSEG Long Island’s customers to lose power at some point, according to a report by a task force assembled by LIPA to look at the utility’s response to the storm.
That, however, was just where PSEG Long Island's troubles started. According to the report, the utility first gave its customers an estimated time to restoration (ETR) of two days, later updated that to four days and wound up taking eight days to restore power to all of them. On top of that, the report said, many PSEG Long Island customers couldn’t reach the utility by phone to report outages or receive status updates for three days — the day Isaias hit and the two days after.
PSEG Long Island’s problems in both estimating the time it would need to restore power and communicating with its customers stemmed from its outage management system being overloaded. The report said the OMS “experienced multiple issues with the high volume of data [it was receiving], rendering it effectively non-functional at times and negatively impacting all communications channels and field management activities.” The utility’s communications problems were exacerbated by a weird bottleneck in the method it used to route calls during times of high call volume, according to the report.
The report said that PSEG Long Island’s OMS gets the majority of its outage reports from the utility’s supervisory control and data acquisition (SCADA) systems. It gets most of the rest from the utility's customers as PSEG Long Island's advanced metering infrastructure is limited.
When Isaias struck, the report said, the OMS began getting a lot of outage notifications in a short amount of time, causing its performance to degrade. As that happened, PSEG Long Island began working with the system’s vendor, CGI, to try to improve the system’s performance, but issues with it persisted throughout the duration of the utility’s restoration efforts, according to the report.
The task force is still investigating the cause of the OMS’ problems, but it said in the report that a major one was likely the system’s inability to process the outage reports it was getting from PSEG Long Island’s SCADA system and customers fast enough to match the rate at which they were coming in. That, the report said, was a primary cause of the utility being unable to provide accurate ETRs. It also led to PSGE Long Island being unable to process some text messages it was receiving from customers and to keep its outage map current.
The problems with PSEG Long Island’s OMS were especially surprising because the utility had recently updated it. After the storm, PSEG Long Island went back to a previous version of the system and tested that by making it try to handle the volume of data it would have to process as a result of a storm such as Isaias. The Isaias Task Force said the test found that even when the OMS was running the earlier version of the software its performance degraded significantly once the volume of notifications it was receiving from PSEG Long Island’s SCADA system exceeded a certain threshold.
Another problem with the OMS, according to the task force, was that it was unable to enter a “comprehensive ‘OMS Storm Mode,’” which allows users to control the frequency, accuracy and content of ETR messages across all channels. “Implementation of a comprehensive ‘OMS Storm Mode’ is the single most important technology change that would have made a notable difference in the customer’s experience and their trust in the information provided by PSEG Long Island,” the task force's report said.
Like PSEG Long Island’s OMS, the utility’s infrastructure for handling incoming calls also was overwhelmed, mostly due to an odd bottleneck.
PSEG Long Island has a call center on Long Island, according to the report. Calls to its outage line go there where they’re answered by a person or an interactive voice response system. During high volume periods, the call center’s managers can have a quarter, half or all of the incoming calls to the center redirected to a high-volume call answering (HVCA) service provided by Intrado. Redirected calls go over Verizon’s network to an AT&T-managed number and then to the service.
The task force found that the Verizon lines that move redirected calls from the call center to the AT&T number didn’t have enough capacity to handle the volume of calls they were asked to transport on the day Isaias hit and the following day. As a result, the task force estimated, roughly three quarters of the calls to PSEG’s outage line didn’t go through on the day of the storm. When the call center’s managers realized there was a problem, they notified Verizon, which took 20 hours to diagnose and solve it by adding lines between the call center and the AT&T-managed number. Over that time, the task force said, more than 400,000 calls to PSEG Long Island were given busy signals or dropped.
Late last month, PSEG Long Island, Verizon and Intrado implemented a change that eliminates AT&T from the call-routing process. Next month, they plan to complete moving PSEG Long Island's outage line from Verizon New York, the local exchange carrier on Long Island, to a sister company, Verizon Business Network.