Addressing the Big Data Concern in the Utilities Sector

By Brian Bohan, Vitria Technology, and Bret Farrar and Mark Luigs, Sendero Business Services

It is a time of great change and transition for the electric utilities industry—an evolving regulatory environment, a push toward renewable energy sources, the advent of smart meter and grid technologies, and the potential of competition drive uncertainty.

The most significant opportunity and risk, if not properly addressed, is presented by the coming torrent of new data and events resulting from efforts to modernize the grid and the entire electric utility operation.

One of the largest drivers of data volumes for a utility is the meter reads from all of the smart meters in its territory. Prior to the implementation of a smart meter, utilities would conduct one meter read a month per meter. With the new smart meters that capture usage data in 15-minute intervals, utilities will collect more than 3,000 meter readings a month for each meter. This translates to terabytes (TB) of data being collected and stored at the customer level. We can expect 300 TB per year of meter data by 2012, according to the FPL Group. The diagram from the Electric Power Research Institute via Pike Research illustrates this exponential growth in data and the triggers driving key inflection points.

As the diagram illustrates, a broad array of drivers and sources for this data exists. Utilities will be forced to grapple with many types of data in format and content.

IMPLICAtions, Opportunities

This onslaught of new data also represents opportunity. The utilities industry is in a great position to get in front of this trend by applying the right approaches to capturing, marshalling, analyzing, understanding and acting quickly and intelligently on the unprecedented data amounts. Although this might seem like a new problem, utilities do not have to invent new approaches. Other industries, especially communications service providers (CSPs) and mobile operators, have been addressing their own big data challenge for more than a decade. The fast-moving smart phone market has driven enormous change, forcing operators to grapple with a data explosion. This has accelerated the need to bridge the traditional boundaries between the network operations center (NOC) and business operations to interoperate seamlessly with a large, diverse set of partners and content providers to provide the level of service customers expect.

Utilities also have a large portfolio of existing legacy operational applications and systems that will continue to perform key functions. As utilities face competition and attempt to harness the data to address challenges, they will be tempted to integrate legacy systems as needed or quickly deploy one-off solutions. This approach might seem like the path of least resistance, but following it will lead to unnecessary complexity where all the new smart grid data will end up locked in silos, making it nearly impossible to gain an end-to-end view of the operations.

Utilities that understand this new information does not align with existing boundaries and is most valuable when correlated with broad data from all corners of the business can gain new visibility, insight and management over core operations, processes, customer interactions and experience, energy delivery patterns and security.

Operational Intelligence Solution

It is one thing to understand this conceptually and another to achieve it. It need not be as arduous as it might seem, however. There is no need to rewire, re-factor or merge existing systems. Working with utilities and CSPs, we have seen a trend toward overlaying a noninvasive, real-time analytics layer on top of existing network, grid and operational systems to achieve the visibility, insight and management described. We refer to this solution as operational intelligence (OI).

An OI solution can consume data and events from a broad array of sources, both stored and streaming; perform continuous analytics against that data, looking for anomalies, patterns and trends that might indicate an opportunity or a problem; kick off an action based on rules and policy; and provide robust, business-centric visualization through real-time dashboards to key stakeholders.

Utilities can leverage solutions such as OI to view this data from micro and macro levels. At a microlevel, utilities will be able to analyze usage patterns at the meter level and provide this usage information back to consumers with the intent of achieving demand-side energy reduction. At a macrolevel, utilities will be able to analyze the energy usage patterns of neighborhoods, cities and counties to facilitate infrastructure capacity planning and load demand for their service territories.

Utilities also can provide the information they glean from their smart grid infrastructures back to customers to enhance the overall customer experience. For instance, by integrating advanced metering and outage management systems, utilities will be able to address more proactively outages that occur within their territories and provide more accurate outage information to customers. Leveraging the data at their disposal, utilities will be able to address basic customer concerns by providing interactive outage maps to customers through Web and mobile platforms.

Implied in the scenarios is the need to include key customer processes and interaction points into the analysis and action. Many of these core processes lack end-to-end visibility and control, leading to high rates of exceptions, poor efficiencies and negative impact on the customer experience. This is especially true when processes span numerous systems and interface with partners. Add to these the need to further enrich move in, move out, and disconnect-reconnect processes with usage data originating on the grid side, and the challenge is greater. Again, there is a need to be able to layer OI on top of these processes and systems to track and trace key transactions to provide the necessary visibility, insight and action to catch and remediate issues before they impact customers.

Operational Intelligence Benefits

The benefits of addressing the new landscape and unprecedented data volumes with an initiative such as OI are manifold. Utilities will be able to attach to the information-spigot once and address their operational, energy delivery, customer care and security challenges through noninvasive, real-time data analysis. Utilities will be able to identify trends, anomalies and patterns within given organizational boundaries to identify opportunities and threats. Utilities will be able to achieve these objectives not by rewiring their entire existing infrastructure, but by creating a fabric on top of that infrastructure to provide a common operating picture.

Utilities do not have a choice in addressing the coming deluge of data, but they can choose how they do it. They can try to compartmentalize the data based on type or place of origin in the organization or dump it all in massive warehouses, delaying the decision on how to address it. These all would be mistakes. Instead, utilities could leverage existing infrastructures by tapping into the new event types, looking for trends and patterns apparent only when looking across boundaries and valuable only when tracked and acted upon in near real time. Utilities can skip the false starts of one-off solutions and point-to-point integrations and fast forward to where CSPs are realizing the value of correlating, analyzing and acting upon data flowing from all parts of the organization. Utilities can achieve this by deploying operational intelligence-type solutions to provide visibility, insight and action to deal with these data volumes.

Brian Bohan is vice president of worldwide consulting at Vitria Technology Inc. He has more than15 years of experience with the technical and leadership elements of operational intelligence in the electric utility industry.

Bret Farrar is founding partner of Sendero Business Services. He has 24 years of experience working with business and information technology organizations to improve their effectiveness through their strategies, people, processes and technologies.

Mark Luigs is a manager for Sendero Business Services. He has more than10 years of experience in the customer contact center space working for companies in the utilities, telecommunications and high-tech industries.