Contributed by Vikhyat Chaudhry, Co-Founder, COO and CTO, Buzz Solutions
US utility decision-makers have embraced digitization to modernize and decarbonize the grid. As part of this journey, they have transitioned from analog to smart meters, enabled by IoT sensors that provide a constant stream of data on asset condition and performance for analysis. Smart meter analytics help utilities understand electrical demand patterns, bill individual customers, and more.
However, data modernization processes haven’t kept pace, meaning that meters collect data that isn’t fully exploited. Utility data is often collected in silos to support individual functions, preventing organizations from creating a holistic picture of grid performance and developing granular insights into meter data that can help predict future demand. In addition, utilities maintain legacy systems for years to preserve data for compliance processes and avoid the challenges of passing through unpopular rate increases to support data modernization and business process transformation.
That’s a missed opportunity. Transmission and distribution utilities that leverage advanced analytics in their asset management strategies can unlock 10% to 20% in operational savings while improving the reliability and performance of their networks.
A data-driven industry that’s still struggling with fundamental data challenges
Utilities collect a wide array of data, using modern and legacy third-party smart meter, customer information systems (CIS), geographic information systems (GIS), supervisory control and data acquisition (SCADA), load, temperature, and asset health monitors. These systems, from manufacturers such as Oracle, SAP, ESRI, Siemens, ABB, IBM Maximo, Schneider Electric, Accuweather, and others, may provide basic interoperability, but store data in different databases and formats, making it difficult to combine and analyze data across platforms. They may require custom development or middleware to integrate with other systems: increasing development and maintenance costs and locking utilities into specific vendor ecosystems. In addition, smart meters often use proprietary communication networks, further increasing vendor lock-in.
As a result, many decision-makers lack a holistic view of grid performance to determine power quality and reliability, asset utilization, line losses, maintenance requirements, and customer use and billing. Without centralized, near-real-time access to streaming data, utilities also can’t fully capitalize on the power of AI to transform their businesses, enabling predictive and ultimately prescriptive analytics that automate critical processes, increasing responsiveness while reducing risk and cost.
Utilities can modernize data by deploying cloud infrastructure, data warehouses, data lakes, and a data mesh to connect data centers and information sources. Cloud infrastructure, from providers such as Amazon Web Services, Microsoft Azure, or Google Cloud Platform, provides the computing power, storage, networking, and services companies need to create a solid foundation for processing large data sets and analytics workloads. With cloud capacity, utilities can scale up and down as needed across service areas, which is essential in an industry experiencing constant peaks and valleys in demand. With these capabilities in place, utilities can use more sensors, connect more devices, and collect even more data for analytics.
Data warehouses offer repositories for storing and analyzing structured historical business data using relational databases. They provide consistent data, fast query performance, and business analytics, using popular dashboards to generate insights. Data lakes provide a scalable resource for integrating raw, unprocessed structured, semi-structured, and unstructured data from different sources and in different formats. They enable both real-time streaming and batch processing, empowering utilities to create advanced analytics based on more of their data. Finally, a data mesh is a tool to distribute data ownership across business functions or teams, each of which must maintain the quality of its data, ensure proper governance, and enable sharing using access controls and APIs.
Utilities can also adopt medallion architectures to enable the creation of data sets of varying quality—including gold, silver, and bronze—for analytics and other purposes. By aligning data quality with use cases, utilities can ensure teams have the right data at the right quality levels for key processes, improving trust in data, enabling them to make analytics-based decisions, and reducing operational costs.
Why utilities are centralizing their data
Data warehouses have advanced significantly over the past few decades. They are increasingly cloud-based and use APIs to communicate with internal systems and third-party data sources, which can create more robust analytics and insights. Previously, third-party data was harder for utilities to integrate.
As a result, utility data teams can build AI and machine learning models that leverage advanced analytics to forecast demand, forecast machine failures to enable predictive maintenance, and prioritize and address risks and service restoration. This advanced technology also enables utilities to mix meter types, stagger deployments, and increase the frequency of data delivery, increasing ROI on advanced meter infrastructure, which can cost hundreds of millions of dollars to deploy.
Breaking down barriers to centralizing data
So, what stops utilities from making faster progress on modernizing their data? Many face the following challenges:
- Maintaining strong leadership: Data modernization initiatives are typically championed by multiple members of the C-suite, including chief information and technology officers (CIO/CTO) and the heads of data and operational technology. If utilities don’t have these leaders, they may need to recruit them. These leaders also need continued executive sponsorship, a skilled team, a sufficient budget, and hyperscale and data partners to ensure successful data migration and modernization initiatives. For most, mastering data is a stepping stone to transforming their businesses with AI.
- Competing for top talent: Utilities compete with data and AI companies for data engineers and scientists. The pressure is especially acute on smaller utilities not located in major urban areas where talent typically resides and already lags on their cloud journeys. Larger utilities typically have won the investment for cloud migration, are making progress against their roadmaps, and are supported by strong data, IT, and cybersecurity teams. As a result, it’s somewhat easier for them to hire for key roles.
- Reducing the attack surface: Many utility leaders are concerned about data breaches, as the energy sector is a top five industry for hacking and ransomware. Only one in five utilities in the US is ranked as “strong” or “very strong” in cybersecurity, a concern as many leaders pursue a long-term strategy of integrating IT and operational technology. As utility teams modernize data, they will also want to adopt cloud-native security tools to detect and remediate risks in real-time.
Modernize data to modernize the grid
Many utilities are tapping combined federal and state grants and financial incentives to modernize their grids, putting these capital-intensive investments within reach. For example, President Biden’s Inflation Reduction Act and Bipartisan Infrastructure Law created extensive new resources. The Department of Energy provides $10.5 billion in competitive grant funding and $250 billion in low-interest financing for grid projects. In comparison, the Department of Agriculture offers $9.7 billion in low-interest loans and grants for projects in rural areas. As a result, forward-thinking utilities are moving ahead with grid modernization projects that are already overdue due to aging infrastructure and power issues.
Migrating and modernizing data using streaming data warehouses, lakes, meshes, and medallion architectures can help utility teams identify opportunities to improve power quality and availability, enhance operational resilience and efficiency, and reduce costs. Doing so can enable decision-makers to prove ROI on these investments. Then, when it’s time to apply for new grants or loans or request rate increases, leaders are more likely to win the funding they need to drive data and AI further into their businesses. With highly reliable utility operations powered by data and AI, everyone wins.
