For utilities to truly transform, they will need to harness the untapped power of dark data

Or why utilities struggle with undocumented assets.

The energy sector is undergoing a massive transformation with multiple forces at bay that are driving large replacement, renovation, and digitalization projects. At the same time utilities are trying to understand what the future will look like 30 years from now and wondering what technology to invest in to meet the rising demands in electricity and offer the flexibility to remain agile when demand fluctuates because of the rise of distributed electricity.

This transformation will take years and an estimated 5 Trillion USD to modernize our aging systems and be ready for the future. With changing demands and large investments on the horizon, utilities are facing the challenge of continuing to deliver the same earnings to their investors, investing in the future while also keeping documentation updated on all of their existing legacy technology systems.

And the kicker is that much of that legacy information is locked in with scanned images, mylar or film.

What is Dark Data?

Over the course of 40 years, the utility has amassed hundreds of thousands of pieces of information and while more and more data has become available in digital formats over the last 10 to 15 years, this is not a guarantee that this data can be used in an efficient manner. I will use the example of the recent renovation of a coal facility. All the information from the vendor was provided in digital format and the impact was dependent on how it was provided. PDF documents were delivered as multiple drawings, specifications, maintenance information ࢀ assembled in a single PDF and the information was not easy to use from document control, regulatory, operations and maintenance perspectives.

Additionally, the vendor not only assembled the information in a single PDF but also scanned it in order to scan the professional engineering stamp and legally certify the quality of their work. This rendered the PDF nearly useless, causing teams to spend hours shifting from page to page to extract needed information.

While organizations are well equipped to store and collect data, up to 80% of this data cannot be readily used as actionable information. The definition for dark data is generally understood as data that is collected but those who need it most cannot efficiently use it, find it, or are unaware of that it even exists.

Why is Dark Data a Problem?

Facility operators are bound to numerous regulations that require documentation to be in place; up-to-date and complete. For example, OSHA 1910 (Occupational Safety & Health Administration) requires certain documentation such as P&ID (Process and Instrumentation Diagrams) to be up to date. Failure to comply can result in massive fines for the operator.

Referring again to the coal facility renovation and the recently delivered project, the drawings are locked in a PDF, hard to access, and it is difficult to ensure that all drawings during the first change after the renovations were properly updated. If an organization does not have a strong data management framework, there is a good chance that several copies, all of them updated to different revisions, are floating around in the enterprise in various hardcopy and digital formats.

Part of the problem is that the information is not siloed to a single point. Aside from documentation requirements, an operator also has additional systems, such as an asset management solution, to track and maintain with some version of a plant control system. An often-heard challenge is that the information between the documentation, the asset management system, and the plant control system is not aligned. It is easy to imagine what the consequences are. Engineers will spend hours to find a complete picture of a problem – without ever really knowing if they have the latest information.

The typical solution is to perform a physical walk down of the facility and inspect the information available against the physical situation. After that, drawings need to be updated, redistributed, PHA (Process Hazard Analysis) reviews conducted by engineers, and another maintenance change performed. The downward spiral of information chaos starts almost immediately after a facility is renovated.

Is Digital Twin the Answer?

The new buzzword in technology is Digital Twin. In the true sense of the description, a Digital Twin is a digital representation of the physical facility to its minute detail. It promises to completely disrupt how we design, construct and operate our facilities. At the same time, the concept has established advantages in specific situations. In the case of an aging facility with massive amounts of history and often poorly aligned data, the law of diminishing returns will prove to be a reality very quickly.

Operators need a strategy to work towards the use of Digital Twin technologies and determine how their situation applies. Although Digital Twin is the more famous one, it is part of a larger set of technologies referred to as Industry 4.0. The proliferation of new connected technologies and equipment, driven by the backbone of the internet, enables more connected supply chains and service models.

To utilize these technologies, there is a common thread among the various maturity models that speak to industry 4.0 and Digital Twin. Ideally, all the systems and applications along the MES (Manufacturing Execution System) need to be aligned and integrated.

For example, if a pump with a certain equipment number is producing an error, for this information to automatically be visible and actionable from the asset management system, the data needs to be integrated. If management wants to understand integrated data across the operational cost for pumps from a specific vendor, the data needs to be integrated into the ERP system. The optimal alignment of data and documents across the engineering enterprise is also referred to as BDA (Best Documented Asset). This strategy prepares an organization for utilization of Industry 4.0 technologies, unlocks the existing information, and equally uncovers the needed information.

Unlocking Dark Data’s Value

While the connotation of dark data sounds mysterious and negative, organizations can use these hidden gems of data. New solutions are in the marketplace that will automatically scan, link, index, and extract all valuable information for an engineering environment. Using machine learning technologies, new software will sift through hundreds of thousands of files and can performed on the full history of all data, regardless of quality and format.

One of our utility customers with a single large project was able to reduce the project time frame nearly 20%, simply because the information was more readily available across a large set of historical information. The key: these solutions can support standards such as ISO 15926, that describe the required documentation for each piece of equipment and align information exchange between parties. The standards combined with these innovations allow an organization to scan for critical information that is missing.

Achieving more value from existing information is one of the many options an organization has. It is best to look at the bigger picture and make sure the different data touchpoints are integrated to get the most value from an implementation.

a dtech show sign

Want to walk the DTECH show floor without taking steps?

Factor This content director Paul Gerke takes you on a walkthrough of the show floor at DISTRIBUTECH 2025 in Dallas, Texas.

Getting ‘forever chemicals’ out of the chips race – This Week in Cleantech

This Week in Cleantech is a podcast covering impactful stories in clean energy and climate in 15 minutes or less, featuring John Engel and Paul…