Implementing the Future Today In Naperville, Illinois

By Daniel Gacek, City of Naperville, and Howard Self, Siemens

Taking the initiative to move future technology into the present, the city of Naperville in 2005 began implementing the first IEC 61850 substation while many U.S. utilities were still reviewing and familiarizing themselves with the new standard.

Naperville, Ill., is located a half-hour west of Chicago in the center of the Illinois Technology and Research Corridor. The Department of Public Utilities-Electric purchases wholesale power from Commonwealth Edison and supplies electricity to approximately 56,000 customers through a system that consists of 16 substations and 1,000 miles of distribution lines. To meet rapid growth and future demands within Naperville, the city embarked on a plan to upgrade two existing substations, Chicago 50 and Modaff 70 and to construct one new substation, Jefferson 85.

In February 2005, the city completed its upgrade specifications and new automation design. The specification they had standardized on and implemented in their last eight substations included Siemens 7SJ63 multifunction feeder protective relays connected to a SICAM SAS automation system communicating via Profibus FMS at 1.5 Mb/s. The existing design, which dated back to the year 2000, provided the backbone for the city’s distribution automation system and was viewed as state of the art and extremely robust. However, the Profibus FMS was somewhat proprietary and the engineering tools for this, while effective, were cumbersome.

While attending DistribuTECH 2005 in San Diego, attendees from Naperville were introduced to IEC 61850. One of the DistribuTECH exhibitors, Siemens, met with Naperville city engineers and in less than an hour produced an actual working system that would be comparable to Naperville’s new substation design. After completing this successful demonstration and reviewing Siemens’ IEC 61850 compliance certificates and project reference list, it was agreed that a new specification and design were needed immediately due to the reduced engineering and installation cost that would result from moving to the new standard.

Chicago 50 Substation

Naperville’s Chicago 50 substation is a 34.5-kV to 12.47-kV substation that was built in the late 1980s and today serves approximately 6,000 customers. Chicago 50 and other Naperville substations included a variety of technologies that made up their SCADA, substation automation, and distribution automation designs. The automation system and the various relays and IEDs formed a hierarchy consisting of clusters of devices communicating with a master device. The master devices would then communicate with each other to share data. A variety of protocols were used including Modbus, DNP 3.0 and Profibus FMS. While often cumbersome, these standards were the best available on the market at the time they were selected.

The Chicago 50 switchgear is housed in a modest control house and is typical of its generation: one multi-function microprocessor relay, one multifunction digital meter, and control switches for the recloser, SCADA and local/remote breaker control. The original digital panel mounted meters communicated with the SCADA RTU via Modbus protocol. The Modbus interface was limited to 12 values per meter, which limited data to only the necessary real-time values; the original microprocessor relays were limited to local interrogation.

The New Design

City engineers modified their existing substation drawings to remove the existing relays and add the Siemens 7SJ63 relays with dual-fiber IEC 61850 interfaces. The dual-fiber design allows for shorter fiber runs and provides an extremely reliable redundant loop. The Chicago 50 substation controls were modified to match their existing substation designs, which also used the 7SJ63 relays. Since standards were maintained, the relay files from the existing substations could be reused and modified with the required IEC settings and feeder specific protection settings. Siemens assisted with modifications since it was the first implementation of this type in the United States. Naperville hired a contractor and made the mechanical and electrical modifications. While Naperville was working to get the relays installed, Siemens was busy building an automation cabinet and designing the substation HMI based on the Chicago 50 substation single line.

Naperville had standardized on a common design with the SICAM SAS in the early 2000s. A similar design would be used for consistency with the new system. The system would consist of a hardened substation controller and a SICAM station unit powered directly by the 125 VDC battery system. The station unit operating system runs Embedded XP with enhanced network security. SICAM PAS is the Siemens application software which manages the IEC 61850 client data and operates on the station unit. The station unit is connected via Ethernet RJ-45TP twisted pair to a substation-hardened IEC 61850 managed network switch also powered by the 125 VDC battery. The managed switch has eight 10/100 Mb auto-negotiating RJ-45TP ports and eight 100 Mb 2 km fiber FX ST ports. A highly accurate 125 VDC GPS clock with NTP (network time protocol) is also connected via RJ-45TP to the network via the managed switch. A local PC for the substation HMI, relay configuration, network maintenance, fault recording storage, and an eight-port serial hub are also connected to the network via the managed switch. These two devices are AC powered and supplied by a Wilmore DC-to-AC inverter for high availability. The equipment was all housed in an enclosure with a half-glass door and mouse/keyboard tray for user convenience.

Implementing IEC 61850 in Eight Simple Steps

Cyrus Ashrafi (right) and Dan Gacek (left) stand next to the PAS cabinet at Chicago Substation.Click here to enlarge image

IEC 61850 was designed with a number of goals and features in mind, including:

• Standard communication with TCP/IP based on Ethernet standard;
• Time synchronization using SNTP;
• Communication between bay level IEDs using “GOOSE” (generic object-oriented substation event) messages;
• Services for file transfer, i.e. fault extraction in Comtrade format;
• Defined structure for protection, control, measurement and indication data (Logical Nodes);
• Standard language for describing the substation (SCL), with both IED data (ICD files ) and system data (SCD files); and
• A complete system configuration to link it all together.

By using the IEC 61850 standard, Naperville engineers found that they would be able to quickly configure their devices and interconnect them together. The following steps were used and only took only a day and a half at the Chicago 50 substation.

Step 1. Since this would be an Ethernet network, every connection and device in the substation (relays, PC, station unit, clock, switch, etc.) was assigned an IP address.

Step 2. Protection devices were then configured. Siemens used its DIGSI4 software for configuring relays. The software is used to configure settings, define control functionality, and any other logic required to protect, monitor and control a feeder. It also generates IEC 61850 “ICD” files which contain the data objects required for control and monitoring of the feeder (i.e. watts, VARs, amps and breaker position). Since the standard defined these objects, Naperville engineers did not need to know register addresses or point numbers as they did with the legacy protocols. They only needed to know the names of the devices they wanted to see in their control system. City engineers created a file for each relay. Most of the files were identical, so engineers used “copy and paste” to do most of the work once the standard file was created. Engineers then went back to each file and made minor protection changes as required.

Step 3. This step may vary slightly by manufacturer, but the standard gives good instructions on how to proceed. Engineers imported the “ICD” files into an IEC 61850 station configurator software. The configurator serves multiple functions:

• It allows the user to define the network topology ( i.e. subnets, gateways and IP addresses);
• It is the place where the user assigns “GOOSE” message connections from one IED to another;
• It can create report applications (to allow a client to see a high-speed GOOSE message in real time, a report application must be created);
• It allows a central location for defining the NTP clock server for the system; and
• It generates an SCD file.

The SCD file is one central file that contains all the ICD files from IEDs plus IP address information, all GOOSE message connection, all report applications and other details as defined in the standard. This SCD file is the information server for each device in the network including the client. Once created, this information can be fed back to the IEDs again for GOOSE connections and a variety of data such as time server information. In the Chicago 50 substation, Naperville used it to create the file for the client, assign the IP addresses, and assign the time server. Finally, the files were downloaded to the relays.

While this may seem like a lot of work, the city engineers, together with Siemens, went through these first steps in less than two hours.

Step 4, day one at the substation: In this step, engineers connect a laptop to the network or use the station PC to ping all the devices and see if they are visible nodes. Naperville experienced their first problem at Chicago 50 during this step. Only two relays, the first and last, were visible on the network, so Naperville engineers did what all good engineers do-started rolling wires. This, however, did not help. Next, each relay diagnostic screen was interrogated to see if any had identical IP addresses or no IP address assignments. This was not the problem either. However, with the help of the Siemens instruction manual, engineers discovered that they had to tell the relays whether they were connected in a ring or a line configuration. It was discovered that all the relays were connected in the “line” mode, so the dual-fiber loop network was not effective. Settings were changed on each relay for the “ring” configuration and, at last, they could ping all the relays. This lesson, while a good one, wasted most of day one at the substation.

Naperville also discovered that when they set their modules for “ring” mode that they also enabled the Ethernet switch feature “RSTP” (Rapid Spanning Tree Protocol). RSTP could be a separate technical paper all in itself, but here is a simple explanation of why it is required: When you connect a ring, messages can travel in both directions through the ring. RSTP has features that are both important in a stable ring and in a broken ring. Naperville was primarily concerned about a broken ring. The RSTP helps redirect the traffic for the most efficient path when the ring is broken so old messages don’t keep getting repeated and the most optimal path for message delivery can be established. Even in a 100 Mb network, managing traffic is very important.

Day two at the substation was very productive. Since there was now a fully functional Ethernet network, all serial cables could be discarded. The relays could be connected with the configuration software via Ethernet (100 Mb is so much faster that any serial connection). Also, the advantage of viewing the relays via a web browser to see diagnostics data and network traffic was now available. Using the Siemens IEC Browser Software all real-time data and logical node data (watts, VARs, amps, breaker position, etc.) could now be viewed.

Step 5. Now it was time to configure the IEC 61850 client to collect all this data. At this point, the SCD file created in Step 3 is used again. SICAM PAS is the software which runs Naperville’s IEC 61850 client application. Using the station PC and the Windows XP “remote desktop” feature, Naperville connected to the station unit and PAS application. By using remote desktop, no extra monitor is required and extra password security is available to keep the IEC 61850 client configuration protected. Next, the PAS application is opened and the user simply imported the SCD file previously created. The SCD file builds the entire database of all IEC 61850 devices that Naperville created in the substation. In a matter of minutes, the system was built. The data can be selected or deselected by going to the mapping tab for PAS and selecting or deselecting the data required. The start button was selected, and, in seconds, all devices were on line and reporting data.

Step 6. Next, data is verified. Sicam PAS comes with a “Value Viewer” that allows you to see all real-time data being collected via the client. It displays the data in tabular form showing the timestamp, validity and cause of the messages generated. With the value viewer displaying data, the city engineers were able to quickly compare the values on the individual IEDs with the data being collected by the client.

Step 7. Now it was time to demonstrate how wonderful IEC 61850 and Ethernet are in a substation. Connectivity via one single port without disruption of data is one of the most powerful features of 61850 and devices connected via Ethernet at 100 Mb/s. To illustrate this, Naperville selected one relay and connected to it with the Siemens DIGSI configuration software and viewed the relay settings and configuration data. They also connected to it with a web browser and the IEC browser. Finally, they manually triggered a fault record from the DIGSI software, and the fault was automatically uploaded to the station PC in a matter of seconds. All of this was accomplished without disrupting any data to the client, the browser or the configuration tool.

Step 8. Scheduling conflicts for the city engineers and Siemens would force the SCADA commissioning and HMI commissioning to another date. Engineers departed the substation at lunch time only one and half days after arriving.

City engineers returned to the substation to do the HMI and SCADA commissioning. This was also accomplished in one and a half days. However with SCADA and HMI systems, much work was done to build the SCADA database and HMI database.

The city collects the data from the IEC 61850 Client via two DNP serial channels. Two channels are used to improve data response as the city collects real-time analogs for all phase, min/max data for all phases and all relay target, and protection trip data thus creating a very large database to its ACS SCADA system. IEC 61850 again added advantages in building these databases. Since the IEDs have descriptive names, the database was simply built by choosing the descriptive name as collected in the IEC 61850 client and assigning DNP address and DNP Class information for change reporting. The city has a fiber backbone extending to each substation which provides the highest reliability available. The fiber optic network is connected in a ring format between each substation. An OC-3 Sonet based JungleMUX multiplexer system then securely transmits Ethernet signals utilizing the fiber optic network. This structure also provides for diverse routing and redundant signal paths in the case of damage to any point of the system.

With this secure fiber network, the city engineers will be able to take advantage of the IEC 61850 tools from their main offices. By extending the WAN/LAN, engineers can connect to relays, view substation HMI data, and also have access to the fault records that are being automatically collected and archived on the station PC. Bad weather, traffic jams and poor phone lines will no longer hinder the city from analyzing disturbances or making system changes. The city has already taken advantage of the NTP time server and the WAN network. They are synchronizing all substation computers throughout the system from the Chicago 50 substation time server.

The Automation System and the HMI

The substation automation system serves as a control and monitoring device for the people operating the switchgear. The operator must be able to detect the substation status and carry out switching commands from an HMI, which is connected to the station unit via the TCP/IP station bus. The substation automation system must record and processes all switchgear events. Each event must be accompanied by a real-time time stamp. The origin of this time stamp must be in the device which acquired the event.

The tasks of the substation automation system must be distributed logically according to field devices, station units and process visualisation systems.

All measured and metered values as well as all events must be captured by the field protection and monitoring devices which are connected to the substation automation system either in a radial or in a bus structure.

The substation automation system is connected to a SCADA system, using a point-to-point or wide area network telecontrol interface.

The substation automation system must fulfil the following functions:

• IEC 61850
• Telecommunication
• Monitoring
• Automation
• Online configuration
• Local and remote control/control with switchgear interlocking/switching sequences
• Serial connection of IEDs and field devices
• Connection to a local HMI
• Archiving and logging of operation and disturbance data
• Open communication channel using OPC client/server mechanisms

The SICAM system consists of the following hierarchy:

• Level 1: protective relays, bay controllers, meters and other IEDs.
• Level 2: substation control level (SICAM station unit/PAS).
• Level 3: HMI (SICAM PASCC/Recpro/DIGSI).

Cyrus Ashrafi (left) and Dan Gacek (right) stand next to a bay controller installed as a retrofit in existing 12-kV switchgear.Click here to enlarge image

The Human Machine Interface (HMI) level consists of the following components:

SICAM PASCC is used by substation personnel to control and monitor the system. SICAM PAS CC is used for process visualization. Specifically designed for energy automation, it assists in operations management optimization. It provides a quick introduction to the subject matter and a clearly arranged display of the system’s operating states.

It is responsible for displaying SCADA data on one-line diagrams, for trending data, for archiving data, and for issuing and recording alarm messages. SICAM PASCC operates on Windows XP and can be configured in a client/server architecture so multiple PCs can access system data.

SICAMPAS is used by system engineers to configure the IEC 61850 system, configure legacy IEDs, and make data available to the local HMI and control centers. It is responsible for identifying which devices are part of the configuration, configuring communications interfaces, specifying which data are available from the devices, where the data has to go, and any processing that has to be performed on the data. The advantages of this platform are low hardware and software costs, ease of operation, scalability, flexibility and constantly available support.

SICAM RecPro is used by protection personnel to analyze fault records after a system fault has occurred. Fault extraction is a service implemented in the IEC 61850 client. When a fault occurs, the records are automatically extracted to the station unit. SICAM RecPro cyclically scans the station unit for new records. SICAM RecPro is configured to delete the fault records from the station unit once collected, but the records are not deleted from the protection devices. The protection devices overwrite the last fault after eight records have been recorded. RecPro organizes the records in the RecPro Explorer in folders specifically associated with the IEDs assigned to the fault. RecPro also notifies the user when any new faults occur and segregates new faults from old fault records. RecPro resides on the station PC, but could be located in the engineers’ office at the city.

The Standard and the Future

The SICAM SAS has proved to be a very reliable integration point within the substation. In the future, city engineers will work toward integrated operation of the autonomous distribution automation teams with the substation breakers.

Today, there is a large variety of regionally standardized and proprietary protocols for substation communication. If a substation’s automation system comes from one supplier, the number of protocols would be kept small and device interoperability is achieved relatively easily. However, if interoperability is required among products of different manufacturers, substantial efforts on engineering and maintenance would be needed on a utility’s part. Because of the globalization and deregulation of electricity markets, not only manufacturers but also large utilities are operating more and more internationally, and the wide variety of incompatible protocols is hindering their activities. Utilities also face high equipment replacement costs because communication equipment of the new devices can be incompatible with that of the old devices, necessitating the use of protocol converters. IEC 61850 will bring these systems together, and the city of Naperville is leading the way.

Daniel Gacek, P.E., is a senior electrical engineering for the City of Naperville, DPU-Electric and has managed substation engineering for the utility for nine years. He has more than 18 years of substation engineering and operating experience and is a member of NSPE and IEEE.

Howard Self, P.E., is the engineering manager for Siemens RTUs and Substation Automation. Self joined Siemens in 1998 as senior application engineer and currently heads the Center of Competence for Siemens Sicam Automation Platform in North America.

What is IEC 61850?

IEC 61850 is an international standard for substation automation systems. It defines the communication between devices in a substation and the related system requirements. It supports all substation automation functions and their engineering. Different from that of earlier standards, the technical approach makes IEC 61850 flexible and future-proof. The ideas behind IEC 61850 are also applicable in areas of automation such as control and monitoring of distributed generation.

IEC 61850 offers a number of advantages over existing protocols. IEC 61850:

• Defines one protocol for the entire substation;
• Fully supports all substation automation functions comprising control, protection and monitoring;
• Is future-proof and facilitates future extensions, therefore it safeguards investments;
• Is a worldwide applicable and accepted standard;
• Defines the quality requirements (reliability, system availability, data integrity, security, etc.), environmental conditions, and the auxiliary services of the system;
• Specifies the engineering processes and its supporting tools, system life-cycle and the quality assurance requirements and maintenance for the entire substation automation system;
• States the conformance tests to be carried out on the products;
• Has flexibility that allows optimisation of system architectures (scalable technology);
• Uses readily available industrial Ethernet and communication components; and
• Facilitates a utility-wide common communication infrastructure, from the control center to the switchyard.