Repairing a Failed Turbine: The Story of Returning G.M. Shrum Unit 3 to Service

G.M. Shrum, BC Hydro's largest facility, began operating in 1968 with 10 turbine-generating units. Units 1 to 5 were installed in 1968 and 1969 and have a capacity of 261 MW each. These Francis turbine runners are fully cast from mild carbon steel and have a stainless steel overlay for cavitation protection. Together, these five units represent 12 percent of BC Hydro's electricity producing capacity. Units 6 to 8 were installed in 1971 and upgraded in 2004 and have a capacity of 275 MW each. Units 9 and 10 were installed in 1974 and 1980, respectively, and have a capacity of 300 MW each.

Because of the importance of Unit 3 at G.M. Shrum, BC Hydro implemented a plan to get it back on line as quickly as possible. This plan involved rebuilding the runner by using a new blade design and reusing the existing crown and band, as well as implementing a novel approach to repair the damaged wicket gates. Using these innovations, BC Hydro was able to return Unit 3 to service in just 14 months.

How and Why the Unit Failed

The failure event began at 5:05 a.m. on March 2, 2008, when unit output dropped to 237 MW from 254 MW. The servomotors began moving open, to 84.3 percent from 71.3 percent. Power output returned to 254 MW at 5:06:27 a.m. By about 5:11 a.m., the servomotors had reached 97.2 percent open, with power output of 262 MW. Turbine bearing temperature and synchronous vibration began increasing.

Then, at about 5:14 a.m., power output dropped to 247 MW before settling at 252 MW. Turbine synchronous vibration increased again. At 5:19:30 a.m., power output increased to 258 MW. Between about 5:21 and 5:22 a.m., the servomotors moved to 93 percent open from 97.2 percent open, with no change in power output. At about 5:22 a.m., power output plummeted to 13 MW, without servomotor movement. Output then settled at 55 MW, and turbine inlet pressure stabilized at a value 3.7 percent higher than nominal. Turbine bearing temperature began climbing rapidly.

At 5:28:30 a.m., the operator tripped the unit and power output dropped to 0 MW. One minute later, the unit reached 133.9 percent speed because the wickets gates were no longer able to regulate the flow of water through the turbine. Two minutes after that, turbine bearing temperature reached 129.4 C. Finally, at about 5:39 a.m., the unit stopped.

In the days after the failure event, BC Hydro personnel performed a unit assessment. Below are results of this assessment, as well as subsequent inspections throughout the disassembly process:

• The scroll case and stay vanes were free of damage from impact or abrasion. There was no indication of material or debris having traveled down the scroll case or passed through the stay vanes and no residual debris was found in the scroll case.
• The skin plate on the pressure side of all 24 wicket gates was in good condition. However, the skin plate on the suction side, downstream of the seal contact line, had deep gouges on up to 75 percent of the surface. The pattern of the damage was indicative of pieces of metal becoming caught in the wicket gate/runner cascade while the turbine was rotating.
• The trailing edges of wicket gates 1 to 10 and 15 to 24 were bent due to impact. On all 20 of these gates, the impact pattern and location of the damage were the same (about 12 inches above the bottom facing plate). However, the trailing edges of wicket gates 11 to 14 were in good condition, with no impact damage.
• Runner blades 4, 11 and 14 were missing significant pieces (about 70 inches by 70 inches) from the outlet. The shapes of these missing pieces were almost identical, and one piece was found intact in the draft tube. In addition, many smaller runner blade pieces (8 inches by 8 inches) were found in the runner/wicket gate cascade, stuck between adjacent blades and in the draft tube.
• The inlet edge of all 17 runner blades had impact and heavy abrasion damage. On "moderately" damaged inlet edges, the damage was localized at 12 inches above the bottom facing plate (the same elevation as the trailing edge damage on wicket gates 1 to 10 and 15 to 24). In addition, significant cracks were visible on 12 of the blades.
• The area between wicket gates 11 to 14 contained a carbon steel deposit on the stainless steel lower seal ring, indicating that rubbing had taken place with the runner band.
• All shear pins had failed, with pin 11 failing due to fatigue, pin 13 failing due to bending at a high rate of strain, and all other pins failing due to shear overload.

BC Hydro concluded that the most likely failure mode was a cascade closure of four adjacent wicket gates due to shear pin failure. In this scenario, the shear pin on wicket gate 11 failed due to fatigue, resulting in rapid closing of the wicket gate. Shear pin 11 functions as a shear pin and link pin. Thus, this element would experience cyclical bending and torsional forces due to increased clearances between the pin and bushing, lubrication issues, misalignments in the operating mechanism and interference between the lever and mechanical stop on the head cover.

The lever of wicket gate 11 then contacted the lever of wicket gate 12, breaking its shear pin. Both wicket gates assumed an almost closed position. The governor then instructed the servomotors to increase the wicket gate opening to maintain power output and to respond to an increased demand for power. Shortly after the servomotor reached its maximum opening, the shear pin on gate 13 failed. Wicket gate 13 closed and its lever contacted that of gate 14, breaking its shear pin.

With four wicket gates closed, water flow in the scroll case became unbalanced and a low-pressure zone was created behind those gates. The runner was forced toward the low pressure until contact occurred between the runner band and lower seal ring. The stresses on the runner caused new cracks and accelerated existing cracks, resulting in failure of blades 4, 11 and 14. As the blades broke apart, one piece was discharged into the draft tube and two were ejected into the wicket gate/runner cascade, slamming the remaining wicket gates closed, breaking the remaining shear pins and further damaging the runner blades and wicket gates.

Top image: The Unit 3 runner suffered significant damage during a failure event, including a large piece missing (middle right) from the inlet edge of one of the runner blades.

Bottom image: The repaired Unit 3 runner was returned to the facility in February 2009, just 11 months after the failure.

Choosing the Repair Method

On March 13, 2008, BC Hydro invited three turbine suppliers to view the damaged turbine. On March 20, BC Hydro issued a request for proposals to these suppliers. In its proposal, each supplier was to describe its repair method for each turbine component, including the cost and expected duration. Proposals were received on March 31. After reviewing the technical, schedule and cost aspects of each proposal, BC Hydro determined that the solution offered by Voith Hydro was the best for its business needs.

On April 15, 2008, BC Hydro issued a notice to proceed to Voith Hydro for repair of the Unit 3 turbine. This turbine is scheduled to be upgraded in 2017 as part of the G.M. Shrum Units 1 to 5 Turbine Upgrade Project. Thus, the goal of the repair work was to return Unit 3 to commercial operation as quickly as possible, with a minimum service life of 10 years.

The runner repair was the most time-intensive work needed to get the turbine operational. To expedite the repair, Voith Hydro proposed to reuse the crown and band and supply new blades. This removed the long lead time for crown and band castings, which would have added about one year to the schedule. To further reduce the time required for the repair, Voith Hydro chose to fabricate the new blades using A516 Gr. 70 plate steel instead of stainless steel. This plate steel was a stock item readily available from a warehouse, whereas the stainless steel would require a special delivery. Thus, the repair cycle time was reduced to nine months.

However, the decision to reuse the crown and band with a new blade design presented challenges. For example, the new blades were designed using only finite element analysis (FEA) and computational fluid dynamics (CFD). No model testing was performed. Additionally, the new blade design had to fit within the existing crown and band water passageway, including leaving locations for the thrust relief holes and space to install new rotating wearing rings on the outside diameter.

For the wicket gate repair, the main goal was to retrieve the functionality of the gates. A time frame of seven months was foreseen for this work.

The wicket gates are of the hollow fabricated type, made of welded carbon steel plates. The body is made of two formed 1.75-inch-thick plates with an internal rib welded on the suction and pressure side plates. The nose contact face is equipped with a rubber strip to obtain a tight seal when closed, held in place with a stainless steel clamp plate and patch bolts. The head cover and bottom ring also feature wicket gate end seals so the distributor is watertight when closed. Additionally, the top and bottom end of each gate body is overlaid with 17-7 stainless steel.

Once the unit was dismantled, BC Hydro personnel discovered that the damage on the wicket gates was extensive. The suction side of the hydraulic surface showed scores in the range of 0.625 inches deep over its entire surface. Some wicket gates also showed deformations of the suction side skin plate, with cracks through the thickness of the plate. Cavitation damage was present immediately downstream of the nose rubber seal and in the vicinity of the stem to blade fillets. Under normal conditions, the wicket gates would have been replaced. However, due to time constraints, this was not an option.

BC Hydro and Voith Hydro worked together to identify three repair options:

• Completely replace the skin plate on the suction side
• Weld overlay and grind the skin plate to restore shape and surface finish or
• Mill a 0.5-inch-deep "pocket" on the suction side and install a new 0.5-inch-thick shaped skin plate. Damage deeper than 0.5 inches would be weld repaired before installing the skin plate.

The two first options were not desirable because they would induce significant distortions due to the welding volume involved. Therefore, BC Hydro and Voith Hydro determined that the third option was the most viable choice.

Making the Repairs

Voith Hydro began disassembling the unit at the end of April 2008, only two weeks after the contract award. Disassembly activities were completed by the end of May 2008.

Immediately after the runner was removed from the turbine pit, Voith Hydro personnel removed the blades, leaving only 3 inches of blade protruding from the base metal. Voith Hydro then shipped the crown and band to its facility in York, Pa., arriving on July 8, 2008.

Because of the large amount of stainless steel overlay on the crown and band, Voith Hydro machined a 0.25-inch-deep cut on the crown and band water passage surfaces. The new blade design took this variation into account. The design also incorporated an overlay on the flange face and spigot area to keep the runner centerline at the distributor centerline (relative to the shaft flange face). After completing rough machining on the water passage surfaces, Voith Hydro personnel performed a magnetic particle inspection. This inspection confirmed that some of the cracking from the damaged blades extended into the crown.

The crown required extensive weld repairs. There were four defects exceeding 3 inches in depth and 18 inches long that were repaired with E309L weld metal. Machining on the crown revealed large areas of cavitation damage and stainless steel overlay, so Voith Hydro personnel machined away a layer of metal 0.625 inches deep in the area of the junction of the crown and blade discharge area. Voith Hydro personnel then overlaid this area with E309L filler metal. This area was then subjected to a non-destructive ultrasonic inspection. In total, the crown required about 1,800 pounds of weld metal. This included overlay and crack and cavitation repairs. The crown required re-machining after the weld repairs and before going to runner assembly.

The band repairs were not as extensive. The band only required about 100 pounds of weld metal and had no major defects. Only cavitation damage and the stainless steel overlay areas needed to be repaired.

Voith Hydro then completed runner assembly. The blade-to-crown and blade-to-band welds were made with E309L weld wire. These welds were inspected using non-destructive dye penetrant and ultrasonic methods. The completed runner was not post-weld heat treated due to the E309L welding. With the previous design, both the crown and band rotating seals had been integral. With the new runner, material was machined off of these areas to install new shrunk-on rotating seal rings. This allowed the seals to run true to the shaft spigot after the fabrication processes. Finally, the coupling holes were reamed oversized by 0.06 inches using a template to match the shaft and new coupling bolts were installed.

The new runner has only 15 blades, compared to 17 for the old runner. Even with the decrease in number of blades, the total area of contact between the blades and crown increased by about 20 percent.

To be confident in the reliability of the repaired runner, Voith Hydro performed a complete FEA. This included a CFD study, static stress analysis and dynamic stress analysis. The dynamic analysis was performed to determine the risk of structural resonance in the runner and to evaluate its dynamic response over the range of exciting frequencies experienced during normal operation. The results from these studies were used for a fatigue study. Results from the fatigue study indicated that the runner will be reliable for more than 10 years. The replacement runner arrived at the site on February 20, 2009.

In early June 2008, the wicket gates were shipped from G.M. Shrum to the shop of a Voith Hydro subcontractor for a complete rehabilitation. Shop personnel began with a complete examination of the wicket gates to map any cracks and abnormal conditions. Many cracks were repaired on the skin plates and also on the stainless steel overlay at both ends of the wicket gate body. Afterwards, a 0.5-inch-deep pocket was milled in the 1.75-inch-thick skin plate on the suction side where the gouges and dents were present. Subcontractor personnel then installed a formed and machined carbon steel plate in the pocket, restoring a clean and uniform hydraulic shape. The carbon steel plate was secured by means of a peripheral weld in addition to 16 patch bolts. The wicket gates went through a post-weld heat treatment before machining work began.

This wicket gate from Unit 3 at the 2,730-MW G.M Shrum plant shows typical damage uncovered after the runner failure. The skin plate on the suction side of all 24 wicket gates had deep gouges on up to 75 percent of the metal surface.

The remaining rehab work consisted of replacing the stainless steel sleeves with oversized ones, which provided the machining allowance needed to correct stem runouts. The contact faces of the wicket gate body had been re-machined to restore their parallelism. Therefore, it was necessary to re-dowel the gate arm on the wicket gate stem. The rubber seal and its clamp plate also were replaced with new ones. Finally, two coats of epoxy paint were applied on the hydraulic surfaces.

The wicket gates at 2,730-MW G.M. Shrum were repaired by milling a 0.5-inch-deep pocket in the 1.75-inch-thick skin plate. The contractor then welded and screwed a formed and machined carbon steel plate into the pocket to fill the cavity.

The structural modification of the wicket gates was validated using an extensive FEA model. The weld in the periphery of the new 0.5-inch skin plate was sized to restore the inertia of the gate body, while the size and number of patch bolts provided for an adequate clamping of the skin plate when subjected to vacuum (worst cases considered).

The wicket gates were returned to G.M. Shrum in January 2009, after only seven months of repair work.

Repairing Other Structures

The other structures repaired on Unit 3 include the embedded parts, head cover, turbine guide bearing and turbine shaft.

Cracks on multiple stay vanes were repaired and the embedded parts were machined at G.M. Shrum to restore an acceptable flatness.

Examination of the head cover showed only a few cracks that were quickly repaired. However, the geometry of the head cover needed reworking, which was accomplished by a Voith Hydro subcontractor.

The turbine guide bearing was heavily damaged. Rehab work consisted of melting the original babbitt, pouring a new coat of babbitt and completely re-machining the coupling flanges and inside and outside diameters.

The turbine shaft was shipped to the shop of a Voith Hydro subcontractor for machining of the bearing journal and shaft seal journal surfaces.

Results

On February 20, 2009, Voith Hydro began reassembling the unit. Seven weeks later, the turbine-generator was ready to be commissioned by BC Hydro. Unit 3 was returned to service on May 5, 2009, just 14 months after the failure. To date, the repaired turbine has performed reliably and has met BC Hydro's expectations. In August 2009, minor cavitation damage was observed on the inlet edge of the runner blades. Voith Hydro implemented stainless steel overlay to repair the damage and to mitigate the effect of future cavitation.

As a result of lessons learned from the Unit 3 failure, BC Hydro has reviewed the risk of reoccurrence of this event on all turbines within its generating fleet. Included with this review is a reevaluation of BC Hydro's requirements for shear pin breakage monitoring and unit automatic shut down after a broken shear pin.

Peter Finnegan, P.Eng., a mechanical engineer in BC Hydro's turbine engineering department, was the project engineer for the Unit 3 failure investigation and repair.

James Bartkowiak, P.E., manager of mechanical engineering with Voith Hydro in York, Pa., was lead engineer for the runner repair and finite element analysis studies of the embedded components.

Luc Deslandes, ing., department head of the turbine engineering group with Voith Hydro in Montreal, Quebec, Canada, was lead engineer for the repair work and was involved in repair of turbine components other than the runner.