Utilities envision role for augmented reality to train and assist workers

Amid the whirring cacophony inside a coal-fired power plant, Eric Zakszewski peers into the side of a coal feeder to see that fuel is flowing properly on its way to a pulverizer. On this day, a flashlight isn’t his only tool. A small monocle hangs from his white hard hat in front of his right eye. On it, he can view a list of tasks and safety checks he is to perform without having to turn to a screen or clipboard.

“It’s like a powerpoint that is hovering in mid-air,” said Zakszewski, a 29-year-old power plant operator at We Energies’ Elm Road Generating Station near Milwaukee.

Zakszewski is among a group of utility workers who recently participated in an industry-backed study looking at the potential benefits and drawbacks of equipping employees with augmented reality (AR) devices, which layer virtual information into a user’s field of vision. A team from the Electric Power Research Institute (EPRI) visited the We Energies coal plant last week to observe workers while they tried out various products.

“The potential uses of AR in the electric utility sphere are already in play,” Eric Bauman, a senior technical leader with EPRI, said. “It is being demonstrated for inspection tasks, performing work, training, and probably more. There is substantial interest amongst utilities in this technology.”

The technology is maturing and coming down in price as the utility industry faces a looming workforce challenge that will require it to hire and train thousands of workers in the coming years as aging workers retire. Still, more vetting is needed to make sure the devices actually help workers and also don’t create dangerous distractions.

AR: “It could be really useful”

Zakszewski performed a series of jobs inside and outside the We Energies plant last week while wearing two types of AR devices — RealWear, the monocle lens, and Microsoft’s HoloLens, a set of glasses.

“For the most part, my job is looking and observing,” Zakszewski said. “So, AR does obstruct it a little, but if you could have it off to the side it could be really useful.”

Besides scrolling through checklists, the devices will become more useful as more meters, poles, and other grid equipment is wired into the internet of things. With AR glasses, all that data streaming from these smart devices can be visualized, which is “absolutely part of the allure of AR,” Bauman said.

Brian Dixon, chief operating officer of Capital Innovators in St. Louis and head of the Ameren Accelerator, an energy technology startup accelerator, agreed. “Imagine somebody working for a utility in the field is able to look at the transmission wires or different types of large assets and get a perfect understanding through the data that’s being projected into their field of vision,” Dixon said.

Ameren, one of Illinois’ largest utilities, is among the companies exploring how AR gadgets can bring efficiency and safety to its operations.

“Ameren sees significant long-term potential in applying augmented reality technology to maintenance operations, as well as other aspects of our business,” said Steve Kidwell, Ameren’s vice president of corporate planning.

Over time, Kidwell predicts the devices will reduce costs and improve service while helping to train and keep workers safe in the field.

Products cost less, perform better

A 2016 EPRI report that reviewed existing research literature concluded that affordability is no longer an obstacle for utilities, as costs for the devices have come down. “More and more players are entering the marketplace. It is a vibrant area of technological development and experimentation,” Bauman said.

Vuzix, a developer based in Rochester, New York, is creator of the Blade, a $1,000 pair of AR glasses that work with Amazon’s voice-controlled Alexa platform. Designed to look like sunglasses, the Blade has received high praise from the technology press — The Verge called it the “next-gen Google Glass we’ve all been waiting for.”

But the company also produces the M-300, a sturdier device designed for industrial use that runs about $1,500. Instead of looking through glasses or a large pair of goggles, a user looks through a small monocle that mounts onto a hard hat. The company’s chief executive envisions a role for the product in the utility industry.

“You can literally just look at the meter, and have the meter reading captured on the device,” Paul Boris, Vuzix CEO, said. “You don’t get errors in transcription, you don’t have to tap the reader, you don’t even have to get close to it — you can literally be on the other side of the fence.”

Meanwhile, other utility hardware is getting smarter, expanding the potential uses of AR glasses to interact with equipment.

Ontario-based Hyperion Sensors, one of the first companies in Ameren’s startup incubator, developed fiber-optic sensors the size of human hair that can be strung into transformers and other high voltage equipment, making the “dumb” equipment smart and providing instant information about temperature and strain.

“Over the next 20 or 30 years, the idea is that this smart technology will be part and parcel of everything that gets connected to the grid,” Anselm Viswasam, CEO of Hyperion Sensors, said. “If you are going to use a solar panel and it’s going to supply power onto the grid. Then it will be necessary that it will have the smarts embedded into it.”

A tool for looming workforce challenge?

The utility industry faces some significant workforce challenges ahead as the baby boomers retire, which was highlighted by a 2017 Department of Energy utility workforce assessment. Few utility workers have the training or qualifications to move up the ranks as older workers retire, it concluded.

“Workforce retirements are a pressing challenge. Industry hiring managers often report that lack of candidate training, experience, or technical skills are major reasons why replacement personnel can be challenging to find—especially in electric power generation,” the Energy Department report said.

We Energies plant central scheduler Randy Sheck sees potential for using augmented reality to train and assist new workers. Utility workers routinely work 12-hour shifts, walk 5 or 6 miles and examine 300 pieces of equipment.

“You always have to be looking for new technology,” he said. “I think augmented reality may still be 5 or 10 years out, but it has a lot of potential. I see the value it can add for us in terms of daily maintenance.”