Heads up: Augmented reality prepares for the battlefield

At last week's Pentagon Lab Day in Washington, DC, the Army's Communications-Electronics Research, Development and Engineering Center (CERDEC) and Army Research Lab demonstrated a prototype of technology straight out of first-person shooter games—an "augmented reality" heads-up display that could help soldiers tap into sensors and other data.

Called Tactical Augmented Reality (TAR), the technology is the latest evolution of the Army's effort to network soldiers together and give them "situational awareness" on the battlefield—where they are, where their friends are, where the adversary is, and everything else they need to know for their mission, tied into tactical communications. Over the past few years, CERDEC, ARL, and the Defense Advanced Research Projects Agency have been working on the core technologies to make augmented reality work on the battlefield, including the development of a platform called the Heads Up Navigation, Tracking and Reporting (HUNTR) system.

While HUNTR is relatively recent, it is built on nearly three decades of efforts by the Army to digitally enhance the foot soldier. Up until recently, those efforts ran up hard against the limitations of wearable computing. Even as the technology finally matures, it's probably years away from seeing service in the field.

Back to the future

The Army has been trying to put things in front of soldiers' eyes since the late 1980s. First was Land Warrior (or, as it was more formally called, the Land Warrior Integrated Fighting System). Land Warrior sprung out of a technology demonstration program in 1989 called Soldier Integrated Protective Ensemble (SIPE), which showed that enhanced sensors and communications could significantly boost the combat capabilities of small infantry units.

The concepts behind Land Warrior would get tried out and refined repeatedly over the next decade, but they didn't take shape as an actual procurement program until 2000. That's when they became part of the Army's massive Future Combat Systems program. When Land Warrior was officially awarded as a full-blown development program to Raytheon, its planned kit included an integrated computer system as part of a soldier's kit, as well as a helmet-mounted display. That device served as a data display for communications and navigation, as well as a viewer for a thermal imaging sight on the soldier's "weapons sub-system" (the M16 rifle or M4 carbine).

The computer, integrated GPS, and radio allowed (in theory) commanders to track, in real time, where troops were. But Land Warrior didn't exactly have a "heads-up" display, per se—it could show positions on a two-dimensional map, and it allowed soldiers to aim weapons around corners. But Land Warrior didn't overlay information on the world around the soldier. And it also had some significant problems—the most obvious of which was the added weight required for batteries, computing power, radios, and a TV screen mounted on a soldier's head.

Full combat gear for an infantryman is already approximately 80 pounds by itself. Land Warrior in its full implementation added 40 pounds more gear (computer plus helmet with video display, plus digital gun sights, plus radio, plus batteries) to a soldier's load. Carrying 120 pounds of gear under combat conditions, for those who've never had the experience themselves, is the equivalent of lugging Hillary Clinton's retired e-mail servers on your back while jogging down the Vegas Strip in July. Based on some of the reviews from soldiers who used the gear, the servers might have been equally combat-effective.

Only one version of Land Warrior ever saw combat: the "Stryker Interoperable" variant was designed to work with the systems aboard the Stryker Infantry Fighting Vehicle. It deployed to Iraq with the 4th Battalion, 9th Infantry Regiment. The Stryker configuration of Land Warrior lowers the weight of the gear primarily by allowing soldiers to leave spare batteries in their vehicles. The gear is still deployed with Stryker mechanized infantry units.

Ironically, the entire Land Warrior program was cancelled in 2007, as its cost mounted—the cost of the gear for a single soldier exceeded $85,000. The program was scaled back and rebooted, becoming Ground Soldier System (GSS). But as the need to get something out to troops quickly mounted, GSS morphed into "Nett Warrior"—named in honor of World War II Medal of Honor winner Colonel Robert B. Nett. To meet needs quicker (and cheaper), the Army dispensed with the helmet-mounted monocle and went to a chest-mounted Android device, using off-the-shelf hardware components to achieve mostly the same thing—except now soldiers had to look down to check their smartphones.

Nett improvement

But Nett Warrior gave the Army a common platform to build from, just as commercial augmented-reality gear like Google Glass and Microsoft's Halo were about to arrive. The Android-based system used for the Nett Warrior "end-user device" (EUD) gives the army both an existing common networking interface and a development platform ideal for building wearable computing applications.

Like Land Warrior before it, the Army's vision for TAR is a combination of enhanced navigation, friendly force tracking, information sharing, and weapons targeting. But instead of having a monocle display "dashboard" to watch, the soldier would have data projected directly on their field of view in a way that meshes seamlessly with the terrain around them.

The HUNTR system developed over the past two years uses an Android port of a software package called ARC 4, according to a CERDEC presentation given at an industry event in October 2015. ARC, developed by Applied Research Associates, was also used in an earlier DARPA augmented-reality program called Ultra-Vis. The data from ARC can be displayed either within enhanced night-vision gear or on a variety of existing head-mounted see-through displays.

The software uses Global Positioning System data, helmet-camera data, and inertial sensors to "geo-register" the soldier's field of view. That allows symbols designating waypoints, points of interest, and friendly forces to be projected on what the soldier sees, as well as a navigational "compass" showing the direction to tracked objects when they're not in view. Additionally, a 3-D model of terrain can be superimposed on the real world to help in navigation. This data can be pulled in from the Nett Warrior device and pushed out to soldiers from an operations center and other sources as needed.

Display of determination

One of the major remaining problems for tactical augmented reality is that off-the-shelf display technology simply hasn't been up to the demands of battlefield use. While it's easy enough, relatively speaking, to project augmented-reality data on night-vision displays, daytime use of augmented reality has been made difficult by the relatively small field of view and low contrast of see-through displays.

Even Microsoft Hololens' relatively large field of view may not be enough for tactical applications. And most displays don't generate enough relative brightness to be visible over brightly lit desert or winter terrain (as shown by the CERDEC slide in the gallery above). So ARL and CERDEC had to work with suppliers to build their own.

Previous prototypes of TAR displays used black-and-white or "greenscreen" projections to maintain enough contrast to be visible. Color displays in the size range used (about one inch diagonally) could not provide enough resolution. But the prototype TAR display shown at Lab Days has significantly more contrast. The prototype allows color images to be superimposed on even the brightest backgrounds.

Meanwhile, Applied Research Associates is looking for commercial and consumer applications of its software—from logistics to augmented-reality gaming to social media. So while the Army pushes augmented reality to the battlefield, we may all be seeing data swimming in front of our eyes.