Inside a former lobster tank in Copenhagen’s meatpacking district, IKEA’s future-living lab, SPACE10, maps out how we’ll be living in the next ten to fifteen years. Downstairs, a hydroponic farm grows tomatoes and leafy greens that the lab shares with a restaurant just down the road. Upstairs, a rotating cast of 20 or so bioengineers, chefs, designers and architects imagine the cities of the future and try to work out where and how IKEA might fit into them.
It was here, more than 800 kilometres from IKEA’s headquarters in Delft, the Netherlands, that the furniture company developed its augmented reality app IKEA Place. The app, which launched on iOS in September 2017 and is now available on Android, lets anyone drop virtual furniture into their own homes and view it through their smartphone camera. It was one of the very first apps to take advantage of ARKit, Apple's augmented reality framework that lets developers use the smartphone's motion sensors and cameras to overlay digital elements on the real world. On the day of its release, Apple CEO Tim Cook touted Place as the future of shopping.
But Place wasn’t IKEA’s first foray into the world of augmented reality, says Bas van de Poel, a creative strategist at SPACE10. “IKEA was really one of the first companies to be experimenting in augmented reality,” he says. Way back in 2013, before Pokémon Go brought AR to our smartphones en masse for the first time, IKEA released an app that gave an early hint of what IKEA Place would later become.
Even for its time, IKEA Catalogue was a little shonky. For it to work you had to have a copy of the paper IKEA catalogue, where certain items were marked with an orange label. After scanning a labelled page, users than had to place the physical catalogue on the floor wherever they wanted to drop their virtual furniture. The physical catalogue worked as a kind of size guide, so any of the few hundred items that were available when the app launched were the right size when recreated virtually.
At the core of Catalogue was IKEA’s secret weapon: a vast database of 3D models of almost every single item the company sells. “We have a lot of experience creating 3D models and making them look real,” says Michael Valdsgaard, digital transformation lead at IKEA. IKEA’s designers create these models as they develop new pieces, but in its autumn 2006 catalogue the company took this a step further and debuted its first computer-generated product image – the Bertil chair. In 2009, it followed up with the first complete 3D-modeled room set. Now, the majority of the catalogue is put together using these photorealistic product models instead of real photographs.
So when IKEA heard about the launch of ARKit at Apple’s Worldwide Developer Conference in June 2017, the company knew it already had the 3D models it needed to populate the app, and now had access to the toolkit it needed to actually build the thing and put it on people’s phones. “Overnight something we had been curious about – augmented reality – became a reality for half a billion people,” says Guillaume Charny-Brunet, co-founder of SPACE10.
To make the models work within the app, each one had to be squeezed down to a file size less than 1.5 megabytes large. This meant a careful balancing act of reducing the size while making sure the detail and texture of the model remained. Too little detail, and the illusion of placing a real item in augmented reality would be shattered. Too much detail, and the app simply wouldn’t be able to load the item of furniture over a 4G connection. Right now, Place, which was developed by Dutch augmented reality agency TWNKLS contains 3,200 separate items, although IKEA hopes that eventually the app will include more than 10,000 products from its catalogue.
For Charny-Brunet, it was absolutely critical that the Place app didn’t just give a vague idea of what a piece of furniture would look like in a room, but came as close as possible to the real thing. “It’s about reducing the risk that’s inherent with any home improvement you make,” he says. Through a combination of room scanning and 3D modelling, each piece of furniture in Place is almost perfectly in proportion with the real world.
It wasn’t just the look of the furniture that had to be right, but the sound. When a piece of furniture lands on the floor in Place, it lands with a little thud and a touch of haptic feedback. Those thuds were designed by the Swedish sound studio Plan8, who recorded the sound of a foley artist hitting a wooden board and then edited them so they fit the size and weight of the piece of furniture being dropped in the app.
The SPACE10 team are slowly rolling out new features within the app, including visual search, which lets a user scan an item in the real world using their smartphone camera and offers up a selection of similar products from the IKEA catalogue. Developed by the deep learning company GrokStyle, it’s like Google’s reverse image search, but in the real world.
This could also potentially be a powerful source of data, letting IKEA look inside its customer’s homes for the first time. But, for the time being, IKEA isn’t interested in looking through the camera lenses of people using its app. “We want to help you but we don’t want to intrude on you,” says Valdsgaard. “We could do a crazy amount of commercial things with the data, but I’ve instructed the company to delete it.”
“We truly believe that it’s going to be a competitive advantage to behave ethically,” says SPACE10 co-founder Kaave Pour. He’s already experimenting with ways to add features to Place that don’t revolve around selling furniture. In one upcoming easter egg, people can shake their phone to summon up an augmented reality portal through to an IKEA warehouse that they can enter through their smartphone camera. Future versions could include entire augmented shops that let people search for products physically rather than scrolling through a lists on a smartphone screen.
The team would also like to integrate more AI into the app, giving users the option to select a budget and have an assistant offer suggestions about how they should furnish their room. But for the time being, there are more prosaic problems to deal with – like walls. The current version of ARKit has no problem scanning floor surfaces, but because most walls are painted white, it struggles to detect them. At the moment, this means users can’t hang anything on their walls in augmented reality, something that the app’s designers would like to make possible in later versions.
But in Copenhagen, SPACE10 is thinking beyond Place to how IKEA will help shape our lives and living rooms in the future. The firm is researching co-living, reimagining the food that IKEA might serve in its restaurants and working out what will happen to IKEA stores when more people have easy access to 3D printers. Eventually, IKEA might become as much a technology company as it is a furniture seller. “Right now, a lot of technology is about consumption," says Van De Poel. "But IKEA Place actually allows you to create stuff.”
This article was originally published by WIRED UK
