People who own a Moto X, the flagship phone of Google-powered handset maker Motorola Mobility, might be wondering this morning: What the hell is that little red sombrero doing on my home screen? Those adventurous enough to poke a finger at the hat are in for a big surprise.
With that single touch, a Moto X turns into a player for a new form of media – one that combines computer animation and street theater with virtual reality. The phone's screen becomes a looking-glass that peers into a stylized virtual forest. Music plays, with the Django-esque, jazzy feel of an old-time Paris bistro. Move the phone up, down, right, left, and you can see more and more of this colorful world – including that red chapeau, and an expressive mouse named Pepe, who is alternatively intrigued and stalked by a hat blown around by a stiff, persistent breeze.
He encounters other creatures – squirrels, raccoons, a bear – who assist or impede him, and sometimes, he gets flattened a lá Wile E. Coyote. You follow the story by peering through the phone at the mouse, the hat, and the creatures as they perform, like using binoculars to pan over a landscape. If you turn away, you can explore your own surroundings, but when you return your eyes once more to Pepe and his cohorts, it’s as if the narrative has paused for you, resuming now that you’re paying attention again.
All of this is presented with astonishingly high quality, and at times, it's actually laugh-out-loud funny. In fact, the whole package – characterization, animation, scriptwriting, art – reaches the notoriously stratospheric level of a Pixar movie. This is not surprising because Motorola hired actual Pixar moviemakers to create it, including the co-director of Ratatouille.
Motorola is concocting what it hopes will be much more than an odd and expensive interactive cartoon. The company has set out to build a new platform that uses contemporary technology – powerful computation, smart sensors, vivid mobile displays – to reinvent the age-old practice of narrative itself.
Windy Day, as the production is called, came from Motorola’s in-house moonshot division, Advanced Technology And Products (ATAP). This research group, begun in May 2012 (the same month that Google’s $12.5 billion purchase of Motorola Mobile became official), shares the high ambitions of its parent company’s own long-term research group, Google X. But ATAP has a different model: DARPA, the US Defense Department’s Advanced Research Projects Agency. DARPA’s accomplishments include game-changers like lasers, stealth bombers, and that little thing we call the internet.
Over the years, DARPA has developed a well-honed process to develop its breakthroughs, a regimen Motorola Mobility’s new CEO Dennis Woodside thought was worth emulating. So he hired DARPA’s charismatic director, Regina Dugan, as a senior vice president. Making the jump with Dugan was her deputy, Ken Gabriel.
They instantly began a mini-DARPA inside Motorola. Like DARPA, ATAP engages researchers for two-year stints, directing them to take on a project just at the point where new technologies make it possible to make a groundbreaking advance. Project leaders are free to contract with outsiders to assist – enlisting some of the world’s best minds for edgy research with specific goals. Some of those goals are just what you’d expect from a commercial R&D division. Just as DARPA undertakes certain projects with obvious military application, ATAP looks at obvious stuff like speeding up graphics, developing thinner materials, or making your phone's battery last longer. If the result won’t be at least five to ten times better, a project is a non-starter. But it doesn't just explore the stuff that other people want. "It’s also our job at ATAP to do things that they don’t know to ask for,” says Ken Gabriel.
One of those things is what’s appearing on Moto X phones today, a new type of storytelling that Motorola calls "Spotlight Stories."
It began with Dugan’s insistence that the company should explore “what it means to have an experiential device.” She felt there was a strange dissonance between the power of our smartphones and the relatively narrow relationships we have with them. Though we are increasingly dependent on our phones, they do not provide the emotional kick of, say, a game console. Yet the technology inside our phones now rivals high-end devices like Xbox 360 and PlayStation 3. Most of the time, however, the powerful graphics capability on the phone is absolutely idle. Was there a way to do something amazing with all that wasted computational power?
The answer came, in part, from a Motorola engineer named Baback Elmieh. Motorola had bought his graphics optimization company not long before the Google deal, and Elmieh found himself at ATAP on one of those two-year sinecures. In June 2012, he and his colleagues took on the idle-chip dilemma, ultimately deciding to use the Moto X’s mighty chipset to "make graphics into a canvas," he says. "Something that’s more human than just another utility."
So the next few months, ATAP worked on building a graphics-rich platform for virtual worlds on smartphones. The project codename was "Avatar." By October, the technology was well advanced. "The whole team was just bad-ass graphics guys," says Dugan. "They figured out how to get at this untapped power." But finding great stuff to do with that platform was a much more difficult task. They tried creating animated assistants to help people master apps, but the creations bore too close a kinship with Microsoft’s not-so-lamented Clippy, the annoying, animated office supply that provided unwelcome tips to Office users. Some stabs at creating games weren’t so successful, either.
>'The honest answer is that we tried painting on this new canvas, and we pretty much sucked at it. So we had to call in the professionals'
Regina Dugan
"The honest answer is that we tried painting on this new canvas, and we pretty much sucked at it," says Dugan. "So we had to call in the professionals." Thus began a collaboration between Motorola’s engineers and Hollywood creatives, which would result in moviemakers becoming more geeky and Regina Dugan getting a producer’s credit in IMDB.
By late 2012, Dugan and Elmieh were in Portland, Oregon, trying to recruit a director. Their target was Jan Pinkava, an Oscar-winning moviemaker who had worked on multiple Pixar films. Pinkava recalls that Motorola team demoed the technology to him, and explained that they needed artists to use it to make beautiful, to engage people emotionally – to make something people will love. And Pinkava thought, "This is the best gig ever!"
Pinkava’s key partner in the production was Doug Sweetland, an animator with key roles in numerous Pixar films and director of Presto, the Oscar-nominated short that ran before showings of Wall-E. Early in the process, they had an idea that would shape the entire project: give the audience the camera. "That was the breakthrough moment," says Pinkava. "We’re not doing a game. We’re doing a narrative. Then the question became what we could do to make it work, so that you could have the sense of watching the story unfold before your eyes."
In other words, instead of a director choosing to pan across a scene, the viewer would do it. That meant that the filmmakers would have to provide visuals for a 360-degree field of vision. But while the Windy Day world is a 360 degree experience, the action itself is always limited to a virtual proscenium of about 90 degrees.
"It’s like a live stage show," says Pinkava.
To create, this "immersive short," as they called it, they needed a story tailored to the new medium. Sweetland and yet another Pixar veteran, Mark Oftedal (who did the character development for A Bug’s Life) worked up the plot of a mouse and a hat. "It’s like the old vaudeville bit of the guy trying to pick up his hat, and he keeps kicking it off the stage," says Sweetland. "In our case it was the wind. It can go in any direction we want, so we have total freedom."
Sweetland says that while the main story provided a backbone, the user-directed field of view allowed for more than one story to be told at once. For instance, at one point, if the viewer keeps looking at a group of chipmunks while the mouse heads elsewhere, they will see a bit of action they’d otherwise miss. "Splitting focus is usually a bad thing," he says. "But here, it’s a feature."
>'Splitting focus is usually a bad thing. But here, it’s a feature'
Doug Sweetland
As the project progressed, the Google-backed Motorola dug into its deep pockets to support it, essentially spending millions to finance a movie project – with no box office in sight to offset the costs. It built a studio in Portland to accommodate Pinkava, and recruited a jaw-dropping collection of A-list talent. It convinced renowned designer Tadahiro Uesugi, the key concept artist for the film Coraline, to sketch the original look of the Windy Day world. The production designer who brought those ideas to life was Caldecott-winning illustrator Jon Klassen.
Eventually, the creative team swelled to around 40 people, most based in Silicon Valley, Portland, or Hollywood, but with others spread over eight countries, including Spain, Denmark, Brazil, and Thailand.
The Avatar team was thrilled to take on the problem-solving involved in pioneering a new medium. Even something like creating a soundtrack had to be rethought. Composer Scot Stafford not only had to do the usual work of creating themes for the characters and sonic punctuations for the action, but weave them coherently into a production with no set running time. Pinkava explains that the problem was solved by creating "layers of music that can overlap each other like armadillo scales. They’re flexible, depending on how the timing stretches and contracts."
Though Pixar veterans were familiar with pushing the technology envelope, this was different, because they were creating for the specific hardware of the Moto X, and depending on the software crafted by the Motorola team to pull off the virtual reality illusion. "We had to make an algorithm that made you feel you were looking through a camcorder," says Elmieh, who was Avatar's Technical Program Lead. "It’s a lot different that the algorithms that help you find what you want on a map. It has to made you feel like the movie is part of the phone itself."
"If we did our job well, we’ve taken all those tech advances – from the graphics to the sensor suite to the sound – and used them in the service of storytelling," says Dugan.
Pinkava says Windy Day is something offered "not just for the user, but the audience. We’re actually bringing this entertainment to you directly and personally. You’re not paying ten bucks and going into a big dark room with a bunch of other people. We’re coming to you, so we have to respect you. And from that thinking came the idea of a gift."
>'If we did our job well, we’ve taken all those tech advances – from the graphics to the sensor suite to the sound – and used them in the service of storytelling'
Regina Dugan
What’s more, it’s a personal gift. Unlike regular movies, Windy Day is unique for each viewer. "These aren’t prerecorded scenes you’re sitting through," says Ken Gabriel. "It’s a world rendered in real time based where you are in the story, what you’re looking at, what your physical orientation is. Every single person is going to see this differently." Gabriel says that while this can be terrifying for artists, they also benefit from unprecedented feedback. "We can tell what parts of the story people spend time on, and know what story to write next. If 80 percent of the people keep watching the B-loop we did with the raccoons, we know that people really like raccoons!"
Where do Spotlight Stories go from here? First, Motorola will produce more of those immersive shorts. "We’ve got some people joining whose name you will recognize," says Elmieh. The Windy Day creators won’t say for sure if they return for an encore, but Sweetland sounds like a convert. "I feel like movies are being taken off the wall for the first time," he says. "Not necessarily as something you sit and watch, but something you hold. That’s a huge paradigm shift with immense creative possibilities."
To make the most of those possibilities, of course, Motorola must open its platform so that anyone could create Spotlight Stories. Motorola says that could happen, but is vague on how and when that might unfold. But if this new technology is going to mean something to Motorola – which has won respect but not huge audience share with the Moto X – it must eventually open the creative gates to anyone wishing to help them invent this medium.
In the meantime, Moto X users can enjoy the gift of Windy Day, and wonder if it's the beginning of something big, or simply the most elaborate Easter Egg since Fabergé.
