When J.J. Abrams’ Star Wars VII: The Force Awakens hit theaters in 2015, it was no surprise that moviegoers came away with lots of questions. Abrams — known for series like Lost and Alias — is notoriously fond of mystery, which he calls “the catalyst for imagination.” In a 2007 TED Talk, he presented a box he bought as a kid, and still hadn’t opened decades later — because unopened, it represented “infinite possibility.” The Force Awakens suggested a similar infinite possibility, in a universe where 30 years had passed since the death of Darth Vader and the Emperor. The time jump opened up endless opportunities. And though the film returned to the same basic pattern as the first Star Wars, with new versions of the Empire, the Rebellion, and Luke Skywalker, the film still ended with a pile of open-ended mysteries that obsessed fans. Why did Luke Skywalker go into exile? Who were Rey’s parents? Who was Supreme Leader Snoke, and how did he start the First Order?

Over the next two years, fan theories filled these gaps. Maybe Snoke was really the Sith Lord Darth Plagueis — or maybe he was Jar Jar Binks, or Mace Windu, or the Emperor reincarnated. Meanwhile, Rey was definitely a long-lost twin sister of Ben Solo. Or was she a daughter of Obi-Wan Kenobi? Fans hoped Rian Johnson’s The Last Jedi might resolve these mysteries. And it certainly addressed some of them. But inevitably, the answers about Rey’s parentage didn’t please everyone, and the lack of answers about Snoke didn’t seem to please anyone. Still, Johnson’s approach doesn’t weaken The Last Jedi. It helps it succeed as a film.

Spoilers for The Last Jedi ahead.

Blockbuster franchises tend to use each major film as an incremental building block toward a larger goal. Abrams anticipates this in The Force Awakens, sprinkling in seeds for other stories meant to fill in the gaps. But there are downsides to this approach. Marvel’s Cinematic Universe sometimes feels as though its individual entries are just placeholders or advertisements for the next big team-up film, or the next big series revelation. And as franchises get deep into their planned stories, the staleness of a familiar story can dampen the audience’s excitement.

In a post-release interview with Entertainment Weekly, Johnson explained his attitude toward setting up a new story and going against expectations. “The easiest thing for Rey and the audience to hear is, ‘Oh yeah, you’re so-and-so’s daughter.’ That would be wish fulfillment and instantly hand her a place in this story on a silver platter,” he said.

By focusing the film extensively on Luke’s redemption, Johnson cuts against many of the expectations Abrams — and the entire history of Star Wars — prepared for fans. The now-non-canon Expanded Universe frequently put the Skywalker and Solo families in the center of major galaxy events, and Abrams seemed to tee up a similar dynamic: teasing Rey’s mysterious past, giving her visions when she touched Luke Skywalker’s long-lost lightsaber, setting up the expectation that she was part of a larger familiar story. Fans eagerly dug into those hints.

But as improbable as it seems, Lucasfilm seems to have given Johnson carte blanche to decide who Rey’s parents were, how to address Snoke, and what story to tell in general. He appears to have used this freedom to avoid letting Abrams paint him into a corner. Johnson pushes back against many Star Wars conventions: he downplays the importance of Rey’s heritage, ignores Snoke’s backstory, reintroduces a broken Luke Skywalker, and pits Poe Dameron against a military leader who doesn’t reward the usual rash tactics of the Rebellion’s pilots.

Instead, Johnson uses one of The Force Awakens’ biggest lingering questions to inform his story. How did Luke Skywalker end up on an island on a remote planet? The answer is tragic: even as a hero, Luke can make horrifying decisions. In a moment of weakness, he considered murdering his own nephew, which sets up an intriguing redemption arc at the movie’s core.

The entire story bucks fan expectations. Tasked with relaunching the franchise to an audience wary of Star Wars films after the prequel trilogy, Abrams played it safe by returning to the franchise’s basic building blocks. But if Star Wars films keep borrowing from the past, they’ll inevitably take on that franchise staleness, that sense of spinning wheels. Disney has plans to continue the franchise for at least another decade, which means the story needs new directions. If Johnson had played by Abrams’ rules (which are ultimately George Lucas’ rules), fans might have received The Last Jedi more warmly — at least, the fans who’ve been most vocal in their complaints. But then the film would have been a placeholder in the middle of a trilogy, rather than pushing Star Wars’ boundaries.

Disney has already experimented with giving directors full control of Marvel movies. Look at this year’s Thor: Ragnarok, or the first Guardians of the Galaxy, both of which let idiosyncratic directors play with well-established worlds in unexpected ways. Johnson follows suit. At times, it feels as though he’s gently trolling his audience by setting up a scene we’ve seen before, then pulling back to play out the scene in another way, such as disguising an iron as a spaceship, having Leia shoot Poe after his short-lived mutiny, or setting up a dramatic reveal of Rey’s parents, only to show her looking at herself in a mirror.

By ignoring the obvious paths Abrams set up, Johnson genuinely surprised his viewers, for better or worse. And fighting the most obvious and familiar story paths is the right direction for the franchise. He decouples Rey’s story from that of the Skywalker saga, concludes Luke’s story, and ultimately puts the fate of the galaxy into the hands of lost kids like Rey. By challenging expectations for the film, Johnson shakes things up and takes the franchise’s iconic heroes out of a safe place to look at them in a new light. That new examination and reinvention is exactly what Star Wars and Disney need. Fans may fight it at first, as they so often fight anything outside the scope of what made them love a franchise in the first place. But the only way to bring Star Wars back to that place of infinite possibilities is to defy expectations. Viewers may quibble about Johnson’s specific answers to Abrams’ questions, but they shouldn’t be so quick to resent him answering them in ways they didn’t expect.