J.J.Abrams gave a TED Talk a dozen years ago where he talked about The Mystery Box and its influence on his storytelling. Everything from Lost to, arguably, The Force Awakens has been driven by this modern day version of the mcguffin. As a result, the great stories of our time have been shrouded in mystery and based upon puzzle solving and cliff hangers, rather than laying out the details and watching how characters respond to them. I am not saying this is good or bad. I am saying it has created a new form of fandom that thrives on trying to "figure stuff out."
The Twilight Zone and Rod Serling were really the first place that this form of storytelling hit the mainstream. Fantastic stories that ended with twists or irony at first shocked the viewers but then slowly turned the tales into a game: what is the twist? What are the clues? And while this works great for 30-60 minutes episodes with a beginning and end, it has led to the emergence of a genre of Mystery Box stories that encourage the fan to theorize about what it all means, what were the breadcrumbs and where is it all going to end.
As a marketer of entertainment, you could not ask for a better situation. Hordes of fans become engrossed in your product. They promote it constantly through blogs, podcasts, discussion groups and social media as they share their insights and discoveries. Shared experience becomes the lifeblood of the story and the anticipation of the next clue in the form of the next episode, film or book grows far beyond what any commercial buy ever could.
What invariably happens to the fanbase is that they begin to develop their own ways that the story could go. Then it evolves to where the story should go and this is where fan theories become a bad thing. If a fan theory catches hold and a large mass of the base buys into it, the story is backed into a no-win situation. Game of Thrones as of The Battle of Winterfell found itself there. Spoilers ahead for GoT...
For years the fans have built theories about who would kill the Night King. They looked at the written word of Mr. Martin for evidence that it would be this character or that. When the character who did issue the deathblow was revealed, it was met with both joy and resentment. It didn't line up with the most popular theories and, in some cases, outright contradicted what some fans were sure would happen. And while a large percentage of the fans welcomed the unexpected, within the first week of it's airing, a groundswell of anger toward the show, the show runners and even the character herself. On the other hand, had it gone exactly the way some had predicted, they may have felt cheated or like the show took the easy way out.
This is what I've seen happening in the Star Wars universe as well. JJ Abrams laid bread crumbs all over The Force Awakens and the audience was almost forced to ask questions of the main character's parentage and of the origin of the big mysterious bad guy. Fan theories have sprung up ranging from the hero being a relative of virtually every character who ever got screen time to the villain being much more important than he turned out to be. When Rian Johnson dared to close the mystery box without feeding the fan theories, regardless of whatever else his film added to the lore, many of the devout theorists had a difficult time with his answers to the Mystery Box.
While we don't yet know how Episode IX will play out and if JJ will revisit the mysteries he put in the box, we know that fans have created more theories - or, even more problematic, hypotheses about how the last movie will negate the answers received in the previous film so that their beliefs will be confirmed.
So is fan speculation and theory good or bad? There is no clear answer. It is good when it generates passion from the fans. It is bad when it creates intense disappointment. Only one of the original six Star Wars movies had anything resembling a Mystery Box, and, thus, I can't completely exclude them from this line of storytelling pre-TFA. One could actually argue that, if the Palpatine/Sidious connection wasn't so clear as a result of the pre-history of the prequels, it too would be a Mystery Box story. However, it wasn't.
But now, Star Wars is a Mystery Box story. Even if Rian tried to seal up the box, it is still open. And add to that the cackling of Sidious in the trailer for IX, amping up fan theories for the next several months. As a result, the fan theories will be good for the marketing of the December 2019 film. They may not be good if theorists get too connected to the story they create in their mind and become disappointed with the story Lucasfilm and Disney decide to tell.
So, here's my unsolicited advice: try not to get too attached to a set of theories. Use them to build excitement, but trust in the filmmakers to deliver a great experience and remember, this is their story to tell and yours to enjoy.