How the LEGO Group Blends the Physical and Digital to Create New Forms of Play

December 16, 2020 0 By Brian Crecente

The LEGO Group’s 25-year-history of video games wasn’t just a journey about learning how to transform a physical playset into a digital one, it also involved a deep examination of how these two forms of play intersected and how that co-mingling created a new form of play.

Today, the company calls this hybrid interaction “fluid play”, and it is perhaps best modeled by the recent release of the LEGO® Super Mario theme, a set of playsets built around a chunky, interactive Mario with a smiling face, coy eyes, and the ability to sense how and where with which it is being played.

Fluid play, though, can trace its roots back to early theory and prototypes created in the late 90s and the advent of the smart toy craze.

Arguably, the LEGO Group’s partnership with MIT and the resulting creation of MINDSTORMS® was what accelerated the company’s interest in injecting technology into its bricks. Both MINDSTORMS and the more game-like CyberMaster were released in 1998. A year later, a company called Zowie Entertainment released Ellie’s Enchanted Garden, a toy that deftly merged playing with plastic toys with interacting with a computer game. In 2000, the LEGO Group purchased the company and tech and brought on the development team to start work on a technology platform it called the KidPad.

The concept was to create a playset based on the popular DUPLO® Circus theme. The set used a plastic tray with an antenna array built into it that could sense a number of plastic figures that each had RF chips in them. Moving characters around on the tray would trigger different animations and interactions on the computer screen via a DUPLO Circus game.

“The goal was to extend that primary physical experience through the capabilities that the digital would provide, and try to do it in a very seamless way,” said John Sakalowsky, one of the original developers of the KidPad and the DUPLO Circus experience.

Playtesting, though, showed early signs of what would become a major roadblock in achieving this. Children who boisterously played with the plastic toys during tests, went sort of slack jawed once the computer game was brought into the mix.

“My first thought was like, ‘Oh crap, how is this gonna work?’,” Sakalowsky said.

While the team continued to work toward a solution, KidPad and DUPLO Circus were eventually shelved, not because the team wasn’t making headway, but because the LEGO Group – several years now into a decline – shuttered the project and let the entire team go.

Over the ensuing decade, the company reinvested in its core products and then once more began expanding into new and intriguing ideas. LEGO.com took a big leap in design and interactivity in 2001, and the company released the massive hit LEGO Star Wars: The Videogame in 2005. By 2009, the company was once more exploring some of the concepts first examined with smart toys like the KidPad, but now they were viewed as part of a “one reality” experience.

Where smart toys were about the injection of technology into traditional toys, one reality was about finding a balanced integration of virtual and physical play. And it was out of this concept, that the LEGO team started working on new hybrid ideas like toy-meets-art-and-coding-experiments of the LEGO WeDo in 2009 and even board games built entirely out of LEGO blocks.

In 2011, the LEGO Group released Life of George, a fun little box set of LEGO pieces that asked players to create flat images shown on their iPhone as quickly as possible and then check how good they were with an app.

The idea started as a puzzle game that used a computer’s webcam, but developer Paal Smith-Meyer and his team quickly realized they needed to move away from the desktop computer to help engage more children. It was Apple, he said, which pointed out that the game needed more personality, and so George, a flat, blond-haired, globe-trotting hero was born.

The creation of that game and Fusion – which would later expand on that idea to include full 3D constructed sets – led to a lot of internal discussions which often became philosophical.

“For some the main drivers was to see how can the digitally really inspire kids to build more and play more physically,” Smith-Meyer said. “So we’re looking at this kind of play loop where you created something and then use it in a digital space and that play would then inspire you to go back, physically, and play more. For others, it was more kind of maybe looking at it as a relevance tool to sell more of the product line.”

Life of George kicked off a renaissance of sorts in one reality design, with the LEGO Group releasing Fusion in 2014 and the toys-to-life hit LEGO Dimensions in 2015. This form of play also found its way into things like the LEGO House, where visitors were encouraged to create flat fish out of LEGO pieces which were then scanned and set loose into a massive digital aquarium. It also influenced the design of Nexo Knights, which made use of shields that could be scanned with a smartphone to unlock Nexo Powers for use in a game on the device.

It was this blossoming of hybrid play that helped the company once more reexamine how it approached the co-mingling of bricks and bits. The company referred to this new approach not as one reality, but fluid play, referencing a desire to create these sorts of toys that allowed players to fluidly transition between the physical and digital seamlessly.

The first major release using this approach was 2019’s Hidden Side, a ghost-hunting theme set that included with it the ability to inject a built model into a virtual world on a mobile phone using augmented reality. Builders would then explore their real-world creations through the lens of the phone, looking for digital ghosts to combat and capture. The approach added a layer of context and play to the sets, something the LEGO Group had never really done before in this way.

“We fundamentally changed what a LEGO experience was,” said Sam Coates, who worked on the project. And that brought with it a major challenge. “We had to take kids with us and they didn’t automatically come with us. We’re still learning how to explain interactive sets and how to explain the play potential.”

The latest exploration in fluid play is LEGO Super Mario, which makes use of a “smart” LEGO Mario that can track and interact with the courses you build for him out of bricks. The LEGO Group’s Michael Vanting believes it is the closest the company has come to nailing physical-digital interaction.

“It inspires you to change your course and rebuilt it,” he said. “And it has this LEGO DNA that the more sets you get, the more value you have, the more variety you can do in your world, where the physical suddenly takes its own role in replicating things from the digital world, but without trying to compete in a way.”

It’s easy for the concept of game-enabled toys or fluid play to feel a bit abstract, notes documentarian and co-host of Bits N’ Bricks Ethan Vincent, but delving into its history makes it clear how vital fluid play is to the LEGO Group’s future innovations.

“For me – in a way – it has less to do with some of the past attempts in the genre but the ability of being able to look into the future and image toys that are more powerful, more sophisticated and truly bring the physical into the digital in new and exciting ways,” he said.

“The LEGO Group’s long-standing, deep interest in all forms of play is a big part of why it continues to dominate the toy industry,” said journalist and Bits N’ Bricks co-host Brian Crecente, and why it will continue to play a big role in how well it does in the game industry.

“While Hidden Side still doesn’t get everything right in its merging of physical and digital, it moves that effort in the right direction,” he said. “LEGO Super Mario, on the other hand, finds a way to take the digital and make it physical while still maintaining the unique aspects of both forms of play.

Mario may not be the embodiment of fluid play, but it is a major innovative step forward in this emerging form of hybrid interaction.”

This article originally ran on LEGO.com as a summary of episode one of the weekly Bits N’ Bricks podcast, which you can listen to here.