Projection mapping is used extensively across themed entertainment, attractions and immersive experiences. Traditionally, though, the surface being mapped stays still.
Once that surface starts moving, things get more complicated. The media system needs to know where the object is in 3D space and continually adjust the projected image so that it stays in the right place as the object moves and changes orientation. It’s an area we’ve been developing at 7thSense for several years.
One of our early demos explored real-time projection mapping onto freely moving 3D objects. High-speed tracking cameras supplied live position and rotation data to the media system, which updated a corresponding 3D model and recalculated the projection from each projector’s viewpoint.
The objects could be picked up, moved, turned and spun without following a predefined path, while the content remained mapped to their surfaces.
Behind this is a mix of 3D mesh mapping, accurate calibration and fast processing. Media is mapped to a digital model of the physical object using a UV map. As tracking data changes the position and orientation of that model, the media server recalculates where the correct pixels need to appear in each projector output. In our original demonstration, this was happening at up to 240 updates per second.
Importantly, the approach isn’t tied to optical tracking. Position data can come from motion capture, IR, LiDAR, mechanical encoders or other systems capable of providing accurate positional information.
That has allowed us to keep developing the technology and look at how it can be applied in more practical ways.
One recent example brings dynamic projection mapping together with animatronics. Instead of using tracking cameras, position data comes directly from the animatronic control system. A 7thSense media server uses that information to drive mapped content across two projectors, continually responding as the physical figure moves.
The development brings together P&P Projects’ themed entertainment and animatronic expertise, 7thSense media technology, projection alignment and calibration from Scalable Display Technologies, ThemedMotion’s custom control system and animatronic technology, and creative technical direction from CurtainUp.
Projection doesn’t have to sit on a static scenic surface, and a physical character doesn’t necessarily have to retain the same visual appearance throughout an experience. Digital content can become part of the physical object itself - changing its appearance, adding effects or responding to movement while remaining aligned with the figure.
The move from freely tracked objects to controlled animatronic systems shows how dynamic projection mapping can start to become a practical creative tool rather than simply a technology demonstration.