Voices of VR

Ant Williams is a freediving athlete featured in the third episode of The Adventure Series titled "Ice Dive" that's co-produced by Apple and Atlantic Studios (formerly Atlantic Productions). In the episode, Williams attempts to swim a world record distance 182 meters under ice, and I wanted to get some additional behind-the-scenes context on his experience as well as what it was like to have the most intense month of his life condensed down into a 15-minute Apple Immersive Video. Williams is a sports psychologist who wanted to put his theories into practice by taking what he calls "positive, calculated risk-taking challenges" that allow him to deal with overwhelming anxiety, and overcome his fears, uncertainty, and self-doubt. I also wanted to get some additional context on the production of the episode as Apple has otherwise been pretty tight-lipped about the series, launched with which launched with "Highlining" on the same day as the Apple Vision Pro launch on February 2, 2024. Apple Immersive Video is a different format than spatial video. Apple says "spatial videos are captured in 1080p at 30 frames per second in standard dynamic range," and these are what can be captured by either an iPhone or Apple Vision Pro, and they are displayed in a windowed frame where you see the stereoscopic effects. Apple describes Apple Immersive Video as "a remarkable storytelling format that leverages 3D video recorded in 8K with a 180-degree field of view and Spatial Audio to transport viewers to the center of the action." Apple Immersive Video is much closer to what we've seen from the XR industry and VR 180 filmmakers from the past decade, and Apple's technology is likely derived from their 2020 acquisition of NextVR. NextVR focused on live stereoscopic broadcasts of sports events on VR headsets starting with the Samsung Gear VR and Oculus Rift in 2014. A lot of the technical specifications of the Apple Immersive Video format have not been officially confirmed by Apple, but there are a couple of breadcrumbs that give us some more details. Thanks to iFixIt's breakdown of the Apple Vision Pro on February 7, 2024, then we know the microOLED display size is reported as "the lit area totals 3660 px by 3200 px." 360 Labs' Mike Rowell wrote a post on March 19, 2024 saying, "Apple Vision Pro’s screens are a whopping 3660 x 3200 pixels per eye. Although they haven’t made any official claims as to the FOV of the headset, 3rd party developers claim that it looks to be around 100° horizontal. With each screen having 3,660 horizontal pixels, this would mean that a 180° immersive experience would need about 6,000 x 6,000 pixels per eye to saturate the display. Apple’s own immersive experiences have been reported at being 4320x4320 per eye at 90fps and in HDR10." The reporting of Apple's immersive experiences was detailed by Mike Swanson, who announced a spatial video tool on March 7, 2024 that leverages the Apple's AVFoundation to properly encode video into the "multiview extensions of the HEVC codec, known as MV-HEVC" format. Swanson says in his post, "I receive multiple messages and files every day from people who are trying to find the limits of what the Apple Vision Pro is capable of playing. You can start with the 4320×4320 per-eye 90fps content that Apple is producing and go up from there. I’ve personally played up to “12K” (11520×5760) per eye 360-degree stereo video at 30fps." Another clue can be found in the Blackmagic URSA Cine Immersive camera that was announced on June 10, 2024, which says, "The sensor delivers 8160 x 7200 resolution per eye with pixel level synchronization and an incredible 16 stops of dynamic range, so cinematographers can shoot 90fps stereoscopic 3D immersive cinema content to a single file." Incidentally, Currents director Jake Oleson told me that he used Sw...

CURRENTS is a stunning debut from Jake Oleson as it pushes the grammar of 180 filmmaking to new heights. Oleson comes from the world of advertising and music videos, and he went through a rigorous pre-production process where he had actually already edited the entire piece before shooting any production footage by using still stereoscopic test photos and videos. The insights from this pre-production animatic are evident, as there are some of the most stylized movements I’ve seen, and he managed to not trigger any of the common motion sickness triggers. There are some really powerful cuts that seamlessly juxtapose different environmental contexts for dramatic effect. CURRENTS tells the story of a young woman who leaves her Vietnam rural countryside home to the city seeking more economic opportunity. Aside from a short exchange with her mother at the beginning, the rest of the entire story is told through spatial storytelling, driving electronic music that was also composed by Oleson, and a shader-filled point-cloud scene with some animated motion capture. It's really quite a beautiful piece that gave me a glimpse of the next phase of cinematic immersive filmmaking. It's also worth mentioning CURRENTS in the context of other musical experiences since Oleson is both a musician who has a background in music video production. The music was specifically composed to serve the narrative in this instance, but it will be interesting to see more and more immersive music videos start to crop up now that the Apple Immersive video format is starting to get licensed out to folks like Vimeo, who commissioned this piece. Apple's push for 180-degree immersive video has brought up some broader discussions about the merits and downsides to 360-degree films. From a creator's point of view, 180-degree filmmaking is a lot easier to do logistically as it is closer to existing production pipelines, which is brilliantly demonstrated in CURRENTS. It's also worth noting that CURRENTS has a point-cloud scene that shows up in the middle of their cinematic 180-degree immersive video that represents a key turning point in the story and journey of the main protagonist. There's also some really poetic shaders that are translating the static scenes into even more of a dynamic visual representation of the business and chaos of the city. It has quite a narrative impact when juxtaposed against the photorealistic footage and custom-written and driving musical score. Even though the Apple Vision Pro has been out for over a year now, SXSW is still an opportunity for folks to check out the Apple Vision Pro for the first time. It was probably easiest to check out on CURRENTS who had six different headsets and was using the Sandwich Vision Theater app in Kiosk mode to seamlessly jump into the experience bypassing the time-consuming eye tracking calibration step. I was also particularly interested in Olson's production pipeline, the camera gear and kit costing less than $5k (Canon EOS R5 Camera Body, RF5.2mm F2.8 L Dual Fisheye, & DJI Ronin-SC Gimbal Stabilizer), Topaz Video AI Enhancer upscaling tools to go from 8k to 16k, and all of the due diligence that he did to understand and mitigate motion sickness triggers. The end result feels like a music video that tells a timeless story of the tension between rural and urban homes and what is lost in the pursuit of career opportunities. Below are links to of all of my SXSW 2025 coverage by the Voices of VR podcast. And be sure to check out this Linked article: Recap of All SXSW XR Experiences Including Voices of VR Podcast Interviews with Every Artist. SXSW XR Experience Competition [Overview of all of the SXSW 2025 XR Experience Program] - #1529: SXSW 2025 XR Experience S...

OTO'S PLANET was being exhibited on the Quest at SXSW, but I had a chance to check out the Apple Pro Version ahead of the SXSW festival and I felt like I much preferred it. OTO'S PLANET picked up the second place prize at Venice Immersive, but I missed being able to interview the director Gwenael Francois there. I did manage to catch up with Francois at SXSW to talk more about the development of this beautiful interactive narrative that uses table top-scale animation, and requires the user to rotate a tiny planet around in order to follow the story of Oto, his pet, and the story of a colonizing intruder who arrives. I found that the interactions were much smoother and immersive on the Apple Vision Pro than the Quest, and the higher resolution also popped a lot more. The Apple Vision Pro version defaults to mixed reality while there is an opportunity to see it in fully immersive VR on the Quest. I normally would prefer the Quest version, but having the better quality mixed reality cameras and overall much higher resolution in the Apple Vision Pro. Apple also featured Oto's Planet, and DPT CEO Nicholas Roy shares in this interview that they've actually received more purchases on Apple devices than Meta's ecosystem. Meta’s storefront has been flooded by Meta's own first-party Horizon world as well as App Lab apps, and so it’s very telling and reflective of the current state of distribution to have more sales for the Apple Vision Pro than Meta Quest. It is promising to see Apple get more serious at immersive storytelling as there was a session titled "Create Interactive Stories for Apple Vision Pro" on Tuesday morning featuring three representatives from Apple. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

Another Apple Vision Pro piece at SXSW 2025 was COSMOS IN FOCUS by Atlantic Studios (recently rebranded from Atlantic Productions). This was a short and sweet educational experience that allowed you to explore the imagery of the James Webb telescope, but it's both spatially contextualized in the night sky planetarium style. It is also fused together with other deep space satellite images at different scales of resolution creating this really cool zoom effect that's like a mash-up of the film POWERS OF TEN with Google Maps tiles, but in the context of space imagery. It's also an experience that demonstrates the power of collaborating with scientific researchers and subject matter experts as they're able to translate contextless 2D jpgs into an incredibly powerful immersive experience that not only preserves the original context, but leverages the immersive medium to provide many more learning opportunities. Many immersive stories on the festival circuit will shy away from more explicit didactic content or learning experiences, but this piece leans into it by reducing the choices a user can make streamlining the user journey. I also really enjoyed my conversation with Aditi Rajagopal, who led this project after transitioning from R&D and AR storytelling at Meta to immersive storytelling at Atlantic Studios. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

There was an projection-based immersive experience called ORIGINS - LIFE'S EPIC JOURNEY that was originally formatted for a digital art museum in Germany called Kunstkraftwerk Leipzig, but it was translated into a three-channel video installation that was showing in a small conference room at SXSW. I only saw the video version at SXSW, and so I have not experienced the piece within its originally-intended, fully-immersive context. But it's collaboration between digital artists Markos Kay, Martin Salfity, Thomas Vanz, Davy Evans, Gokhan Tekin, Susi Sie, Teun van der Zalm and Roman Hill to tell the story of the evolution of life from the smallest quantum to biological to galactic scales, each using different visual techniques to create these 25 different chapters. I had a chance to chat with Paolo Loeffler to get a bit more context on the evolution of Kunstkraftwerk Leipzig, and how ORIGINS was developed and commissioned to play in these projection-based immersive exhibition spaces. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

FLY TO YOU tells the story of Kang Songjeol who was separated from her childhood friends in the aftermath of the 1950 split between North and South Korea. Oral history interviews were captured in 360 video, and her childhood memories were recreated with point scans derived from Leici BLK360 LiDAR scans combined with Azure Kinect volumetric captures. The experience manages to virtually fly over the restricted demilitarized zone into North Korea, which is something that is illegal from them to do physically but serves as a symbolic gesture towards reunification. There are still thousands of families hoping for an opportunity to be reunited, and there's a beautiful spatial memorial that brings home that point at the end of this piece. I had a chance to unpack it all with co-directors Sngmoon Lee and Youngyun Song This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

SYMPHONI is a brand new hand-tracked rhythm game that was inspired by the mechanics of conducting a symphony. It was directed by Ingram Mao and developed by a number of undergrads and graduates of the USC Games program. This game also goes from mixed reality into fully immersive VR as you progress through each of the stages, but it's a really fresh take within the rhythm game genre as I found myself playing for over an hour on my first playthroughs. Mao comes from an interior design and architecture background, but also thinks very deeply about the game mechanics and innumerable tradeoffs to be navigated through the game design process, and so it was a real pleasure to have a chance to dig into the weeds of the process of developing this experience. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

ECHOVISION is the latest experience from multi-disciplinary artist Jiabao Li that has three major parts. The first part is a mobile-phone based mixed reality experience that does a metaphoric translation of echolocation by using LiDAR to detect your immediate physical surroundings, and then reveals it with a rippling shader that is voice activated. The second part is a video that poetically visualizes different bat calls that have been identified by AI into different contextual domains, and there's a really awesome haptic couch experience that goes along with it. The last part is a field trip to the Congress Street bridge to watch the emergence of hundreds of thousands of bats come out at night as they go to eat. I caught up Li to hear more about all of her interdisciplinary and interspecies collaborations on this piece. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

There were a number of pieces at SXSW that were centered around embodied interactions including HONEY FUNGUS, which is a series of interactive embodied experiments telling a broader story of cultivating intimacy with ecology. This piece leans more into embodied dream logic rather than clearly articulating a narrative journey, and I had a fascinating conversation with Jonah King who decoded the underlying symbolism. King’s first step of his creative process was to go down a rabbit hole researching queer ecology and the latest research on fungi, and he wrote an essay titled "Unfathomable Intimacies" that lays out his original inspirations. One thing that really stuck with me from the experience was this intriguing AI mash-up of Smithsonian Field Research and amateur erotica designed in order to cultivate a new form of ecological intimacy with the world around us. I appreciated this experience a lot more after having a chance to learn more about additional context provided on the website as well as insights gained from my conversation with King. To me the dream logic in this piece leans a little bit more into personal symbols that need some decoding rather than more universal archetypes that are easier to project the intended meaning upon. But I always love learning more about Fungi since they represent so many paradigm-shattering insights. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

1906 ATLANTA RACE MASSACRE is a phone-based AR experience directed by Nonny de la Peña that digs into the forgotten history of the violent attacks on the black community by white mobs incited by race-baiting politicians and newspapers. The arc of the story is told through the lens of black journalist Max Barber, who was chronicling the events as they were unfolding. We see a series of volumetrically captured monologues by Barber (played by actor Bryonn Bain). As the massacre unfolds, we travel to different key locations around Atlanta represented by spatial facades and Quill illustrations. There's also a series of 2D AI animations ranging from white politicians in newspapers to black activists who are in picture frames. I spoke with de la Peña around some of the constraints and limitations of working with mobile phones that are not completely optimized for XR experiences like this just yet, but it's much more accessible form factor that the executive producer of the National Center for Civil and Human Rights is interested getting into schools. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality