Designing for Virtual Reality Experiences

I experimented with a workflow that combines Gravity Sketch, mixed reality, and Runway's Gen-3 video-to-video AI and got some impressive results, here is what I did: 🚀 Step 1: Using Gravity Sketch in VR, I designed stasis tubes with humanoid figures inside. I placed these models throughout my hallway, integrating them into the real space, using mixed reality mode on my Meta Quest 3 headset. 🎥 Step 2: I filmed myself walking through this mixed reality set, holding a 3D object, capturing my real environment with the 3D models layered in. This gave a first-person view of the scene, as if I were navigating through an alien ship. 🧪 Step 3: Finally, I ran the footage through Runway’s Gen-3 video-to-video AI, using prompts to transform the scene into a space marine navigating an alien ship, complete with eerie stasis tubes and ambient sound effects to drive the atmosphere home. A fast, intuitive way to pre-visualize complex scenes that would otherwise take much longer to design and film traditionally. What this means for creative workflows: 🔹Advanced Storyboarding: With mixed reality, you can set up rough models and get a realistic sense of scale and positioning. You can actually walk through you scene, interacting with it and capturing raw footage directly. 🔹 Quick Pre-Visualization: Using video-to-video genAI, this rough footage can quickly be transformed into something more. It’s a great way to experiment with looks, check in with your client vision, and even lighting before diving into final production. 🔹 Future-Ready Workflows: As video-to-video AI improves, this workflow won’t just be for pre-viz. We’re looking at a future where you could create final-quality outputs straight from this setup, acting out scenes in a mixed reality environment while the AI enhances and polishes everything in real time. Moving towards final generated outputs vs rendered. This opens up a lot of possibilities. You could set up a mixed reality scene, interact with it, and create an entire short film without needing a massive crew or extensive post-production. For now, it’s a powerful way to prototype, storyboard, and explore creative concepts quickly and intuitively. ❓Curious about how mixed reality and AI could transform your creative process? Let’s connect-I’d love to share more insights and explore how these tools can push your projects to the next level.

In XR, a bad user experience can break immersion in seconds. 👀" Let’s be honest. When you’re in an immersive XR experience, everything needs to feel natural. The second something feels off whether it’s awkward controls, confusing navigation, or a glitchy interface the magic is gone. 💨 That’s why UX design is critical in XR. It’s not just about making things look good it’s about making them feel right. When you’re designing for XR, you’re not just dealing with flat screens anymore. You’re designing entire worlds that users need to move through, interact with, and believe in. But here’s the catch: If the experience isn’t seamless, users get frustrated and disengage quickly. And once they’re out of the experience? It’s hard to pull them back in. Why is UX so important in XR? 1. Immersion is fragile In XR, you’re trying to create a sense of presence that feeling where users forget they’re wearing a headset or holding a device. But one clunky interaction or poorly designed interface can snap them out of that immersion instantly. Imagine trying to pick up an object in VR, but your hand glitches through it… frustrating, right? ↳ That’s why intuitive interactions are key. 2. Users need intuitive controls In traditional apps, users click buttons or swipe screens. In XR? They might be using hand gestures, voice commands, or even eye-tracking! If these interactions don’t feel natural or intuitive, users will struggle—and that breaks the experience. Think about it: Would you want to wave your hands around awkwardly just to open a menu? 🤔 3. Comfort and safety matter Ever heard of VR motion sickness? It happens when there’s a disconnect between what users see and how their body feels. Poor UX design can make this worse with jerky movements or disorienting transitions. A well-designed experience should feel smooth and comfortable no nausea required! 🤢 4. Accessibility for all users Great UX also means making sure everyone can enjoy the experience, regardless of their abilities. This means designing for accessibility—whether that’s adding subtitles for audio cues or ensuring interactions are simple enough for non-tech-savvy users to navigate easily. What can you do to improve UX in XR? Test early and often: Don’t wait until the final stages to test your design! Get real users into your XR environment as soon as possible and collect feedback on what works and what doesn’t. Focus on natural interactions: Mimic real-world actions as much as possible (e.g., grabbing objects with hand gestures). The more intuitive it feels, the better. Prioritize comfort: Ensure smooth transitions between scenes and avoid sudden movements that could cause discomfort. (alot of games published in meta horizon store lacks this basic thing) Make it accessible: Think about how different users will interact with your experience whether they have disabilities or are new to XR technology. What broke the immersion for you? Share your thoughts below! 👇

We put Apple Vision Pro on over 150 heads during React Conf and App.js Conf. Here’s what we learned. 1. The excitement is real. People (mostly developers) are genuinely curious and excited about the device. They want to touch it, wear it, feel it. Even skeptics. It’s too pricey today, but they eagerly try it when presented with the opportunity. 2. The interface is incredibly intuitive.  Most only needed a quick intro to the tapping gesture. People often tried touching virtual elements and they were surprised that one could highlight UI elements by looking and tap with fingers. However, in busy environments, the tap gesture sometimes failed when other hands were nearby. But when it worked, moving and rotating objects in space was the most fun. 3. The experience is indeed immersive.  The virtual environment, with immersion controlled through the Digital Crown (which most users needed to be told), was a universal "wow" moment. Only few experienced the audio: in a quiet room, with an extended display from my Mac and minimal immersion background, sounds like rain, wind, and birds made it feel more real. 4. React is ready for Spatial Computing. We let folks use an app we created with Oskar Kwaśniewski. People were positively surprised it was made with React Native. From the app one could display 3D items in another window, which they could intuitively grab, put where convenient and rotate with their hands, all seamlessly driven from JavaScript libraries like RNGH and Reanimated. As a R&D effort at Callstack, together with Matt Hargett and Oskar Kwaśniewski we've worked on bringing React Native to XR devices, specifically to Vision Pro, since last September. Today it's a viable option for hundreds of thousands of React developers to enter the platform while reusing their skills. Many got hyped to build spatial apps and we can't wait to see them popping in the App Store! 5. Physical comfort is a real issue. The headset is heavy, causing discomfort on cheekbones. Adjusting the strap helped a bit. Some had trouble with eyesight adjustment to flat screens after wearing it. We reported no dizziness or sickness though. I was wearing it for few hours on and off and experienced no neck pain (yet!). 6. We’re optimistic about the future. With developers at both conferences we discussed evolving the device and tech to be more user-friendly. We are also experimenting with easier ways to display immersive content for JS devs. Hope to share findings with Apple engineers at WWDC 2024. At one of React Conf parties, we overheard a group of 7 developers planning to do a ride share to the Apple Store to buy an AVP 😂. Overall, it was a fun experience. Building and using the platform and apps with React was a joy once past some hiccups. And using the device for the first time was like nothing I've ever experienced so far. If you can’t visit an Apple Store in the USA, try our headsets at Callstack's office or at RenderATL conf in June.

⏳ Looking back 20 years, here’s some advice I wish someone had given me when I started grad school designing wearable and assistive technologies: Designing wearable tech isn’t just about biomechanics—it’s about comfort. Too often, as scientists and engineers, we overly focus on performance metrics, technical specs, or biomechanical outcomes while overlooking the human experience of wearing a device. Or we simply value novelty, gadgetry, and complexity over practicality, elegance, and usability of solutions. 🎯 But comfort is key. And it's multi-dimensional, especially for technologies like #exoskeletons. If you want to design wearable tech with end users truly in mind, consider: • Physical comfort: fit, weight, pressure distribution • Thermal comfort: heat buildup and sweat retention • Psychosocial comfort: aesthetics, noise, user perception ☝🏽 And don’t forget comfort-adjacent factors that also impact real-world adoption. Things like: • Freedom of movement: bulk and mechanical interference • Ease of use: practicality and cognitive load These are all microcosms of a bigger challenge in wearable technology: balancing engineering performance with the user experience. It’s something I wish I had been much more in touch with early in my academic research career—and something I’m grateful has become an integral part of our R&D over the past decade. 👉🏽 If you work in wearable tech, assistive robotics, or human-centered design—what forms of “comfort” do you think engineers and designers most often overlook?

🕶️ 𝗔𝗽𝗽𝗹𝗲’𝘀 𝗧𝘂𝗻𝗮𝗯𝗹𝗲 𝗟𝗲𝗻𝘀 𝗣𝗮𝘁𝗲𝗻𝘁: 𝘁𝗼𝘄𝗮𝗿𝗱 𝗮𝗹𝗹-𝗱𝗮𝘆 𝗫𝗥 🔭 Apple’s new patent outlines a tunable lens module for smartglasses/HMDs that dynamically adjusts optical power to keep imagery sharp and comfortable across viewing distances. The promise: less eye strain, broader user fit, and fewer prescription inserts. Why it matters: varifocal-style, software-driven optics can tackle the long-standing vergence–accommodation gap—bringing us closer to lightweight, all-day wearables with human-centric visual comfort. The systems view: per-user vision profiles, rapid auto-calibration, and rendering pipelines that react to optical state. Done right, this reduces SKU complexity and unlocks enterprise wins in training, field service, and hands-free workflows. Challenges remain—latency, calibration drift, battery/thermals, and cost—but the direction is clear: smarter optics guided by smarter software. What’s your take—will tunable lenses be the unlock for mass-market XR? More information here: https://lnkd.in/dtcV_8Zk #XR #SmartGlasses #HMD #Optics #Varifocal #DisplayTech #AI #AR #VR #Customertimes #FutureOfWork #DigitalTransformation

3 fire departments in 3 states just walked through the same fire scene at the same time. California. Texas. New York. All standing inside the same 3D model, seeing the same details, watching each other's personas move through the space in real time. Nobody got on a plane. Nobody scheduled a site visit. Nobody squinted at a screenshot on a Zoom call. This is #Enterprise #VR by SkyeBrowse. We put the largest city and county fire personnel from 3 states into a single shared environment built from actual large area maps of a hill fire and a railroad fire. Every person could look around, move through the model, and see exactly where the others were standing. If everyone can see the same scene, collaboration gets faster. If collaboration gets faster, decisions get better. If decisions get better, scene documentation stops being something you review after the fact and starts being something you work inside together. Not just capture. Shared presence. Same model. Same moment. Thousands of miles apart. #VR #virtualreality #publicsafety #firedepartment #3dmodeling #realitycapture #videogrammetry #dronemapping

🤢 There was a time in #VR — a long, long time ago in an immersive world far, far away… :) — when one of the standard attractions at corporate events was a VR rollercoaster film. You would put on the headset and feel like you were riding a high-speed rollercoaster. To this day, I still meet people who remember it as a terrible experience. User age and motion sickness in VR remain one of the reasons why companies are still cautious about adopting immersive technologies. And the same argument appears again and again: 🔴 Older employees will struggle more with VR. But… recent research is starting to show a much more nuanced picture. While reviewing studies published between 2023 and 2026, I noticed an interesting pattern: Susceptibility to VR discomfort is often linked more to a lack of familiarity with immersive environments than to the user’s age itself. And that is a very important distinction. Because when we think about motion sickness, we usually focus on hardware: ➡︎ resolution, ➡︎ FPS (performance parameters affecting visual fluidity) ➡︎ latency, ➡︎ and of course overall headset quality. 💎 But in reality, major factors also include: ✔︎ how the user is guided through the experience, ✔︎ the pace of adaptation to virtual space, ✔︎ movement design inside the application, ✔︎ predictability of interactions, ✔︎ the user’s sense of control. In practice, this means poorly designed VR can exhaust even a young gamer. And a well-designed immersive environment can be comfortable even for users aged 50+ with no previous headset experience. This is the moment where VR stops being a “gadget” and starts becoming a mature design tool. After years of XR deployments, one thing is becoming very clear: companies are gradually shifting from being impressed by the technology itself to asking a much more important question: 👉 how do we make users feel natural and confident inside immersive environments? Because ultimately, the real metric is not the “wow effect.” What matters is: 💎 whether employees are willing to return to VR training a second time, 💎 whether users feel in control, 💎 whether the technology creates fatigue, 📈 and whether the experience genuinely supports a business objective. This is where the next stage of VR will evolve: not only becoming more realistic, but above all becoming more human-centered. My sources: “The Effect of Age Introduced to Virtual Reality on Susceptibility to Motion Sickness” (arXiv, 2024) “Effects of user factors on user experience in virtual reality” (Quality and User Experience, 2023) #AR #XR #VirtualReality #EnterpriseVR #UserExperience #VRtraining #ConnectedRealities

This project gave me the chance to collaborate with Jayse Hansen and Jeff Hansberger from #Øffgrid to design a VR simulation system that recreates real-world mission scenarios for soldiers. Our goal was simple: build a training experience that feels immersive, effective, and engaging — one that helps users learn faster while staying fully focused in the moment. We designed each interface to make critical information easy to understand at a glance, from risk zone alerts to interactive mission tracking. With real-time feedback, soldiers can quickly adjust tactics, respond under pressure, and make better decisions in the field. By combining the power of Apple Vision Pro with a human-centered design approach, we created a training environment that is both safe and highly realistic — helping strengthen performance, readiness, and confidence. This is the kind of work that shows how immersive technology can reshape military training, moving preparation toward smarter, safer, and more effective virtual experiences. What excites me most is the potential of VR and spatial computing to transform how people train for high-stakes situations. Where else do you think this kind of technology could make a real impact?

As the CEO of a company inventing tech products all the time, the most important question I ask myself when building an idea is: “Can an undertrained person use this tool in a busy, time-pressured moment on a Tuesday afternoon?” Millions of conversations with clinicians about their experiences with VR tools all come down to one thing: “If I need to be fast, I will just not use it.” I spend a lot of time talking with clinicians and Child Life Specialists about their daily routines. Even though they work in completely different hospitals, their visions are quite aligned. VR headsets not intuitive by itself. They often feel that using it means losing time with a patient. Some even see it as an inconvenience rather than an advantage. This became a major concern of mine, and I decided to put an end to it. When healthcare professionals are asked to use XR with children, the real complication is how using a tech product impacts their confidence and credibility. If a clinician feels unsure about starting, controlling, or adapting the session, the technology becomes a barrier instead of support. VR can be powerful for reducing stress, but only when it fits naturally into the workflow. That is where the tablet layer becomes so important in our system. The VR headset is what the child sees, but the tablet is what gives the clinician control, structure, and confidence. With a tablet-based application, the healthcare professional can select the right experience, launch it quickly, monitor the session, and stay in charge without needing to struggle with complicated controls during a busy appointment. This changes the experience from "tinkering with XR" into using XR with purpose. Instead of handing over a device and hoping everything works, the clinician can guide the process smoothly and focus entirely on the child’s comfort. In practice, this makes XR feel like a natural part of care, which is exactly what helps adoption in real clinical settings. Imagine a nurse preparing a child for a needle procedure. Without a simple control system, the nurse may hesitate, lose time, or worry about making a mistake. With the tablet in hand, the nurse can start the VR session in seconds, choose the most suitable child-friendly content, and keep attention on the patient rather than the technology. The result is a calmer child, a more confident clinician, and a smoother procedure overall. In the end, the headset delivers the immersive experience, but the tablet makes the experience usable in healthcare. The real value is not in the XR content alone, it is in giving clinicians a tool they can trust, understand, and use with total confidence when caring for children. See full blog here: https://lnkd.in/gtC4fspQ PS: eager to learn more about XR in clinical care? Join our 5 part Masterclass Series! Send me a DM for the registration link.

𝗪𝗵𝗮𝘁 𝗶𝗳 𝘆𝗼𝘂 𝗰𝗼𝘂𝗹𝗱 𝗲𝗮𝘀𝗶𝗹𝘆 𝘁𝘂𝗿𝗻 𝘆𝗼𝘂𝗿 𝗿𝗲𝗮𝗹 𝘄𝗼𝗿𝗹𝗱 𝗶𝗻𝘁𝗼 𝗮 𝘃𝗶𝗿𝘁𝘂𝗮𝗹 𝘀𝗽𝗮𝗰𝗲𝘀? We've talked about digital twins for years — mostly for factories, supply chains, and smart cities. But what if creating digital replicas was as simple as walking around with your headset for a few minutes? Hyperscape just made spatial capture mainstream. High-quality, immersive digital copies of real-world spaces in minutes. Not complex 3D scanning. Not expensive equipment. Just guided capture and you're done. Industry use cases that get me excited: 🏭 𝗠𝗮𝗻𝘂𝗳𝗮𝗰𝘁𝘂𝗿𝗶𝗻𝗴 & 𝗪𝗮𝗿𝗲𝗵𝗼𝘂𝘀𝗶𝗻𝗴 - Capture production floors for remote training - Plan layout changes in VR before moving equipment - Train safety procedures in exact replicas of work environments 🏥 𝗛𝗲𝗮𝗹𝘁𝗵𝗰𝗮𝗿𝗲 & 𝗠𝗲𝗱𝗶𝗰𝗮𝗹 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 - Recreate operating rooms for surgical practice - Capture patient rooms for staff orientation - Build libraries of medical environments for education 🏢 𝗖𝗼𝗿𝗽𝗼𝗿𝗮𝘁𝗲 & 𝗥𝗲𝗺𝗼𝘁𝗲 𝗪𝗼𝗿𝗸 - Share office spaces with distributed teams - Capture meeting rooms for hybrid collaboration - Preserve company spaces for remote onboarding 🎓 E𝗱𝘂𝗰𝗮𝘁𝗶𝗼𝗻 & 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 - Bring students into real lab environments virtually - Capture historical spaces for preservation - Create accessible learning environments for all students Soon, when you can invite others into these spaces, we're looking at a new way to collaborate, train, and connect across distance. What kind of spaces did you already scan with Hyperscape?