WWDC26: visionOS Group Lab - Q&A
Direct answers from Apple Engineers during WWDC26
visionOS is still young enough that many of the best questions are not only about APIs. They are about workflows, debugging, 3D asset pipelines, spatial accessories, WebXR, visual intelligence, testing, and how to design experiences that feel native to Apple Vision Pro instead of simply ported from another platform.
As usual, the goal is simple: make the questions easier to scan, easier to revisit, and easier to connect with real app development problems.
I tried to preserve the original wording and combine related answers where appropriate. However, some inaccuracies or mismatches are still possible.
Enjoy! And subscribe so you don’t miss the next Lab.
As an independent developer, what is the best way to gain access to the front-facing cameras for learning or trying something for an app? Do you need to be part of an enterprise development group, or is it impossible for an individual developer to gain access?
Access to the main cameras on Apple Vision Pro exists, but it is handled through an application process and is primarily intended for professional and enterprise-oriented use cases.
It is not limited only to developers in the Apple Developer Enterprise Program. Developers with a standard Apple Developer Program account can apply as well, especially when the account is tied to a business and the app has a concrete use case that requires camera access.
If you have a real use case and cannot access the capability today, post in the developer forums and file feedback. Apple specifically asked developers to share use cases that would benefit from front-facing camera access, because that helps the team understand where broader access or different APIs may be needed.
Hitting a breakpoint in an immersive space means standing inside a frozen universe of your own making, squinting at Xcode through passthrough. How do you debug on device? Am I missing a trick?
Use Mac Virtual Display inside the immersive environment. There is a developer setting that allows Mac Virtual Display to appear while you are in an immersive experience. That lets you keep Xcode visible while staying inside the app you are debugging.
This helps both productivity and safety. When a breakpoint hits, you do not need to leave the headset, remove the device, or try to inspect Xcode awkwardly through passthrough. You can keep the immersive context, inspect the breakpoint, and continue iterating.
The panel noted that this is also a common internal workflow: develop in Xcode through Mac Virtual Display while running the immersive content alongside it.
Are any of the agent skills Apple is providing this week especially relevant for visionOS development?
The most directly relevant skill is the one that helps resize an existing iOS app for visionOS. A 2D iOS interface usually assumes a particular form factor, density, and layout. On visionOS, those assumptions often need to change.
The skill is meant to help translate iOS UI into a form that better fits visionOS, while preserving platform best practices and principles. It is especially useful when bringing an existing iOS app to visionOS rather than starting from a fully spatial design.
What is your usual workflow for exporting 3D models into USD? Some tools apply different transforms or remove certain materials on complex models. The usual models are FBX or GLB files.
The recommended approach is to start with USD as early as possible in the pipeline. If you can export directly from the original 3D content creation tool into USD, that is usually better than transiting through another format first. Intermediate formats can leave their own “fingerprints” on transforms, materials, or scene structure.
There is still variability between 3D tools and how they export data, because the industry is still adopting USD at different levels of maturity. But starting from the original authoring tool and exporting USD directly gives you the best chance of preserving intent.
If you are iterating on models, consider loose USD formats such as USDA or USDC instead of repackaging an entire scene every time. That can make iteration faster because individual files can be modified without rebuilding the whole package.
The panel also noted that the USD core specification is now available, which makes it easier for modern AI tools to understand USD structure and potentially help inspect or repair model issues.
Can spatial accessories be tracked outside the range of direct view? I’m imagining foot trackers, but obviously the user will not always be looking down at their feet.
Spatial accessories use a combination of Bluetooth, an IMU, and infrared emitters that Apple Vision Pro can see with its infrared cameras. The IMU can continue providing motion information even when the accessory is briefly occluded or not perfectly visible.
That said, tracking fidelity depends on the accessory, distance, occlusion, size, and whether the Vision Pro cameras can see the infrared constellation. If an accessory is behind the body or far down near the feet, tracking quality may vary depending on the use case and required precision.
visionOS 27 opens the spatial accessory specification so developers and hardware makers can build new accessories. Companies such as DF Robot and MikroE were mentioned as providing components or boards that can be used to prototype accessories.
There is also a new debugging mode that shows the infrared feed Vision Pro sees, which helps you verify whether your accessory is visible in the positions you care about.
For foot-tracking use cases, Apple specifically asked for feedback explaining why lower-body tracking is needed and what the experience is trying to achieve.
When using Gaussian splatting, we notice that the splats are culled if you move your face close or inside the splat. We are trying to reconstruct an immersive scene through sensor data for telepresence. The culling is not ideal. Is there a way to disable this?
There does not appear to be a way to disable that vignette or culling behavior today.
The behavior is not limited only to Gaussian splats. Similar treatment can apply to 3D content that gets too close to the user’s face, because the system tries to preserve an unobstructed field of view and avoid uncomfortable or unsafe experiences.
Apple asked developers with this kind of use case to file feedback. The panel emphasized that Gaussian splats are a new and evolving area, and concrete use cases are especially important. A telepresence scene is different from a small scanned object, so feedback should include the creative intent, sample project if possible, and screenshots or screen recordings showing the issue and distance where it happens.
Is Gaussian splat rendering supported on platforms other than visionOS? The documentation shows support for iOS, but Xcode does not show the symbols available.
Gaussian splatting should generally be supported across both iOS and visionOS. If the documentation and Xcode availability do not match, that is a beta issue worth filing as feedback.
The panel encouraged developers to file feedback when documentation says an API is available but Xcode symbols are missing or the feature does not work as expected. During beta, this kind of mismatch is exactly the sort of issue Apple can still investigate and correct.
Why aren’t there user profiles like on Mac? I want to put on the headset and choose who I am: owner, child one, child two, grandma.
Apple Vision Pro is treated more like a deeply personal device, closer in spirit to an iPhone or iPad than a Mac with multiple user profiles. The device is calibrated to a primary user, including eyes and hands, and the experience is designed around that personal model.
That said, Apple has been reducing the friction of sharing. Guest User workflows have improved, and enrollment can be saved to an iPhone. After someone completes enrollment once, they can store it on their iOS device, and the next time they use another Vision Pro in Guest User mode, the phone can present an App Clip code to restore that setup quickly.
There is also Guest User with nearby device, where an iPhone or iPad paired to the same iCloud account can help approve access and mirror what the guest sees. So while full Mac-style profiles are not the model today, sharing workflows have become much faster.
Is there any API to detect the lips and face of the owner and mimic that on 3D, like Persona but differently?
The ARKit face tracking API is iOS-only. visionOS does not provide raw face or lip-tracking data to third-party apps in the same way.
Third-party apps can access a camera-style video feed containing the user’s Persona for use cases like video conferencing, but not raw face geometry or data that would let you build custom lens-style effects or drive a different 3D face directly.
The panel said this is an interesting use case and recommended filing feedback with details about what you want to build.
For visual intelligence on Apple Vision Pro, when Siri answers questions about content the user is looking at, either app windows or real-world objects, does a third-party app need access to the main camera, or is visual intelligence handled entirely at the system level?
Visual intelligence is handled at the system level. Third-party apps do not need camera access for this capability, and the information is not shared with third-party apps.
The panel emphasized that this is done in a privacy-preserving way. The system can answer questions about what the user is looking at, but that does not mean an app receives the raw camera feed or scene data.
For apps that support game controllers but are not games, how does Apple envision controllers fitting into the long-term interaction model of visionOS alongside eyes, hands, and voice? Are there new controller capabilities in visionOS 27 that developers should design for today?
The input model depends on the experience. Eyes and hands remain fundamental because they map naturally to how people interact with the physical world: you look at something, then reach or act on it. Voice also fits many workflows.
Controllers and spatial accessories are useful when they add something beyond natural input: precision, haptics, physical affordances, repeatable controls, or specialized interaction. Examples include a stylus-like accessory for refined input, PlayStation VR2 Sense controllers for games, traditional game controllers, or custom spatial accessories built with the newly opened specification.
visionOS 27 expands this direction by opening spatial accessory support so developers can build custom tracked hardware. These accessories can integrate at the system and SDK level, while still fitting into the broader visionOS interaction model.
The guidance is not to use a controller just because it exists. Use it when it meaningfully improves the experience.
Are there any updates to WebXR in Safari with visionOS 27? In particular, immersive AR mode. We released a WebXR app and would love to run it with immersive AR mode enabled, but we are currently limited to immersive VR only.
Safari on visionOS supports WebXR, but the panel described it as immersive-only rendering mode today.
For AR-style experiences, Apple encouraged developers to also look at the native platform APIs, because visionOS has a rich set of APIs for spatial and augmented reality experiences. Those APIs may allow a deeper integration than the current WebXR path.
The panel also mentioned web-side improvements such as the model tag, which lets web pages surface 3D content and personal environments without using WebXR in the same way.
If a WebXR app needs a specific AR capability, the recommendation is to file feedback and include the app link, desired behavior, and concrete use case.
For visual intelligence on visionOS, is it purely camera-based, or does it leverage the depth sensors under the hood?
The panel did not disclose the exact implementation details. The high-level answer is that Apple aims to provide the best possible result on each platform using the sensors available on that device.
Different devices have different cameras and sensors, so there is not necessarily one identical implementation everywhere. The system can blend data from available sources depending on the device and the context of the query.
The same general principle applies across visionOS features such as mapping, hand tracking, and spatial understanding: the system uses the sensor data that makes sense for the experience while preserving the platform’s privacy and interaction model.
The device sleeps the instant it leaves your head, taking the debug session with it. What is the sanctioned way to keep it awake on a desk during development?
There is no “keep awake on desk” mode equivalent to caffeinating another device. The reason is security: taking Vision Pro off is effectively the lock gesture. The expectation is that the next person who puts it on must authenticate with Optic ID or passcode.
The practical workflow is to use Mac Virtual Display while wearing the device. That keeps Xcode, SwiftUI previews, and Reality Composer Pro workflows available in headset while you build and iterate.
If you do need to remove the headset, having Optic ID configured makes returning much faster. The panel described Mac Virtual Display plus Optic ID as the common development workflow.
Can you give us an update on automated testing for visionOS, ARKit, and RealityKit?
For RealityKit, look at RealityRenderer. It can drive RealityKit rendering programmatically and can be useful for automated tests or other non-interactive rendering workflows.
The panel also recommended new testing sessions, including “Migrate to Swift Testing” and “Get the most out of Device Hub.” Device Hub can support more testing workflows than many developers may expect.
RealityKit is written in Swift, so existing Swift testing and Xcode testing infrastructure should also apply where appropriate.
What are the biggest mistakes new visionOS developers make?
One major mistake is assuming that an app that works in the simulator will feel right on device. On Vision Pro, information density, window size, tap target size, and interaction style feel very different because users interact with eyes and hands in a spatial environment.
Another mistake is stopping at a direct 2D port. A good visionOS app should consider what is uniquely possible on Vision Pro. For an e-commerce app, that might mean letting users pull a 3D product model out of the window and place it on a desk or floor. For other apps, it might mean using room understanding, lighting, spatial placement, or immersive presentation.
The panel compared this to the early iPhone transition. Early apps often copied old interaction models, but the most interesting apps eventually embraced touch as a new medium. visionOS is at a similar stage: developers should explore what spatial computing makes newly possible rather than only reproducing existing UI patterns.
Regarding agentic coding in visionOS, I tried agentic coding in Xcode to generate a Vision Pro visionOS project — just a Hello World entity in a RealityView, following a local LLM WWDC26 session — but it generated the app with wrong syntax. Is there any way to fix this or teach the AI to know more about visionOS code?
Generated code can be wrong, especially when APIs are new and models were trained on older versions of frameworks. The first recommendation is to be deliberate with prompts and always verify the generated output. Treat it as “trust, but verify.”
For local models, Apple’s sample projects can help provide better context. Examples mentioned include Hello World, Petite Asteroids, Canyon Crosser, and the newer Model Manipulator sample. Feeding or referencing strong sample code can help the model produce more accurate visionOS and RealityKit patterns.
The panel also noted that some local LLM experiments can work surprisingly well, but results will vary. visionOS APIs move quickly, so there may be a transition period where models lag behind the latest SDK syntax. Smaller prompts, real sample projects, and compile-checking the output are the practical workflow.
For Object Tracking, how fast it can get now? What will be possible frame rate and latency?
visionOS 27 improves object tracking in a few important ways. First, the Object Tracking tool in Create ML has been improved so it can support more kinds of shapes. The earlier version worked best when objects had enough visual features and clear geometry; the new version can handle a wider range of objects.
Second, RealityKit and ARKit now align object-tracking updates with frame display time. That matters when you are rendering content against a tracked object, because you no longer have to guess whether the tracking update corresponds to the previous frame or the next frame. The tracked pose is aligned with the actual frame being presented.
The practical advice is to use the new tool, test with the real object, and file feedback with sample assets or recordings when tracking quality is not good enough. For larger or awkward physical objects, it may still be better to track a smaller attached feature or use a spatial accessory.
Vision Pro has awesome sensors and impressive frameworks like Persona Kit and CoreIK running under the hood. Will Apple expose more of these to developers, particularly for accessibility-focused telepresence apps for disabled users working in hybrid environments?
Apple did not announce new access to those lower-level internal capabilities in this lab. The answer was to file feedback with the exact accessibility and telepresence use case.
The panel emphasized that Apple cares deeply about accessibility, and that concrete use cases matter. A general request for access to internal frameworks is less useful than a clear explanation of the user need, what you are trying to build, what data or capability is missing, and why current public APIs cannot solve it.
The discussion also noted that accessibility features often help many more people than the original target group. So if the request is tied to disabled users, hybrid work, or telepresence, it is especially worth documenting carefully in Feedback Assistant.
Is there any way to use Mac Virtual Display with a shared Mac or an enterprise Mac setup, especially where the Apple ID on the Mac and Vision Pro may not match?
The recommended path is to file feedback for the exact enterprise or shared-device scenario. Mac Virtual Display is designed around a personal workflow, and some enterprise environments introduce constraints around Apple IDs, device ownership, and managed hardware.
The panel did not give a workaround or announce a new capability here. The key point was that Apple needs concrete details: whether this is a shared lab Mac, a managed enterprise Mac, a device used by multiple employees, or a training/development setup. Those details help the team understand what kind of support would actually solve the problem.
For day-to-day development today, the strongest recommended workflow remains Mac Virtual Display with the developer’s own Mac and Vision Pro, especially when debugging immersive content.
Are Gaussian splats supported in USDZ files, and can they be distributed through the same workflows as other USD content?
Gaussian splats are tied to USD workflows, but the exact packaging and support details depend on the current beta tools and APIs. The panel’s broader recommendation was to file feedback when documentation, Xcode symbols, or asset behavior do not line up with what you expect.
Because Gaussian splatting is still a new area for RealityKit and visionOS, concrete examples matter. If a USDZ packaging workflow does not work, include the asset, the expected result, the observed result, and any sample project needed to reproduce the issue.
This is especially important during beta, when format support, tooling behavior, and documentation can still be clarified or corrected.
🏆 Acknowledgments
A huge thank-you to everyone who joined and shared thoughtful visionOS questions throughout the session. Your questions made the discussion useful for developers working on camera access, debugging workflows, USD pipelines, spatial accessories, Gaussian splats, visual intelligence, WebXR, testing, native visionOS design, agentic coding, object tracking, accessibility, travel scenarios, and thermal behavior.
Question acknowledgments: Wes Matlock, Thomas Bastible, Jodor Tree, Sismeia, Eric, Ethan, Oliver Cologne, Dilliam, Raymond Yay, and the online WWDC audience who submitted and upvoted the remaining questions.
Finally, a heartfelt thank-you to Adesh Pavani, Katie, Norman, John, Matt, and Travis for leading the session, sharing practical guidance, and explaining how to think about visionOS as a spatial platform rather than just another screen.