WWDC26: Camera and Photo Technologies Group Lab - Q&A
Direct answers from Apple Engineers during WWDC26
Camera and Photos APIs are full of tradeoffs: speed versus quality, preview versus capture, metadata versus UI, and convenience APIs versus low-level control. This lab focused on practical questions around PhotoKit, AVFoundation, Core Image, thumbnails, depth, deferred start, and capture behavior.
As usual, the goal is simple: make the questions easier to scan, easier to revisit, and easier to connect with real app development problems.
I tried to preserve the original wording and combine related answers where appropriate. However, some inaccuracies or mismatches are still possible.
Enjoy! And subscribe so you don’t miss the next Lab.
When Reframe or Clean Up is used to modify a photo, does iOS 27 tag it with metadata or content credentials so someone can tell the image was edited by AI?
Yes. The file metadata is modified.
The panel said that IPTC metadata is updated, together with EXIF, and the IPTC metadata reflects which AI modification was used, such as spatial reframe or Clean Up.
In the Photos app, this information is also visible in the info panel. On iOS, when you swipe up on a photo, the info panel can show which edit was used.
Keywords were mentioned as a feature coming to Photos. Will there be an API for third-party apps to view or edit keywords?
Keywords are available in Photos on iOS, matching a feature that has existed for a long time on macOS.
In the Photos app, users can see and edit keywords in the info panel. There is also UI for managing keywords and searching by them.
When exporting, keywords are reflected in IPTC metadata.
However, there is currently no PhotoKit-level API for fetching, editing, or querying Photos library assets by those keywords.
When showing thumbnails of images in a lazy grid view that also use matched geometry effect to a full image detail view, what is the optimal way to load thumbnails for performance?
Do not decode full-size images when all you need is a thumbnail.
On the Core Graphics side, there are image-opening options that let Core Graphics use an existing embedded thumbnail if one is available, or decode and scale down the main image as efficiently as possible.
On the Core Image side, you can request a scale factor when opening images. That lets Core Image scale the image down early in the processing pipeline, so later work operates on the smaller image and runs faster.
The practical goal is to make thumbnail generation match the UI’s real size needs. Store or request smaller representations where possible, and avoid doing expensive full-resolution work for grid cells.
Can deferred start cause issues where the user may attempt to capture a photo before the capture photo output has attached to the session?
Yes, if you only defer initialization, you can still miss the shot.
Deferred start pushes initialization work later so the camera can launch quickly. But if the user taps capture before the photo output is fully ready, that can create a race unless you use the right capture behavior.
The panel pointed to isResponsiveCaptureEnabled on AVCapturePhotoOutput. When enabled together with deferred start, the system adds buffering so the app can launch quickly and still queue the capture even if the output is not fully initialized yet.
Deferred start was introduced in iOS 26. The common pattern is to prioritize the preview first and defer less critical outputs so the camera feels responsive immediately.
What is the best way to get the depth map and the image for a live viewfinder and for the taken image to create a nice 3D effect with both?
There are two parts: preview and still capture.
For a live preview where you want direct access to depth samples, use AVCaptureDepthDataOutput together with the video stream, and use AVCaptureDataOutputSynchronizer so the RGB and depth frames arrive with matching timestamps.
For still capture, enable depth delivery on AVCapturePhotoOutput. The specific setting mentioned was depthDataDeliveryEnabledon AVCapturePhotoSettings, so the still photo capture includes depth data.
If you only need a live depth-style preview effect similar to the Camera app’s cinematic behavior, there is a higher-level shortcut: enable the cinematic video capture API. That can give you the depth effect in the preview without manually handling all depth samples yourself.
For custom effects, once you have the video/depth or still/depth pair, you can combine them with Core Image filters.
When photoQualityPrioritization is set to .quality, AVCaptureDevice often overrides manual exposure duration and ISO during capture. Is there a way to know the final exposure settings before the photo is captured?
Manual exposure settings are only fully respected for speed-prioritized captures.
For photo mode, both balanced and quality capture can use multi-image processing and fusion. That may include underexposed and normally exposed frames, and the pipeline chooses capture settings to support that processing.
So if balanced or quality appears to respect your manual settings, that may only be because the current scene and pipeline happened to choose something close. It is not a guarantee.
The practical answer is: if you need strict manual exposure control, use speed prioritization. If you use balanced or quality, expect the system to optimize for capture quality and processing needs rather than preserving your manual exposure and ISO settings exactly.
Can you elaborate on PHAsset original resource choice and what it is used for?
PHAsset original resource choice is related to assets that contain more than one original-like resource, such as RAW plus JPEG or RAW plus HEIF.
In Photos, users can choose whether edits are based on the RAW image or the compressed image. That choice also affects which resource Photos uses to generate smaller derivatives and thumbnails.
The new API exposes that choice to developers. You can inspect or change which resource is treated as the original, and therefore which source is used for editing and derivative generation.
The panel mentioned APIs around content editing input source, and change requests on assets for toggling which resource is considered original.
Will iOS 27 support a linear scene-referred preview stream for Bayer RAW capture through AVCaptureVideoDataOutput or AVCaptureVideoPreviewLayer, without tone mapping or computational processing?
Today, the way to get scene-referred linear-style data from camera capture is through the log formats, including Log and Log 2.
The panel explained that if the request is really for RAW data coming out of camera capture and then using the same filters used for still images, that is not currently available.
For ProRAW capture, the RAW frames include metadata, but that metadata is not compatible with what CIRAWFilter would need to render the image as if it were a still DNG workflow.
The team said the idea is worth exploring and thanked the developer for filing Feedback Assistant. For now, this is not supported as an out-of-the-box preview stream.
For an app that lives and dies by its capture experience, what are the must-have recommendations?
Performance and reliability come first.
A capture app exists to catch moments. If the app launches slowly or misses the shot, the user may not get that moment back. The panel’s first recommendations were fast launch, responsive capture, and deferred processing.
But the right capture experience depends on the product. A pro photography app, a social video app, and a playful filtered camera may all need different priorities. The team encouraged developers to understand who the app is for and what differentiates it.
Use Apple’s sample code rather than starting from scratch. AVFoundation is large and easy to misuse. A common mistake is driving AVCaptureSession from the main thread, which can block UI during long operations. The sample code shows best practices such as using a dedicated serial queue.
Also think about the Photos side of the experience. If your app captures photos, users expect those photos to go somewhere. Integrate with PhotoKit permissions thoughtfully: you can request save-only access first, then ask for broader read access only if the app needs it.
What is the optimal recommended way for streaming video and audio simultaneously without sync drift?
Use AVCaptureSession for both audio and video when possible.
Attach a camera input and a microphone input to the same session, then use audio and video outputs from that session. AVFoundation will do the hard work of putting the audio and video samples on a coherent timeline.
If you use a different audio API, such as AU RemoteIO, you need to synchronize clocks yourself. Each device is backed by a time source, and you may need to convert presentation timestamps between clocks using Core Media timing APIs.
The usual approach is to preserve the audio timeline and align video timestamps to it, because audio glitches and timing shifts are easier for users to notice than tiny video timing adjustments.
If you stream over a network, keep the timestamps coherent before sending. On playback, use AVFoundation playback APIs such as AVPlayer or AVSampleBufferDisplayLayer to handle synchronization on the receiving side.
Why does ProRAW support 48 MP output from a Quad Bayer sensor, while native Bayer RAW output is limited to binned resolution?
ProRAW and Bayer RAW are different kinds of output.
Bayer RAW contains Bayer sensor data. ProRAW goes through debayering and Apple’s Photonic Engine-style merging before output. By that point, the data is already linearized RGB, so ProRAW no longer has to care whether the original sensor pattern was Bayer or Quad Bayer.
Quad Bayer RAW output would not simply be ordinary Bayer RAW at full resolution. It would be Quad Bayer data, and the ecosystem would also need a way to decode it, including support in tools such as CIRAWFilter.
The panel said this is heavier than it sounds, and encouraged developers who need Quad Bayer RAW output to file feedback.
What is the recommended process to generate and write ISO-conformant gain maps and gain map metadata for HEIF and JPEG on iOS?
Apple already covered this in a previous WWDC session focused on HDR and gain maps.
The panel pointed developers to that session for API details. The high-level answer is that Core Graphics and Core Image both support gain map workflows.
Core Image gives deep control over the output: the look, the headroom, and how SDR/RGB content is combined with the HDR addition. Core Graphics also supports writing the relevant metadata.
The ISO gain map specifications have been added to both HEIF and JPEG, and Apple has worked on that standards support. Developers who want to understand the metadata deeply can also look at the underlying specifications.
How should setPreparedPhotoSettingsArray work together with deferred start?
They are complementary, not conflicting.
Deferred start is about launching quickly: it lets the session get preview running first while deferring initialization for less immediate branches like photo or movie outputs.
setPreparedPhotoSettingsArray is about telling the photo output what the worst-case still capture configuration might be, so the pipeline can pre-allocate resources for that case.
You can use both. Deferred start moves initialization after preview starts, while prepared settings let the photo output prepare for expensive capture settings. You can also prepare or re-prepare settings later.
Without deferred start, photo output quality can affect launch time because quality capture may require heavier allocations. With deferred start, preview can start quickly and those allocations can happen later.
Will adjusting PHAsset’s new rating property require a Photo Library change request?
Yes.
The panel said this is handled through PHAssetChangeRequest. The new rating property can be changed per asset, and the rating enum includes unset plus one-through-five values.
Is there a native way to obtain the original file type, such as PNG or JPEG, from a photo selected with PhotosPicker?
Check how you configure Transferable.
The panel suggested making sure you set the UTType explicitly rather than relying on a default image type. If you leave it too generic, you may see output that does not match what you expected.
There can also be conversion in the picker in some situations. For example, if the user disables sharing location or captions with your app, Photos may convert from some formats such as RAW.
If everything is coming out as PNG, double-check the UTType used for transfer and review Apple’s PhotosPicker sample code.
Is there a way to learn about the adjustment plist format that Apple Photos writes, and can those adjustments be imported back into Photos?
No public API exists for this today.
The panel said there is no supported way to decode that adjustment plist or import those Photos app adjustments back into Photos through a public API.
If this is important for your app, file a Feedback Assistant request.
What are the biggest mistakes developers make when building camera-heavy apps on iPhone?
The first big mistake is using AVCaptureSession on the main thread.
AVCaptureSession can block while it reconfigures the capture graph. If you start or reconfigure it on the main thread, your UI can hang. Use a dedicated serial queue for capture session work.
The second mistake is changing multiple session properties without beginConfiguration and commitConfiguration. If you make disruptive changes one by one, the session may re-evaluate and reconfigure the graph after each change. Wrap related changes in one transaction so the graph is updated once.
Another common mistake is using AVCaptureVideoDataOutput just to render preview. If you do not need to inspect or modify frames, use AVCaptureVideoPreviewLayer. It is highly optimized. Use video data output only when you need direct access to buffers for processing, overlays, histograms, metadata, or similar work.
Is there any API to access the new Spatial Reframe pipeline used by Photos?
No. There is no public developer API for the Photos app’s Spatial Reframe capability.
Some parts of ARKit can capture and reconstruct 3D scenes, but that is not the same as a ready-made Photos Spatial Reframe editing pipeline.
If you are interested in live capture with depth, AVFoundation can use LiDAR depth on supported Pro iPhones, or TrueDepth on the front-facing camera. But that is separate from the Photos app’s Spatial Reframe feature.
The panel recommended filing feedback if developers want access to this kind of pipeline.
When will camera capture rotation/orientation handling be made easier across iOS, iPadOS, and macOS?
The panel acknowledged that orientation handling can be tricky across Apple platforms, because different devices and UI contexts treat orientation differently.
The recommendation was to use the newer rotation coordination APIs where available, rather than manually building one-off orientation logic for every platform.
They also encouraged developers to file feedback for cases that are still painful, especially when platform differences make capture output hard to reason about.
Can third-party apps capture 24 MP photos with depth from both cameras, similar to the system Camera app?
The panel’s answer was cautious: capabilities depend on device, camera, format, and selected capture configuration.
Depth delivery and high-resolution photo capture are each supported through AVFoundation APIs, but not every combination is available on every device or every camera. Some system Camera app behaviors depend on private pipeline choices or supported formats that third-party apps must query at runtime.
The practical guidance is to inspect the available AVCaptureDevice formats, supported photo dimensions, and depth delivery support for the exact device and camera you are using. Do not assume the system Camera app’s full behavior is exposed as a single third-party API combination.
How should developers think about multicam capture and streaming from multiple iPhone cameras?
Use the supported multicam APIs and design around device limits.
Multiple simultaneous camera streams can be expensive in terms of processing, memory bandwidth, thermal load, and power. The available combinations depend on hardware and formats.
For streaming, keep audio/video synchronization in mind, use coherent timestamps, and avoid doing unnecessary per-frame processing on the critical path. If you are only showing preview, prefer preview layers. If you need buffers for encoding or effects, keep processing efficient and expect to manage back pressure.
What should developers know about the new camera control APIs and niche capture features?
The later part of the lab covered several specialized APIs and edge cases.
The overall pattern was consistent: query capabilities at runtime, use the most specialized API only when the app really needs it, and file feedback when the system Camera app exposes behavior that third-party APIs cannot yet reproduce.
For pro camera apps, the team emphasized careful format selection, performance testing, and clear user experience decisions. A feature may be technically possible, but it still needs to fit the app’s purpose and the user’s expectations.
🏆 Acknowledgments
A huge thank-you to everyone who joined and shared practical Camera and Photos questions throughout the session. Your questions made the discussion useful for developers working on AI-edited image metadata, Photos keywords, thumbnails, fast camera launch, depth maps, still capture, deferred start, responsive capture, photo quality prioritization, PhotoKit resource choice, RAW workflows, gain maps, audio/video synchronization, PhotosPicker, rotation, Spatial Reframe, and camera-heavy app performance.
Question acknowledgments: Florent INF, Ben, SJK_27, Yelon, Eric, and the online WWDC audience who submitted and upvoted the remaining Camera and Photo Technologies questions.
Finally, a heartfelt thank-you to Sergey, Matt Deoff, Brad Ford, Ivan Cavo, David Konchon, Jake, and the teams behind the scenes for leading the session and explaining how to build better camera and photo experiences.