Skip to main content

Performance

Performance of VisionCamera​

VisionCamera is highly optimized to be as fast as a native Camera app, and is sometimes even faster than that. I am using highly efficient native GPU buffer formats (such as YUV 4:2:0, or lossy compressed YUV 4:2:0), running the video pipelines in parallel, using C++ for the Frame Processors implementation, and other tricks to make sure VisionCamera is as efficient as possible.

Making it faster​

There are a few things you can do to make your Camera faster which requires a core understanding of how Cameras work under the hood:

Simpler Camera Device​

Selecting a "simpler" Camera Device (i.e. a Camera Device with less physical cameras) allows the Camera to initialize faster as it does not have to start multiple devices at once. You can prefer a simple wide-angle Camera (['wide-angle-camera']) over a triple camera (['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']) to significantly speed up initialization time.

const fasterDevice = useCameraDevice('back', {
  physicalDevices: ['wide-angle-camera']
})
const slowerDevice = useCameraDevice('back', {
  physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera']
})

See "Camera Devices" for more information.

Note: By default (when not passing the options object), a simpler device is already chosen.

No Video HDR​

Video HDR uses 10-bit formats and/or additional processing steps that come with additional computation overhead. Disable videoHdr for higher efficiency.

Buffer Compression​

Enable Buffer Compression (enableBufferCompression) to use lossy-compressed buffers for the Camera's video pipeline. These buffers can use less memory and are more efficient.

Note: When not using a frameProcessor, buffer compression is automatically enabled.

GPU buffers​

Enable GPU Buffer flags (enableGpuBuffers) to optimize the Video Pipeline for zero-copy buffer forwarding. If this is enabled, the Video Pipeline can avoid an additional CPU -> GPU copy, resulting in better performance and more efficiency.

Note: This only takes effect when using a frameProcessor.

Video Stabilization​

Video Stabilization requires additional overhead to start the algorithm, so disabling videoStabilizationMode can significantly speed up the Camera initialization time.

Pixel Format​

By default, the native PixelFormat is used, which is much more efficient than rgb.

  • On iOS, native is yuv
  • On Android native is some kind of vendor specific format, which might be yuv

Disable unneeded pipelines​

Only enable photo, video, codeScanner or frameProcessor if needed.

Using isActive​

The isActive prop controls whether the Camera should actively stream frames. Instead of fully unmounting the <Camera> component and remounting it again, keep it mounted and just switch isActive on or off. This makes the Camera resume much faster as it internally keeps the session warmed up.

Fast Photos​

If you need to take photos as fast as possible, use a qualityBalance of 'speed' to speed up the photo pipeline:

return <Camera {...props} qualityBalance="speed" />

Snapshot Capture​

If photo capture is still too slow for your use-case, consider taking snapshots instead:

const snapshot = await camera.current.takeSnapshot({
  quality: 85
})

Appropriate Format resolution​

Choose formats efficiently. If your backend can only handle 1080p videos, don't select a 4k format if you have to downsize it later anyways - instead use 1080p already for the Camera:

const format = useCameraFormat(device, [
  { videoResolution: { width: 1920, height: 1080 } }
])

Appropriate Format FPS​

Same as with format resolutions, also record at the frame rate you expect. Setting your frame rate higher can use more memory and heat up the battery. If your backend can only handle 30 FPS, there is no need to record at 60 FPS, instead set the Camera' fps to 30:

return <Camera {...props} fps={30} />

🚀 Next section: Camera Errors​