Control camera pan, tilt, and zoom

Room-scale video conferencing solutions deploy cameras with pan, tilt, and zoom
(PTZ) capabilities so that software can point the camera at meeting
participants. Starting in Chrome 87, the pan, tilt, and zoom features on
cameras are available to websit…

Room-scale video conferencing solutions deploy cameras with pan, tilt, and zoom
(PTZ) capabilities so that software can point the camera at meeting
participants. Starting in Chrome 87, the pan, tilt, and zoom features on
cameras are available to websites using media track constraints in
MediaDevices.getUserMedia() and MediaStreamTrack.applyConstraints().

Using the API

Feature detection

Feature detection for hardware is different from what you’re probably used to.
The presence of "pan", "tilt", and "zoom" constraint names in
navigator.mediaDevices.getSupportedConstraints() tells you that the browser
supports the API to control camera PTZ, but not whether the camera hardware
supports it. As of Chrome 87, controlling camera PTZ is supported on
desktop, while Android still supports zoom only.

const supports = navigator.mediaDevices.getSupportedConstraints();
if (supports.pan && supports.tilt && supports.zoom) {
// Browser supports camera PTZ.
}

Request camera PTZ access

A website is allowed to control camera PTZ only if the user has explicitly
granted the camera with PTZ permission through a prompt.

To request camera PTZ access, call navigator.mediaDevices.getUserMedia() with
the PTZ constraints as shown below. This will prompt the user to grant both
regular camera and camera with PTZ permissions.

Screenshot of a camera PTZ user prompt in Chrome for macOS.
Camera PTZ user prompt.

The returned promise will resolve with a MediaStream object used to show the
camera video stream to the user. If the camera does not support PTZ, the user
will get a regular camera prompt.

try {
// User is prompted to grant both camera and PTZ access in a single call.
// If camera doesn't support PTZ, it falls back to a regular camera prompt.
const stream = await navigator.mediaDevices.getUserMedia({
// Website asks to control camera PTZ as well without altering the
// current pan, tilt, and zoom settings.
video: { pan: true, tilt: true, zoom: true }
});

// Show camera video stream to user.
document.querySelector("video").srcObject = stream;
} catch (error) {
// User denies prompt or matching media is not available.
console.log(error);
}

A previously-granted camera permission, specifically one without PTZ access,
does not automatically gain PTZ access if it becomes available. This is true
even when the camera itself supports PTZ. The permission must be requested
again. Fortunately, you can use the Permissions API to query and monitor the
status of PTZ permission.

try {
const panTiltZoomPermissionStatus = await navigator.permissions.query({
name: "camera",
panTiltZoom: true
});

if (panTiltZoomPermissionStatus.state == "granted") {
// User has granted access to the website to control camera PTZ.
}

panTiltZoomPermissionStatus.addEventListener("change", () => {
// User has changed PTZ permission status.
});
} catch (error) {
console.log(error);
}

To know whether a Chromium-based browser supports PTZ for a camera, go to the
internal about://media-internals page and check out the "Pan-Tilt-Zoom" column
in the "Video Capture" tab.

Screenshot of the internal page in Chrome OS to debug PTZ camera support.
Internal page to debug PTZ camera support.

Control camera PTZ

Manipulate camera PTZ capabilities and settings using the preview
MediaStreamTrack from the stream object obtained earlier.
MediaStreamTrack.getCapabilities() returns a dictionary with the supported
capabilities and the ranges or allowed values. Correspondingly,
MediaStreamTrack.getSettings() returns the current settings.

Pan, tilt, and zoom capabilities and settings are available only if supported by
the camera and the user has granted PTZ permission to the camera.

Controlling Camera PTZ.

Call videoTrack.applyConstraints() with the appropriate PTZ advanced
constraints
to control camera pan, tilt, and zoom as shown in the example below.
The returned promise will resolve if successful. Otherwise it will reject if
either:

  • the camera with PTZ permission is not granted.
  • the camera hardware does not support the PTZ constraint.
  • the page is not visible to the user. Use the Page Visibilty API to detect
    page visibility changes.
// Get video track capabilities and settings.
const [videoTrack] = stream.getVideoTracks();
const capabilities = videoTrack.getCapabilities();
const settings = videoTrack.getSettings();

// Let the user control the camera pan motion if the camera supports it
// and PTZ access is granted.
if ("pan" in settings) {
const input = document.querySelector("input[type=range]");
input.min = capabilities.pan.min;
input.max = capabilities.pan.max;
input.step = capabilities.pan.step;
input.value = settings.pan;

input.addEventListener("input", async () => {
await videoTrack.applyConstraints({ advanced: [{ pan: input.value }] });
});
}

if ("tilt" in settings) {
// similar for tilt...
}
if ("zoom" in settings) {
// similar for zoom...
}

It is also possible to configure camera pan, tilt, and zoom by calling
navigator.mediaDevices.getUserMedia() with some camera PTZ ideal constraint
values. This is handy when camera PTZ capabilities are known in advance. Note
that mandatory constraints (min, max, exact) are not allowed here.

const stream = await navigator.mediaDevices.getUserMedia({
// Website asks to reset known camera pan.
video: { pan: 0, deviceId: { exact: "myCameraDeviceId" } }
});

Playground

You can play with the API by running the demo on Glitch. Be sure to check out
the source code
.

Tip: If you don’t have a camera that supports PTZ, you can run Chrome with the
switch
--use-fake-device-for-media-stream to simulate one on your machine.
Enjoy!

Security Considerations

The spec authors have designed and implemented this API using the core
including user control, transparency, and ergonomics. The ability to use this
API is primarily gated by the same permission model as the Media Capture and
Streams API
. In response to a user prompt, the website is allowed to control
camera PTZ only when the page is visible to the user.

Helpful links

Acknowledgements

This article was reviewed by Joe Medley and Thomas Steiner.
Thanks to Rijubrata Bhaumik and Eero Häkkinen at Intel for their work on the
spec and the implementation.
Hero image by Christina @ wocintechchat.com on Unsplash.


Print Share Comment Cite Upload Translate
APA
François Beaufort | Sciencx (2024-03-28T08:57:07+00:00) » Control camera pan, tilt, and zoom. Retrieved from https://www.scien.cx/2020/10/05/control-camera-pan-tilt-and-zoom/.
MLA
" » Control camera pan, tilt, and zoom." François Beaufort | Sciencx - Monday October 5, 2020, https://www.scien.cx/2020/10/05/control-camera-pan-tilt-and-zoom/
HARVARD
François Beaufort | Sciencx Monday October 5, 2020 » Control camera pan, tilt, and zoom., viewed 2024-03-28T08:57:07+00:00,<https://www.scien.cx/2020/10/05/control-camera-pan-tilt-and-zoom/>
VANCOUVER
François Beaufort | Sciencx - » Control camera pan, tilt, and zoom. [Internet]. [Accessed 2024-03-28T08:57:07+00:00]. Available from: https://www.scien.cx/2020/10/05/control-camera-pan-tilt-and-zoom/
CHICAGO
" » Control camera pan, tilt, and zoom." François Beaufort | Sciencx - Accessed 2024-03-28T08:57:07+00:00. https://www.scien.cx/2020/10/05/control-camera-pan-tilt-and-zoom/
IEEE
" » Control camera pan, tilt, and zoom." François Beaufort | Sciencx [Online]. Available: https://www.scien.cx/2020/10/05/control-camera-pan-tilt-and-zoom/. [Accessed: 2024-03-28T08:57:07+00:00]
rf:citation
» Control camera pan, tilt, and zoom | François Beaufort | Sciencx | https://www.scien.cx/2020/10/05/control-camera-pan-tilt-and-zoom/ | 2024-03-28T08:57:07+00:00
https://github.com/addpipe/simple-recorderjs-demo