dev-overlay.rst (13338B)
1.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later 2 3.. _overlay: 4 5*********************** 6Video Overlay Interface 7*********************** 8 9**Also known as Framebuffer Overlay or Previewing.** 10 11Video overlay devices have the ability to genlock (TV-)video into the 12(VGA-)video signal of a graphics card, or to store captured images 13directly in video memory of a graphics card, typically with clipping. 14This can be considerable more efficient than capturing images and 15displaying them by other means. In the old days when only nuclear power 16plants needed cooling towers this used to be the only way to put live 17video into a window. 18 19Video overlay devices are accessed through the same character special 20files as :ref:`video capture <capture>` devices. 21 22.. note:: 23 24 The default function of a ``/dev/video`` device is video 25 capturing. The overlay function is only available after calling 26 the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. 27 28The driver may support simultaneous overlay and capturing using the 29read/write and streaming I/O methods. If so, operation at the nominal 30frame rate of the video standard is not guaranteed. Frames may be 31directed away from overlay to capture, or one field may be used for 32overlay and the other for capture if the capture parameters permit this. 33 34Applications should use different file descriptors for capturing and 35overlay. This must be supported by all drivers capable of simultaneous 36capturing and overlay. Optionally these drivers may also permit 37capturing and overlay with a single file descriptor for compatibility 38with V4L and earlier versions of V4L2. [#f1]_ 39 40A common application of two file descriptors is the X11 41:ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application. 42While the X server controls video overlay, the application can take 43advantage of memory mapping and DMA. 44 45Querying Capabilities 46===================== 47 48Devices supporting the video overlay interface set the 49``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct 50:c:type:`v4l2_capability` returned by the 51:ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O 52method specified below must be supported. Tuners and audio inputs are 53optional. 54 55 56Supplemental Functions 57====================== 58 59Video overlay devices shall support :ref:`audio input <audio>`, 60:ref:`tuner`, :ref:`controls <control>`, 61:ref:`cropping and scaling <crop>` and 62:ref:`streaming parameter <streaming-par>` ioctls as needed. The 63:ref:`video input <video>` and :ref:`video standard <standard>` 64ioctls must be supported by all video overlay devices. 65 66 67Setup 68===== 69 70Before overlay can commence applications must program the driver with 71frame buffer parameters, namely the address and size of the frame buffer 72and the image format, for example RGB 5:6:5. The 73:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and 74:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and 75set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is 76privileged because it allows to set up DMA into physical memory, 77bypassing the memory protection mechanisms of the kernel. Only the 78superuser can change the frame buffer address and size. Users are not 79supposed to run TV applications as root or with SUID bit set. A small 80helper application with suitable privileges should query the graphics 81system and program the V4L2 driver at the appropriate time. 82 83Some devices add the video overlay to the output signal of the graphics 84card. In this case the frame buffer is not modified by the video device, 85and the frame buffer address and pixel format are not needed by the 86driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application 87can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` 88ioctl. 89 90A driver may support any (or none) of five clipping/blending methods: 91 921. Chroma-keying displays the overlaid image only where pixels in the 93 primary graphics surface assume a certain color. 94 952. A bitmap can be specified where each bit corresponds to a pixel in 96 the overlaid image. When the bit is set, the corresponding video 97 pixel is displayed, otherwise a pixel of the graphics surface. 98 993. A list of clipping rectangles can be specified. In these regions *no* 100 video is displayed, so the graphics surface can be seen here. 101 1024. The framebuffer has an alpha channel that can be used to clip or 103 blend the framebuffer with the video. 104 1055. A global alpha value can be specified to blend the framebuffer 106 contents with video images. 107 108When simultaneous capturing and overlay is supported and the hardware 109prohibits different image and frame buffer formats, the format requested 110first takes precedence. The attempt to capture 111(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay 112(:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error 113code or return accordingly modified parameters.. 114 115 116Overlay Window 117============== 118 119The overlaid image is determined by cropping and overlay window 120parameters. The former select an area of the video picture to capture, 121the latter how images are overlaid and clipped. Cropping initialization 122at minimum requires to reset the parameters to defaults. An example is 123given in :ref:`crop`. 124 125The overlay window is described by a struct 126:c:type:`v4l2_window`. It defines the size of the image, 127its position over the graphics surface and the clipping to be applied. 128To get the current parameters applications set the ``type`` field of a 129struct :c:type:`v4l2_format` to 130``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the 131:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the 132struct :c:type:`v4l2_window` substructure named ``win``. It is not 133possible to retrieve a previously programmed clipping list or bitmap. 134 135To program the overlay window applications set the ``type`` field of a 136struct :c:type:`v4l2_format` to 137``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and 138call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver 139adjusts the parameters against hardware limits and returns the actual 140parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the 141:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn 142about driver capabilities without actually changing driver state. Unlike 143:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled. 144 145The scaling factor of the overlaid image is implied by the width and 146height given in struct :c:type:`v4l2_window` and the size 147of the cropping rectangle. For more information see :ref:`crop`. 148 149When simultaneous capturing and overlay is supported and the hardware 150prohibits different image and window sizes, the size requested first 151takes precedence. The attempt to capture or overlay as well 152(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error 153code or return accordingly modified parameters. 154 155 156.. c:type:: v4l2_window 157 158struct v4l2_window 159------------------ 160 161``struct v4l2_rect w`` 162 Size and position of the window relative to the top, left corner of 163 the frame buffer defined with 164 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the 165 frame buffer width and height, the ``x`` and ``y`` coordinates can 166 be negative, and it can lie completely outside the frame buffer. The 167 driver clips the window accordingly, or if that is not possible, 168 modifies its size and/or position. 169 170``enum v4l2_field field`` 171 Applications set this field to determine which video field shall be 172 overlaid, typically one of ``V4L2_FIELD_ANY`` (0), 173 ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or 174 ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different 175 field order and return the actual setting here. 176 177``__u32 chromakey`` 178 When chroma-keying has been negotiated with 179 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field 180 to the desired pixel value for the chroma key. The format is the 181 same as the pixel format of the framebuffer (struct 182 :c:type:`v4l2_framebuffer` ``fmt.pixelformat`` 183 field), with bytes in host order. E. g. for 184 :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should 185 be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host. 186 187``struct v4l2_clip * clips`` 188 When chroma-keying has *not* been negotiated and 189 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, 190 applications can set this field to point to an array of clipping 191 rectangles. 192 193 Like the window coordinates w, clipping rectangles are defined 194 relative to the top, left corner of the frame buffer. However 195 clipping rectangles must not extend the frame buffer width and 196 height, and they must not overlap. If possible applications 197 should merge adjacent rectangles. Whether this must create 198 x-y or y-x bands, or the order of rectangles, is not defined. When 199 clip lists are not supported the driver ignores this field. Its 200 contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` 201 are undefined. 202 203``__u32 clipcount`` 204 When the application set the ``clips`` field, this field must 205 contain the number of clipping rectangles in the list. When clip 206 lists are not supported the driver ignores this field, its contents 207 after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are 208 supported but no clipping is desired this field must be set to zero. 209 210``void * bitmap`` 211 When chroma-keying has *not* been negotiated and 212 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, 213 applications can set this field to point to a clipping bit mask. 214 215It must be of the same size as the window, ``w.width`` and ``w.height``. 216Each bit corresponds to a pixel in the overlaid image, which is 217displayed only when the bit is *set*. Pixel coordinates translate to 218bits like: 219 220 221.. code-block:: c 222 223 ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7)) 224 225where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_ 226 227When a clipping bit mask is not supported the driver ignores this field, 228its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are 229undefined. When a bit mask is supported but no clipping is desired this 230field must be set to ``NULL``. 231 232Applications need not create a clip list or bit mask. When they pass 233both, or despite negotiating chroma-keying, the results are undefined. 234Regardless of the chosen method, the clipping abilities of the hardware 235may be limited in quantity or quality. The results when these limits are 236exceeded are undefined. [#f3]_ 237 238``__u8 global_alpha`` 239 The global alpha value used to blend the framebuffer with video 240 images, if global alpha blending has been negotiated 241 (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see 242 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`, 243 :ref:`framebuffer-flags`). 244 245.. note:: 246 247 This field was added in Linux 2.6.23, extending the 248 structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>` 249 ioctls, which take a pointer to a :c:type:`v4l2_format` 250 parent structure with padding bytes at the end, are not affected. 251 252 253.. c:type:: v4l2_clip 254 255struct v4l2_clip [#f4]_ 256----------------------- 257 258``struct v4l2_rect c`` 259 Coordinates of the clipping rectangle, relative to the top, left 260 corner of the frame buffer. Only window pixels *outside* all 261 clipping rectangles are displayed. 262 263``struct v4l2_clip * next`` 264 Pointer to the next clipping rectangle, ``NULL`` when this is the last 265 rectangle. Drivers ignore this field, it cannot be used to pass a 266 linked list of clipping rectangles. 267 268 269.. c:type:: v4l2_rect 270 271struct v4l2_rect 272---------------- 273 274``__s32 left`` 275 Horizontal offset of the top, left corner of the rectangle, in 276 pixels. 277 278``__s32 top`` 279 Vertical offset of the top, left corner of the rectangle, in pixels. 280 Offsets increase to the right and down. 281 282``__u32 width`` 283 Width of the rectangle, in pixels. 284 285``__u32 height`` 286 Height of the rectangle, in pixels. 287 288 289Enabling Overlay 290================ 291 292To start or stop the frame buffer overlay applications call the 293:ref:`VIDIOC_OVERLAY` ioctl. 294 295.. [#f1] 296 In the opinion of the designers of this API, no driver writer taking 297 the efforts to support simultaneous capturing and overlay will 298 restrict this ability by requiring a single file descriptor, as in 299 V4L and earlier versions of V4L2. Making this optional means 300 applications depending on two file descriptors need backup routines 301 to be compatible with all drivers, which is considerable more work 302 than using two fds in applications which do not. Also two fd's fit 303 the general concept of one file descriptor for each logical stream. 304 Hence as a complexity trade-off drivers *must* support two file 305 descriptors and *may* support single fd operation. 306 307.. [#f2] 308 Should we require ``w.width`` to be a multiple of eight? 309 310.. [#f3] 311 When the image is written into frame buffer memory it will be 312 undesirable if the driver clips out less pixels than expected, 313 because the application and graphics system are not aware these 314 regions need to be refreshed. The driver should clip out more pixels 315 or not write the image at all. 316 317.. [#f4] 318 The X Window system defines "regions" which are vectors of ``struct 319 BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and 320 ``height = y2 - y1``, so one cannot pass X11 clip lists directly.