Video4Linux2 part 5b: format negotiation

This article is a continuation of the irregular LWN series on writing videodrivers for Linux. The introductory article describes theseries and contains pointers to the previous articles. In the last episode, we looked at how the Video4Linux2 API describes video formats: image sizes andthe representation of pixels within them. This article will complete thediscussion by describing the process of coming to an agreement with anapplication on an actual video format supported by the hardware.

As we saw in the previous article, there are many ways of representingimage data in memory. There is probably no video device on the marketwhich can handle all of the formats understood by the Video4Linuxinterface. Drivers are not expected to support formats not understood bythe underlying hardware; in fact, performing format conversions within thekernel is explicitly frowned upon. So the driver must make it possible forthe application to select a format which works with the hardware.

The first step is to simply allow the application to query the supportedformats. The VIDIOC_ENUM_FMT ioctl() is provided for thepurpose; within the driver this command turns into a call to this callback(if a video capture device is being queried):

    int (*vidioc_enum_fmt_cap)(struct file *file, void *private_data,

			       struct v4l2_fmtdesc *f);

This callback will ask a video capture device to describe one of itsformats. The application will pass in a v4l2_fmtdesc structure:

    struct v4l2_fmtdesc

    {

	__u32		    index;

	enum v4l2_buf_type  type;

	__u32               flags;

	__u8		    description[32];

	__u32		    pixelformat;

	__u32		    reserved[4];

    };

The application will set the index and type fields.index is a simple integer used to identify a format; like theother indexes used by V4L2, this one starts at zero and increases to themaximum number of formats supported. An application can enumerate all ofthe supported formats by incrementing the index value until the driverreturns EINVAL. The type field describes the data streamtype; it will be V4L2_BUF_TYPE_VIDEO_CAPTURE for a video capture(camera or tuner) device.

If the index corresponds to a supported format, the driver shouldfill in the rest of the structure. The pixelformat field shouldbe the fourcc code describing the video representation anddescription a short textual description of the format. The onlydefined value for the flags field isV4L2_FMT_FLAG_COMPRESSED, which indicates a compressed videoformat.

The above callback is for video capture devices; it will only be calledwhen type is V4L2_BUF_TYPE_VIDEO_CAPTURE. TheVIDIOC_ENUM_FMT call will be split out into different callbacksdepending on the type field:

    /* V4L2_BUF_TYPE_VIDEO_OUTPUT */

    int (*vidioc_enum_fmt_video_output)(file, private_date, f);



    /* V4L2_BUF_TYPE_VIDEO_OVERLAY */

    int (*vidioc_enum_fmt_overlay)(file, private_date, f);



    /* V4L2_BUF_TYPE_VBI_CAPTURE */

    int (*vidioc_enum_fmt_vbi)(file, private_date, f);



    /* V4L2_BUF_TYPE_SLICED_VBI_CAPTURE */ */

    int (*vidioc_enum_fmt_vbi_capture)(file, private_date, f);



    /* V4L2_BUF_TYPE_VBI_OUTPUT */

    /* V4L2_BUF_TYPE_SLICED_VBI_OUTPUT */

    int (*vidioc_enum_fmt_vbi_output)(file, private_date, f);



    /* V4L2_BUF_TYPE_VIDEO_PRIVATE */

    int (*vidioc_enum_fmt_type_private)(file, private_date, f);

The argument types are the same for all of these calls. It's worth noting that drivers can support special buffer types with codesstarting with V4L2_BUF_TYPE_PRIVATE, but that would clearlyrequire a special understanding on the application side.For the purposes of this article, we will focus on video capture and outputdevices; the other types of video devices will be examined in futureinstallments.

The application can find out how the hardware is currently configured withthe VIDIOC_G_FMT call. The argument passed in this case is av4l2_format structure:

    struct v4l2_format

    {

	enum v4l2_buf_type type;

	union

	{

		struct v4l2_pix_format		pix;

		struct v4l2_window		win;

		struct v4l2_vbi_format		vbi;

		struct v4l2_sliced_vbi_format	sliced;

		__u8	raw_data[200];

	} fmt;

    };

Once again, type describes the buffer type; the V4L2 layer willsplit this call into one of several driver callbacks depending on thattype. For video capture devices, the callback is:

    int (*vidioc_g_fmt_cap)(struct file *file, void *private_data,

    			    struct v4l2_format *f);

For video capture (and output) devices, the pix field of the unionis of interest. This is the v4l2_pix_format structure seen in theprevious installment; the driver should fill in that structure with thecurrent hardware settings and return. This call should not normally failunless something is seriously wrong with the hardware.

The other callbacks are:

    int (*vidioc_s_fmt_overlay)(file, private_data, f);

    int (*vidioc_s_fmt_video_output)(file, private_data, f);

    int (*vidioc_s_fmt_vbi)(file, private_data, f);

    int (*vidioc_s_fmt_vbi_output)(file, private_data, f);

    int (*vidioc_s_fmt_vbi_capture)(file, private_data, f);

    int (*vidioc_s_fmt_type_private)(file, private_data, f);

The vidioc_s_fmt_video_output() callback uses the samepix field in the same way as capture interfaces do.

Most applications will eventually want to configure the hardware to providea format which works for their purpose. There are two interfaces providedfor changing video formats. The first of these is theVIDIOC_TRY_FMT call, which, within a V4L2 driver, turns into oneof these callbacks:

    int (*vidioc_try_fmt_cap)(struct file *file, void *private_data,

			      struct v4l2_format *f);

    int (*vidioc_try_fmt_video_output)(struct file *file, void *private_data,

			      	       struct v4l2_format *f);

    /* And so on for the other buffer types */

To handle this call,the driver should look at the requested video format and decide whetherthat format can be supported by the hardware or not. If the applicationhas requested something impossible, the driver should return-EINVAL. So, for example, a fourcc code describing an unsupportedformat or a request for interlaced video on a progressive-only device wouldfail. On the other hand, the driver can adjust size fields to match animage size supported by the hardware; normal practice is to adjust sizesdownward if need be. So a driver for a device which only handlesVGA-resolution images would change the width and heightparameters accordingly and return success. The v4l2_formatstructure will be copied back to user space after the call; the drivershould update the structure to reflect any changed parameters so theapplication can see what it is really getting.

The VIDIOC_TRY_FMT handlers are optional for drivers, but omittingthis functionality is not recommended. If provided, this function iscallable at any time, even if the device is currently operating. It shouldnot make any changes to the actual hardware operating parameters; itis just a way for the application to find out what is possible.

When the application wants to change the hardware's format for real, itdoes a VIDIOC_S_FMT call, which arrives at the driver in thisform:

    int (*vidioc_s_fmt_cap)(struct file *file, void *private_data,

    			    struct v4l2_format *f);

    int (*vidioc_s_fmt_video_output)(struct file *file, void *private_data,

    			             struct v4l2_format *f);

Unlike VIDIOC_TRY_FMT, this call cannot be made at arbitrarytimes. If the hardware is currently operating, or if it has streamingbuffers allocated (a topic for yet another future installment), changingthe format could lead to no end of mayhem. Consider what happens, forexample, if the new format is larger than the buffers which are currentlyin use. So the driver should always ensure that the hardware is idle andfail the request (with -EBUSY) if not.

A format change should be atomic - it should change all of the parametersto match the request or none of them. Once again, image size parameterscan be adjusted by the driver if need be. The usual form of thesecallbacks is something like this:

    int my_s_fmt_cap(struct file *file, void *private, 

                     struct v4l2_format *f)

    {

	struct mydev *dev = (struct mydev *) private;

	int ret;



	if (hardware_busy(mydev))

	    return -EBUSY;

	ret = my_try_fmt_cap(file, private, f);

	if (ret != 0)

	    return ret;

	return tweak_hardware(mydev, &f->fmt.pix);

    }

Using the VIDIOC_TRY_FMT handler avoids duplication of code andgets rid of any excuse for not implementing that handler in the firstplace. If the "try" function succeeds, the resulting format is known towork and can be programmed directly into the hardware.

There are a number of other calls which influence how video I/O is done.Future articles will look at some of them. Support for setting formats isenough to enable applications to start transferring images, however, andthat is what the purpose of all this structure is in the end. So the nextarticle, hopefully to come after a shorter delay than happened this timearound, will get into support for reading and writing video data.