A V4L2 - Media Controller mini summit has taken place from 14 to 16 of June 2010 in Helsinki, Finland. The
agenda and the list of participants were both quite impressive. On the first day we have not managed to go through all the planned talks. We had to postpone three of them to Tuesday. This left us less time to discuss single topics in depth, but we managed that too on Tuesday afternoon. On Wednesday, according to the plan, we once again went over all the topics to make conclusions and plan further actions for each of them. Presentation slides and, probably, some summary will be posted on the
linuxtv web site, below I'll just post a few short notes about aspects from some of the topics, that I personally found especially interesting. But before doing that I'd like to thank
Nokia for hosting this event and for sponsoring my trip and accommodation.
Qualcomm video architecture
Qualcomm presented their video hardware architecture and their current software solutions and future plans. They are working on a V4L2 solution for their hardware for all mobile Linux flavors like WebOS, Bada, Chrome OS, and want to use it with Android as their standard video API too.
ST-Ericsson video architecture
Also
ST-Ericsson is trying to base their Linux multimedia stack on V4L2 and fbdev for video output. They presented their in-house solution for handling of video buffers, which should be helpful for designing the next version of the videobuf API (more about this below).
Media Controller, libv4l, removal of V4L1
Media Controller is a new API for the Linux Kernel, designed to support dynamically reconnectable hardware blocks. This API introduces notions of
entities,
pads, and
links. Entities have input (sink) and output (source) pads, which can be used to connect them, using links. This API is being developed in the V4L2 context, but is kept generic and can be used in any other Linux kernel subsystem. Applied to V4L, entities can represent sensors, displays, video processing units. Its introduction will allow dynamic rerouting of video data between source, processing and sink nodes, and targeted configuration of each entity and their pads.
Since entities are hardware specific, their configuration cannot be performed by generic software. This task should be carried out by dynamically loadable modules for the libv4l library. To support such modules a plugin API will be added to libv4l. It shall also be possible to use additional hardware-specific libraries and programs to interact with plugins.
Libv4l has also recently been extended to support legacy V4L1 applications completely in user-space without any V4L1 support in the kernel. This will allow to deprecate V4L1 in the kernel and remove it after a certain grace period.
soc-camera status
I gave a short overview of the current soc-camera framework status and next steps on the way of making it fully v4l2-subdev compliant. It has been discussed, whether the soc-camera specific way of configuration of media bus parameters should be implemented for each driver separately, using custom platform data, or a standard set of bus configuration parameters shall be introduced. Although the current plan is to have each driver configure its hardware interface independently, which means, that standardisation is not required, it has been decided to create a standard set of such parameters and a standard struct, that shall be embedded in larger hardware-specific types.
fbdev to V4L2 output translation
I proposed to create a wrapper for V4L2 output drivers, that would allow exporting of a framebuffer driver API to the user-space. Many found the idea interesting, but not interesting enough to invest their time and effort into its implementation.
Driver compliance, reference programs
Ways to test V4L drivers for standard compliance have been discussed. Also a list of user-space programs, that should "officially be supported" by V4L has been proposed.
Remote controllers
Recently a new infrared remote controller framework in the Linux kernel has been developed and has received a lot of attention. Ways to support various IR control code tables and HDMI (CEC) commands have been discussed.
Status of Intel V4L drivers
The patch series for Intel's Moorestown Langwell camera subsystem contains support for the MIPI CSI interface and MIPI sensors, which seems to be the first implementation of this standard in the Linux kernel.
Status of Texas Instruments drivers
TI OMAP video drivers - this is where the whole Media Controller development is taking place. Davinci SoCs have also been discussed.
Multiplanar support
Some hardware types require planes in planar video formats to be stored in separate (aligned?) memory buffers. Currently V4L cannot support such buffers. It is also unclear what is the best way to implement this feature. It has to be further discussed on the mailing list.
videobuf2
This, probably, has been the most complex topic of the mini-summit. Recently a number of bugs, design flaws, and artificial limitations have been discovered in V4L video buffer (videobuf) implementation. Incremental fixing of the current implementation shall be attempted, although a complete new reimplementation might seem easier and more meaningful.
Last but not least - have a look at this
post by Hans Verkuil for a more detailed report of the summit.
