Android

My first version of the Android generic V4L2 camera HAL had 2 CPU-intensive stages in the camera preview pipeline. The first of them was converting video data from the camera output format (YUV) to an RGB format, suitable for displaying on the framebuffer. The second one is surface compositing onto the framebuffer. I tried to address the former by utilising a VEU (Video Engine Unit) block on SH7372. I used an existing vidix driver for MPlayer as a basis and implemented a Mem2Mem V4L2 VEU driver. Tests under Linux were successful and showed good performance with reduced CPU load. However, making Android to use the driver to convert data from camera output buffers directly into the preview layer buffer turned out to be difficult, because Android uses system memory to allocate virtually-continuous surface buffers. VEU, however, requires physically-continuous buffers for its DMA engine. In recent (4.0-4.1) Android versions this should be doable, using the ION memory manager on top of the Linux dmabuf subsystem. However, ION hasn't yet been integrated in the Android 4.0 version, that I've been using and switching to it would be a major development. As a proof of concept a short-cut has been implemented to force Android to use user-supplied buffers for the camera preview surface. These buffers are obtained from the VEU driver, using the MMAP method and are thus physically-continuous. This indeed increased the capture framerate from about 4 to the full 27 frames per second. However, this still hasn't produces any visible improvements, because software compositing still kept the display framerate below 5 fps. A proper solution would involve using ION for preview buffer management and implementing a hardware-accelerated compositing solution.