Tri's external memory

V4L2 Driver Development Lessons

2 min read

📚 V4L2 Driver Development Lessons

This document contains detailed, step-by-step lessons on how to build a working Video4Linux2 (V4L2) driver from scratch, with buffer streaming, media controller support, and full platform integration.

🔧 Step 1: Writing a V4L2 Driver from Scratch

✅ Key Concepts:

  • Register v4l2_device and video_device

  • Define file operations: open(), release(), and ioctl()

  • Create /dev/videoX node

🔨 Implementation Highlights:

v4l2_device_register()
video_device_alloc()
video_register_device()
video_device_release()

📂 Required Headers:

#include <media/v4l2-device.h>
#include <media/v4l2-dev.h>

🧠 Step 2: Buffer Management Using vb2

✅ Key Concepts:

  • Setup vb2_queue

  • Define vb2_ops:

    • queue_setup

    • buf_prepare

    • buf_queue

    • start_streaming

    • stop_streaming

  • Enable streaming via VIDIOC_STREAMON/VIDIOC_QBUF

🔨 Implementation Highlights:

vb2_queue_init()
vb2_buffer_done()
video_device.queue = &vb2_queue

📂 Headers:

#include <media/videobuf2-vmalloc.h>

🎥 Step 3: Simulating Frame Streaming

✅ Key Concepts:

  • Use kthread to simulate camera frame generation

  • Feed buffers asynchronously using list_head

  • Fill buffers with dummy data (YUYV 0x80)

🔨 Functions:

kthread_run()
kthread_should_stop()
msleep()
vb2_plane_vaddr()

🔄 Buffer Lifecycle:

  • queue buffer → add to list

  • kthread picks from list → fills → vb2_buffer_done()

🧼 Step 4: Resource Cleanup and Error Handling

✅ Key Concepts:

  • Ensure clean stop of streaming (kthread_stop)

  • Mark any unused buffers as VB2_BUF_STATE_ERROR

  • Clean up during video_unregister_device and v4l2_device_unregister

🔨 Defensive Practices:

if (IS_ERR(kthread)) { mark buffers as ERROR; return; }

🧩 Step 5: Subdevice Integration with Media Controller

✅ Key Concepts:

  • Create a dummy v4l2_subdev for sensor

  • Register with v4l2_device_register_subdev

  • Connect subdevice and video node using media_create_pad_link

📐 Media Topology:

[subdev (source pad)] → [video device (sink pad)]

🔨 Implementation:

v4l2_subdev_init()
subdev.entity.function = MEDIA_ENT_F_CAM_SENSOR;
media_create_pad_link(...);

📁 Step 6: Device Tree and Platform Driver Binding

✅ Key Concepts:

  • Use of_device_id to match Device Tree

  • Bind using platform_driver and platform_get_drvdata()

  • Example compatible string: "tri,v4l2-dummy"

🔧 Device Tree Node:

v4l2_dummy@0 {
    compatible = "tri,v4l2-dummy";
    reg = <0x0 0x1000>;
    status = "okay";
};

🔨 Driver Structure:

.driver = {
    .name = "v4l2-dummy",
    .of_match_table = dummy_dt_ids,
}

🧪 Step 7: Testing and Debugging

✅ Key Tools:

  • media-ctl: inspect media graph

  • v4l2-ctl: query formats and stream buffers

  • mpv, GStreamer: display frames

  • dmesg, strace: debug driver output

📦 Test Commands:

v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=10 --stream-to=test.yuv
media-ctl -p -d /dev/media0

🧠 Buffer Visualization:

hexdump -C test.yuv | head

🎞️ Real-Time Playback:

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink

✅ You now have a fully functional V4L2 platform driver

Includes:

  • Media graph integration

  • Streaming with buffer queuing

  • Device Tree compatibility

  • Userspace testing tools

Next Topics (Optional): V4L2 format negotiation, CSI integration, I2C subdevice migration, video cropping/scaling support.

Graph View

Backlinks