|
|
|
|
| Recently, there has been increased interest in the usage of streaming video textures in 3D applications. Streaming video textures are textures that continously change, mostly based on media data stored in a common video format. The demo movies below demonstrate how the media component framework GStreamer and OpenGL 1.5 for Nvidia cards can be used to create surprising real-time video effects in Linux.
Figure 1. Video rendered as 3D pixels, each pixel is offset
from its 2D video position, based on its luminance value.
Technical backgroundGstreamer provides YUV planes of the video frames that are uploaded as 3 luminance only textures (using Pixel Buffer Objects, PBO).Instead of drawing a simple quad with these textures, an array of point primitives is being generated, one for each video pixel. A Vertex Buffer Object (VBO), bound to the same memory area as the PBO, holds the original 2D positions of the points, and grants access to the Y (luminance) values. A vertex shader reads the point array, and displaces them according to the provided Y values. The following fragment shader reconstructs the RGB pixel colors from the 3 given luminance textures, using a commonly known YUV->RGB conversion formula.
It is important to note that this video effect is not the only one achievable with this technology. The sourcecode is available on request.
Comments and ideas are very welcome! |
