GLipKit Information

This page contains some basic information about GLipKit.

1. What is GLipKit?

GLipKit (OpenGL Image Processing Kit) is a library for image processing exploiting programmable 3D graphics hardware. Its main focus is on shape from stereo and structure from motion algorithms. It is a descendent of an experimental epipolar matching procedure running mostly on 3D graphics hardware. This library addresses mainly structure from motion to estimate image disparities (optical flow).

2. What are the basic concepts? What is unique with this approach?

The texturing capability of 3D graphics hardware always allowed image processing tasks like image warping to be done with fast hardware. Vertex programs and especially pixel programs enable accelerated image processing with commodity 3D hardware. Of course, there are limitations: the image resolution is usually bounded by 2kx2k, the precision of color channels is very limited (exception: DX9 hardware) etc. Furthermore, the image processing pipeline can be reordered for better performance. Changing OpenGL contexts between different pixel buffers is rather expensive, therefore reducing the number of context switches is one central goal in this library.

3. Why Objective C?

There are mainly two reasons for implementing GLipKit in Objective C:

1. It must be a language derived from C to use new OpenGL extensions easily without the need to write own language bindings for those extensions.
2. Objective C is better for rapid prototyping than C++ and I like the language. Since MingW supports Objective C, it is possible to use it under Win32. Otherwise I would use plain C.
4. License?

This code is published under the GNU General Public License (see COPYING).

5. Where is the source code?

The source is available at the The SourceForge project page

6. Documentation?

This code is not really documented at all. On the other hand, the source should be relatively easy to understand ;-)

7. Available Algorithms
• A hierachical disparity matcher useful for constrained matching on rectified images (rather fast) and unconstrained matching (aka optical flow, structure from motion, not so fast because of 2D search space).
• A non-hierarchical disparity matcher for comparison. The performance of this matcher can be enhanced...
• NEW: A plane sweep depth estimation procedure closely following the method described in Yang et al: Real-Time Consensus Based Scene Reconstruction Using Commodity Graphics Hardware, Pacific Graphics 2002. Additionally, segmentation data can be used to restrict the depth values to stay inside the visual hull.
• A few classes to visualize disparity maps and other things useful for debugging.
• A few (synthetic) datasets.
8. Near Future Plans?

There are a number of smaller items on the to-do list. A general epipolar matcher working on non-rectified images has high priority and should be rather easy to implement. With my Radeon 9000 hardware, the calculation of epipolar lines for every pixel in the key image will be done as a preprocessing step with the CPU. The epipolar constraints will be encoded in a texture, which is used during matching as a look-up table.

The focus will be on computational stereo in the near future. Image filters will be implemented only as needed.

9. Acknowledgements

The synthetic dino and bowl (schale) datasets were generated with virtual turntable software written by Bernhard Reitinger.

10. Further Links
• The SourceForge project page
• www.gpgpu.org General Programming on GPUs, a link collection to projects and people using programmable 3D hardware for other tasks than rendering.
• www.ati.com The OpenGL SDK from ATI contains a small image processing library based on fragment shaders. There are linear filters and an implementation of an FFT for programmable 3D hardware.
• www.middlebury.edu/~stereo Contains well known datasets used to evaluate stereo reconstruction methods.
• www.mingw.org GCC with Objective C compiler for Windows
• www.gnustep.org GNUstep is a set of Objective C libraries based on the OpenStep standard. Although the GLipKit is independent of GNUstep/OpenStep, this site contains useful links to Objective C related information.