Microsoft Kinect Overview
Kinect for Xbox 360 is a low-cost vision device equipped with one IR camera, one color camera and one IR projector to produce RGB images as well as voxel (depth-pixel) images. The RGB video stream gives a 8-bit VGA resolution (640 x 480 pixels) with a Bayer color filter, while th monochrome depth-sensing video stream is in VGA resolution. The sensor has an angular field of view of 57 degrees horizontally and 43 degrees vertically. Kinect has been reverse engineered to a great extent by the open source community which has revealed many facts on how depth is measured. Kinect uses a structured light approach form which we can extract time of return. They use a standard off-the shellf CMOS sensor for the same.
Libraries for MS Kinect Interfacing in Ubuntu
There are many different open source libraries that can be chosen from for interfacing with Kinect in Ubuntu. Kinect for Windows provides direct interfacing for Windows based computers. The libraries for Ubuntu are:
OpenNI and OpenNI2
- The older version of Kinect supports openNI whereas the newer version of Kinect uses openNI2. The installation and usaage of openNI2 as as a standalone library can be found here: http://structure.io/openni
libfreenect and libfreenect2
libfreenect is also a reliable library that can be used and in my experience have proven to be more reliable than openNI. The only drawback is that while openNI and openNI2 can be used for other sensors such as Asus Xtion Pro or Orbecc Astra depth camera, libfreenect is mostly suited for Kinct only. Useful information on getting started with libfreenect can be found here: https://github.com/OpenKinect/libfreenect
The full distribution of ROS also includes these libraries by default and the documentation for that can be found at the following links:
- General libfreenect: http://wiki.ros.org/libfreenect
- freenect_launch: http://wiki.ros.org/freenect_launch
freenect_launchto access depth data stream which can be visualized on RVIZ.
- openni_launch: http://wiki.ros.org/openni_launch
- Simiar to libfreenect_launch
PCL is a large scale open source library for processing 2D and 3D images and point cloud processing. It is a state of the art library used in most perception related projects. PCL has an extensive documentation and ready to use examples for segmentation, recognition and filtering. It has data structures for kdtree, octree and pointcloud arrays which allows for very fast processing and efficient implementation. A complete documentation for PCL can be found on their official website.
Using VocelGrid filter pointcloud can be initially downsampled and further sparsification of Kinect data can be done using PassThrough filters. After these basic filtering, you can further perform clustering, cylinderical, or planar segmentation in real time applications. The PCL tutorials are helpful even if you are an expert and can be found here.
Installing PCL from source and ros_pcl
Although PCL comes installed with ROS full installation by default, a complete installation of PCL from source mught be required in some circumstances alongwith CUDA for GPU processing. An exhaustive tutorial for the same and also shows how to install openNI can be found here.
This Masters thesis is a complete guide for using MS Kinect for navigation.