Sensing

The “Sensing” section serves as an introduction to a broad spectrum of sensory technologies critical for robotic perception, navigation, and interaction. It outlines tools and techniques that empower robots to effectively sense their surroundings and make informed decisions.

This section dives into various sensing modalities such as GPS modules, fiducial markers, radar, thermal cameras, stereo vision, and many more. It provides details on their principles of operation, implementation, and practical use cases, along with helpful troubleshooting tips. Whether you’re working on autonomous navigation, precise localization, object detection, or environmental mapping, this section is designed to equip you with the foundational knowledge and resources to integrate these sensors into your projects.

Key Subsections and Highlights

  • Adafruit GPS: Discusses the Adafruit Ultimate GPS module for precise localization, including its features, technical specifications, and usage tips.

  • AprilTags: Introduces AprilTags as a fiducial marker system for visual identification and 6D pose estimation. Explains their use in robotics and potential pitfalls.

  • Azure Block Detection: Outlines the use of the Azure Kinect for object detection, with applications like Jenga block assembly. Explains the step-by-step detection pipeline.

  • Camera Calibration: Emphasizes the importance of calibrating cameras for minimizing errors and improving vision system accuracy. Includes references to key calibration resources.

  • IMU-Camera Calibration using Kalibr: Details the Kalibr library for simultaneous IMU and camera calibration, including example setups and tips for accurate calibration.

  • Computer Vision for Robotics – Practical Considerations: Highlights key considerations when deploying computer vision in robotics, including lighting, frame rates, calibration, and error mitigation.

  • Delphi ESR Radar: Provides an overview of Delphi’s ESR radar for detecting objects and estimating their range, speed, and position.

  • Comparison of Fiducial Markers: Compares various fiducial marker systems like ArUco, AprilTags, and STag, listing their pros, cons, and ideal use cases.

  • Using an RTK GPS: Explains how to achieve centimeter-level accuracy using RTK GPS systems, along with practical lessons and setup guidance.

  • OpenCV Stereo Vision Processing: Introduces OpenCV libraries for stereo vision, including camera calibration and 3D triangulation.

  • Point Cloud Library (PCL), 3D Sensors and Applications: Discusses PCL’s features for processing 3D point clouds and its applications in object detection, segmentation, and mapping.

  • Photometric Calibration: Explains the need for calibrating camera sensors to accurately map light intensity to pixel values, and methods to achieve this.

  • Realsense RGB-D Camera: Introduces Intel RealSense cameras and details SDK installation, ROS integration, calibration, and tuning methods.

  • Robotic Total Station (Leica TS16): Discusses the use of robotic total stations for high-precision 3D positioning and their applications in surveying and robotics.

  • Speech Recognition: Explores speech recognition as a robotic interface, including offline and online solutions, wakeword detection, and speech synthesis.

  • STag: Presents the STag fiducial marker system for stable pose estimation, with details on implementation and experimental results.

  • Thermal Cameras: Examines the use of thermal cameras in robotics, including types of thermal cameras, calibration techniques, and debug tips.

  • Tracking Vehicles Using a Static Traffic Camera: Describes a system for extracting vehicle trajectories using static traffic cameras, incorporating detection, tracking, and homography estimation.

  • DWM1001 UltraWideband Positioning System: Covers the setup and calibration of the DWM1001 UWB system for accurate indoor positioning.

Resources