Kinect Calibration ================== Introduction ------------ Kinect has been on the market for 8 years. First generation Kinect 1 is based on the theory of `Structured Light `_, and second generation Kinect 2 is based on the theory of `Time of Flight `_. Kinect 1 and Kinect 2 respectively look as in the following picture (cited from imgur). .. image:: http://i.imgur.com/QqSOlWC.jpg :align: center :target: http://i.imgur.com/QqSOlWC.jpg :alt: Kinect 1 vs. Kinect 2 Theories of how to calibrate a depth sensor has been widely discussed in state-of-the-art publications, such as: - `Easy Depth Sensor Calibration `_ - `Robust Intrinsic and Extrinsic Calibration of RGB-D Cameras `_, etc. Demonstrations -------------- Preparation ^^^^^^^^^^^ Here, we're not discussing the background theories of RGB-D sensor calibration, but using `MRPT `_ to calibrate a Kinect 1 directly. .. code-block:: bash ➜ ~ lsusb ...... Bus 001 Device 014: ID 045e:02ae Microsoft Corp. Xbox NUI Camera Bus 001 Device 012: ID 045e:02b0 Microsoft Corp. Xbox NUI Motor Bus 001 Device 013: ID 045e:02ad Microsoft Corp. Xbox NUI Audio Bus 001 Device 011: ID 0409:005a NEC Corp. HighSpeed Hub ...... Calibration Kinect Using MRPT ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Details about **how to calibrate a Kinect** can be found on `MRPT's tutorial Kinect Calibration `_. .. code-block:: bash ➜ ~ sudo kinect-stereo-calib [sudo] password for longervision: (kinect-stereo-calib:3602): IBUS-WARNING **: 20:07:51.836: The owner of /home/longervision/.config/ibus/bus is not root! (kinect-stereo-calib:3602): Gtk-CRITICAL **: 20:07:51.956: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton ...... Calling CKinect::initialize()...Calling CKinect::initialize()...OK OK ...... Intermediate Process ^^^^^^^^^^^^^^^^^^^^ .. list-table:: * - .. image:: ../resource/chapter2/images/kinect/mrpt/01_mrpt_kinect_stereo_calibration_welcome.png :align: center :target: ../resource/chapter2/images/kinect/mrpt/01_mrpt_kinect_stereo_calibration_welcome.png :alt: MRPT Welcome To Kinect & Stereo Camera Calibration - .. image:: ../resource/chapter2/images/kinect/mrpt/02_mrpt_instructions.png :align: center :target: ../resource/chapter2/images/kinect/mrpt/02_mrpt_instructions.png :alt: MRPT Instructions * - .. image:: ../resource/chapter2/images/kinect/mrpt/03_mrpt_testconnection.png :align: center :target: ../resource/chapter2/images/kinect/mrpt/03_mrpt_testconnection.png :alt: MRPT Test Connection - .. image:: ../resource/chapter2/images/kinect/mrpt/04_mrpt_connected.png :align: center :target: ../resource/chapter2/images/kinect/mrpt/04_mrpt_connected.png :alt: MRPT Connected * - .. image:: ../resource/chapter2/images/kinect/mrpt/05_mrpt_capture_IR.png :align: center :target: ../resource/chapter2/images/kinect/mrpt/05_mrpt_capture_IR.png :alt: MRPT Capture IR - .. image:: ../resource/chapter2/images/kinect/mrpt/06_mrpt_capture_RGB_buggy.png :align: center :target: ../resource/chapter2/images/kinect/mrpt/06_mrpt_capture_RGB_buggy.png :alt: MRPT Capture RGB Buggy * - .. image:: ../resource/chapter2/images/kinect/mrpt/07_mrpt_kinect_stereo_calibration.png :align: center :target: ../resource/chapter2/images/kinect/mrpt/07_mrpt_kinect_stereo_calibration.png :alt: MRPT Capture Kinect & Stereo Camera Calibration - .. image:: ../resource/chapter2/images/kinect/mrpt/08_mrpt_run_optimization.png :align: center :target: ../resource/chapter2/images/kinect/mrpt/08_mrpt_run_optimization.png :alt: MRPT Capture Run Optimization Finally, click **Save calibration**, you'll save your calibration data into a file with default name **calib.ini**. For my Kinect 1, I obtained the following result: .. code-block:: bash ➜ ~ cat calib.ini # Stereo camera calibration report # Generated by kinect-stereo-calib - MRPT at Tue Jan 28 20:20:36 2020 # (This file is loadable from rawlog-edit and other MRPT tools) # --------------------------------------------------------------------- # Left camera (IR/Depth in Kinect) calibration parameters (and 95% confidence intervals): [CAMERA_PARAMS_LEFT] resolution = [640 488] cx = 293.491411 // +/- 0.050 cy = 231.095236 // +/- 0.050 fx = 518.022798 // +/- 0.248 fy = 520.176112 // +/- 0.202 dist = [-8.236128e-07 -5.520703e-07 0.000000e+00 0.000000e+00 0.000000e+00] // The order is: [K1 K2 T1 T2 K3] # Right camera (RGB in Kinect) calibration parameters (and 95% confidence intervals): [CAMERA_PARAMS_RIGHT] resolution = [640 480] cx = 453.990585 // +/- 0.050 cy = 372.857445 // +/- 0.050 fx = 509.606926 // +/- 0.250 fy = 509.452555 // +/- 0.110 dist = [5.007498e-07 1.003845e-06 0.000000e+00 0.000000e+00 0.000000e+00] // The order is: [K1 K2 T1 T2 K3] # Relative pose of the right camera wrt to the left camera: # This assumes that both camera frames are such that +Z points # forwards, and +X and +Y to the right and downwards. [CAMERA_PARAMS_LEFT2RIGHT_POSE] translation_only = [4.149084e-01 1.468625e-01 -9.246930e-02] rotation_matrix_only = [9.9604425735067e-01 1.5786321022517e-02 -8.7445008246997e-02 ;-8.5960299502565e-03 9.9659520633124e-01 8.2000627965236e-02 ;8.8441764273660e-02 -8.0924574674031e-02 9.9278863185775e-01 ] pose_yaw_pitch_roll = [0.414908 0.146862 -0.092469 -0.494460 -5.073969 -4.660013] // (YPR in degrees) pose_quaternion = [0.414908 0.146862 -0.092469 0.998177 -0.040806 -0.044052 -0.006107] # Info about calibration parameters: [CALIB_METAINFO] number_good_cheesboards = 1 average_reprojection_error = 1.190948 // RMSE pixels cheesboard_nx = 7 cheesboard_ny = 9 cheesboard_square_size_x = 0.040000 // (m) cheesboard_square_size_y = 0.040000 // (m) Intermediate Images ^^^^^^^^^^^^^^^^^^^ And, the intermediate 9 pairs of RGB and IR images are respectively captured as: .. list-table:: :align: center * - **IR** - **RGB** * - .. image:: ../resource/chapter2/images/kinect/imgPairs/left_0000.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/left_0000.png :alt: 0000 IR - .. image:: ../resource/chapter2/images/kinect/imgPairs/right_0000.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/right_0000.png :alt: 0000 RGB * - .. image:: ../resource/chapter2/images/kinect/imgPairs/left_0001.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/left_0001.png :alt: 0001 IR - .. image:: ../resource/chapter2/images/kinect/imgPairs/right_0001.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/right_0001.png :alt: 0001 RGB * - .. image:: ../resource/chapter2/images/kinect/imgPairs/left_0002.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/left_0002.png :alt: 0002 IR - .. image:: ../resource/chapter2/images/kinect/imgPairs/right_0002.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/right_0002.png :alt: 0002 RGB * - .. image:: ../resource/chapter2/images/kinect/imgPairs/left_0003.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/left_0003.png :alt: 0003 IR - .. image:: ../resource/chapter2/images/kinect/imgPairs/right_0003.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/right_0003.png :alt: 0003 RGB * - .. image:: ../resource/chapter2/images/kinect/imgPairs/left_0004.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/left_0004.png :alt: 0004 IR - .. image:: ../resource/chapter2/images/kinect/imgPairs/right_0004.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/right_0004.png :alt: 0004 RGB * - .. image:: ../resource/chapter2/images/kinect/imgPairs/left_0005.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/left_0005.png :alt: 0005 IR - .. image:: ../resource/chapter2/images/kinect/imgPairs/right_0005.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/right_0005.png :alt: 0005 RGB * - .. image:: ../resource/chapter2/images/kinect/imgPairs/left_0006.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/left_0006.png :alt: 0006 IR - .. image:: ../resource/chapter2/images/kinect/imgPairs/right_0006.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/right_0006.png :alt: 0006 RGB * - .. image:: ../resource/chapter2/images/kinect/imgPairs/left_0007.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/left_0007.png :alt: 0007 IR - .. image:: ../resource/chapter2/images/kinect/imgPairs/right_0007.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/right_0007.png :alt: 0007 RGB * - .. image:: ../resource/chapter2/images/kinect/imgPairs/left_0008.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/left_0008.png :alt: 0008 IR - .. image:: ../resource/chapter2/images/kinect/imgPairs/right_0008.png :align: center :target: ../resource/chapter2/images/kinect/imgPairs/right_0008.png :alt: 0008 RGB Assignments ----------- - Calibrate one of the following sensors: * `Kinect 2 `_ * `Intel Real Sense D435 `_ * `Structure Sensor `_