Pose Estimation For Monocular Vision Cameras ============================================ Extrinsic Parameters -------------------- With the calibration parameters (namely, camera `intrinsic parameters `_), by providing any type of calibration board as the reference, the camera posture relative to this calibration board should be able to be accurately estimated. If the calibration board's position in the world frame is known, the camera's `extrinsic parameters `_ (namely, the camera's position relative to the world frame) can be further calculated. `ArUco markers `_ are often used to estimate camera poses. Since we're going to test on `JeVois camera `_ with its `120-degree fisheye lens `_ again, a variety of `4x4_50 ArUco markers `_ provided on `JeVois Demo ArUco `_ are recommended from us. You are also welcome to use other online applications to generate various types of ArUco markers, such as: `ArUco markers generator `_. Demonstration ------------- Preparation ^^^^^^^^^^^ Dataset """"""" In total, ``371`` images are captured consecutively and save on host PC, using `jevois_streaming.py <./03_calibrate_monocular_vision_cameras_with_fisheye_lens.html#code-snippet-jevois-streaming-py>`_. 2 groups of images are carefully selected to demonstrate the feasibility of our implementation. - A consecutive sequence of images indexed from ``015`` to ``026``. - A carefully selected bunch of images with arbitrary indexes. ArUco Marker 6x6_250-0 """""""""""""""""""""" We use a single ArUco marker 6x6_250-0 in our experiments. .. image:: ../resource/chapter2/images/markers/ArUco/6x6_250-0.jpg :align: center :target: ../resource/chapter2/images/markers/ArUco/6x6_250-0.svg :alt: 6x6_250-0.jpg - dictionary: 6x6 - marker ID: 0 - size: 250mm , which is generated from `ArUco markers generator `_ and displayed on an monitor at percentage 100%. Code Snippet: posture_estimation.py """""""""""""""""""""""""""""""""""" .. literalinclude:: ../resource/chapter2/code/posture_estimation.py :language: python3 :emphasize-lines: 52,61,72-73,77,79-81 :linenos: Demon 1: JeVois Fisheye Pose Estimation Based On ArUco Marker Detection ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Original Captured Images """""""""""""""""""""""" - Sequence 1: index from 15 to 26 .. list-table:: * - .. image:: ../resource/chapter2/images/posture/original/img051.jpg :align: center :target: ../resource/chapter2/images/posture/original/img051.jpg :alt: img051.jpg - .. image:: ../resource/chapter2/images/posture/original/img052.jpg :align: center :target: ../resource/chapter2/images/posture/original/img052.jpg :alt: img052.jpg - .. image:: ../resource/chapter2/images/posture/original/img053.jpg :align: center :target: ../resource/chapter2/images/posture/original/img053.jpg :alt: img053.jpg - .. image:: ../resource/chapter2/images/posture/original/img054.jpg :align: center :target: ../resource/chapter2/images/posture/original/img054.jpg :alt: img054.jpg * - .. image:: ../resource/chapter2/images/posture/original/img055.jpg :align: center :target: ../resource/chapter2/images/posture/original/img055.jpg :alt: img055.jpg - .. image:: ../resource/chapter2/images/posture/original/img056.jpg :align: center :target: ../resource/chapter2/images/posture/original/img056.jpg :alt: img056.jpg - .. image:: ../resource/chapter2/images/posture/original/img057.jpg :align: center :target: ../resource/chapter2/images/posture/original/img057.jpg :alt: img057.jpg - .. image:: ../resource/chapter2/images/posture/original/img058.jpg :align: center :target: ../resource/chapter2/images/posture/original/img058.jpg :alt: img058.jpg * - .. image:: ../resource/chapter2/images/posture/original/img059.jpg :align: center :target: ../resource/chapter2/images/posture/original/img059.jpg :alt: img059.jpg - .. image:: ../resource/chapter2/images/posture/original/img060.jpg :align: center :target: ../resource/chapter2/images/posture/original/img060.jpg :alt: img060.jpg - .. image:: ../resource/chapter2/images/posture/original/img061.jpg :align: center :target: ../resource/chapter2/images/posture/original/img061.jpg :alt: img061.jpg - .. image:: ../resource/chapter2/images/posture/original/img062.jpg :align: center :target: ../resource/chapter2/images/posture/original/img062.jpg :alt: img062.jpg - Sequence 2: ``indexes = 6 + 30*i`` .. list-table:: * - .. image:: ../resource/chapter2/images/posture/original/img006.jpg :align: center :target: ../resource/chapter2/images/posture/original/img006.jpg :alt: img006.jpg - .. image:: ../resource/chapter2/images/posture/original/img036.jpg :align: center :target: ../resource/chapter2/images/posture/original/img036.jpg :alt: img036.jpg - .. image:: ../resource/chapter2/images/posture/original/img066.jpg :align: center :target: ../resource/chapter2/images/posture/original/img066.jpg :alt: img066.jpg - .. image:: ../resource/chapter2/images/posture/original/img096.jpg :align: center :target: ../resource/chapter2/images/posture/original/img096.jpg :alt: img096.jpg * - .. image:: ../resource/chapter2/images/posture/original/img126.jpg :align: center :target: ../resource/chapter2/images/posture/original/img126.jpg :alt: img126.jpg - .. image:: ../resource/chapter2/images/posture/original/img156.jpg :align: center :target: ../resource/chapter2/images/posture/original/img156.jpg :alt: img156.jpg - .. image:: ../resource/chapter2/images/posture/original/img186.jpg :align: center :target: ../resource/chapter2/images/posture/original/img186.jpg :alt: img186.jpg - .. image:: ../resource/chapter2/images/posture/original/img216.jpg :align: center :target: ../resource/chapter2/images/posture/original/img216.jpg :alt: img216.jpg * - .. image:: ../resource/chapter2/images/posture/original/img246.jpg :align: center :target: ../resource/chapter2/images/posture/original/img246.jpg :alt: img246.jpg - .. image:: ../resource/chapter2/images/posture/original/img276.jpg :align: center :target: ../resource/chapter2/images/posture/original/img276.jpg :alt: img276.jpg - .. image:: ../resource/chapter2/images/posture/original/img306.jpg :align: center :target: ../resource/chapter2/images/posture/original/img306.jpg :alt: img306.jpg - .. image:: ../resource/chapter2/images/posture/original/img336.jpg :align: center :target: ../resource/chapter2/images/posture/original/img336.jpg :alt: img336.jpg Intermediate Images: Undistorted Gray Images """""""""""""""""""""""""""""""""""""""""""" Clearly, all 371 original images will get undistorted as the intermediate results. - Sequence 1: index from 51 to 62 .. list-table:: * - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray051.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray051.jpg :alt: remappedgray051.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray052.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray052.jpg :alt: remappedgray052.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray053.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray053.jpg :alt: remappedgray053.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray054.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray054.jpg :alt: remappedgray054.jpg * - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray055.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray055.jpg :alt: remappedgray055.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray056.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray056.jpg :alt: remappedgray056.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray057.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray057.jpg :alt: remappedgray057.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray058.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray058.jpg :alt: remappedgray058.jpg * - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray059.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray059.jpg :alt: remappedgray059.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray060.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray060.jpg :alt: remappedgray060.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray061.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray061.jpg :alt: remappedgray061.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray062.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray062.jpg :alt: remappedgray062.jpg - Sequence 2: ``indexes = 6 + 30*i`` .. list-table:: * - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray006.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray006.jpg :alt: remappedgray006.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray036.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray036.jpg :alt: remappedgray036.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray066.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray066.jpg :alt: remappedgray066.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray096.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray096.jpg :alt: remappedgray096.jpg * - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray126.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray126.jpg :alt: remappedgray126.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray156.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray156.jpg :alt: remappedgray156.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray186.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray186.jpg :alt: remappedgray186.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray216.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray216.jpg :alt: remappedgray216.jpg * - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray246.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray246.jpg :alt: remappedgray246.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray276.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray276.jpg :alt: remappedgray276.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray306.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray306.jpg :alt: remappedgray306.jpg - .. image:: ../resource/chapter2/images/posture/remapped/remappedgray336.jpg :align: center :target: ../resource/chapter2/images/posture/remapped/remappedgray336.jpg :alt: remappedgray336.jpg Results: Posture Axis Drawn """"""""""""""""""""""""""" Among all 371 test images, only 284 camera postures have been successfully estimated. That is to say, among all 371 images, the ArUco marker has been successfully detected only in 284 images. - Sequence 1: index from 51 to 62 .. list-table:: * - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn022_051.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn022_051.jpg :alt: posturedrawn022_051.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn023_052.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn023_052.jpg :alt: posturedrawn023_052.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn024_053.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn024_053.jpg :alt: posturedrawn024_053.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn025_054.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn025_054.jpg :alt: posturedrawn025_054.jpg * - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn026_055.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn026_055.jpg :alt: posturedrawn026_055.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn027_056.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn027_056.jpg :alt: posturedrawn027_056.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn028_057.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn028_057.jpg :alt: posturedrawn028_057.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn029_058.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn029_058.jpg :alt: posturedrawn029_058.jpg * - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn030_059.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn030_059.jpg :alt: posturedrawn030_059.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn031_060.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn031_060.jpg :alt: posturedrawn031_060.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn032_061.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn032_061.jpg :alt: posturedrawn032_061.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn033_062.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn033_062.jpg :alt: posturedrawn033_062.jpg - Sequence 2: ``indexes = 6 + 30*i`` .. list-table:: * - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn001_006.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn001_006.jpg :alt: posturedrawn001_006.jpg - .. image:: ../resource/not-found-image-15383864787lu.jpg :align: center :target: ../resource/not-found-image-15383864787lu.jpg :alt: NOT Detected - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn037_066.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn037_066.jpg :alt: posturedrawn037_066.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn067_096.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn067_096.jpg :alt: posturedrawn067_096.jpg * - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn097_126.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn097_126.jpg :alt: posturedrawn097_126.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn127_156.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn127_156.jpg :alt: posturedrawn127_156.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn157_186.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn157_186.jpg :alt: posturedrawn157_186.jpg - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn187_216.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn187_216.jpg :alt: posturedrawn187_216.jpg * - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn211_246.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn211_246.jpg :alt: posturedrawn211_246.jpg - .. image:: ../resource/not-found-image-15383864787lu.jpg :align: center :target: ../resource/not-found-image-15383864787lu.jpg :alt: NOT Detected - .. image:: ../resource/not-found-image-15383864787lu.jpg :align: center :target: ../resource/not-found-image-15383864787lu.jpg :alt: NOT Detected - .. image:: ../resource/chapter2/images/posture/posturedrawn/posturedrawn251_336.jpg :align: center :target: ../resource/chapter2/images/posture/posturedrawn/posturedrawn251_336.jpg :alt: posturedrawn251_336.jpg Assignments ----------- Please try to estimate camera posture using: - a variety of markers: * `A circle gird <./02_calibrate_monocular_vision_cameras_with_narrow_angle_lens.html#asymmetrical-circle-pattern>`_: Refer to my blog `Camera Posture Estimation Using Circle Grid Pattern `_ * `An ChArUco diamond marker `_: Refer to my blog `Camera Posture Estimation Using An ArUco Diamond Marker `_ * `An ArUco board `_: Refer to my blog `Camera Posture Estimation Using An ArUco Board `_ * `A ChArUco board `_: Refer to my blog `Camera Posture Estimation Using A ChArUco Board `_