diff --git a/CMakeLists.txt b/CMakeLists.txt index a9c1c3d..59d47aa 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -14,7 +14,7 @@ cmake_minimum_required(VERSION 3.0) -project(mynteye VERSION 2.3.6 LANGUAGES C CXX) +project(mynteye VERSION 2.3.7 LANGUAGES C CXX) include(cmake/Common.cmake) diff --git a/README.md b/README.md index f1e4585..c789396 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # MYNT® EYE S SDK -[![](https://img.shields.io/badge/MYNT%20EYE%20S%20SDK-2.3.6-brightgreen.svg?style=flat)](https://github.com/slightech/MYNT-EYE-S-SDK) +[![](https://img.shields.io/badge/MYNT%20EYE%20S%20SDK-2.3.7-brightgreen.svg?style=flat)](https://github.com/slightech/MYNT-EYE-S-SDK) ## Overview diff --git a/docs/conf.py b/docs/conf.py index 8f904d3..2dcca99 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -24,7 +24,7 @@ copyright = '2018, MYNTAI' author = 'MYNTAI' # The short X.Y version -version = '2.3.6' +version = '2.3.7' # The full version, including alpha/beta/rc tags release = version diff --git a/docs/src/sdk/changelog.rst b/docs/src/sdk/changelog.rst index be48e62..db04c80 100644 --- a/docs/src/sdk/changelog.rst +++ b/docs/src/sdk/changelog.rst @@ -3,6 +3,12 @@ Changelog ========= +2019-04-19(v2.3.7) +------------------- +1. Improve VINS-Fusion supporting +2. Improve ORB-SLAM2 supporting + + 2019-04-15(v2.3.6) ------------------- diff --git a/docs/src/slam/contents.rst b/docs/src/slam/contents.rst index 10763dd..aa5b82b 100644 --- a/docs/src/slam/contents.rst +++ b/docs/src/slam/contents.rst @@ -5,7 +5,6 @@ Open Source project Support .. toctree:: - how_to_use_kalibr vins vins_fusion orb_slam2 diff --git a/docs/src/slam/how_to_use_kalibr.rst b/docs/src/slam/how_to_use_kalibr.rst deleted file mode 100644 index fde25af..0000000 --- a/docs/src/slam/how_to_use_kalibr.rst +++ /dev/null @@ -1,300 +0,0 @@ -.. _how_to_use_kalibr: - -How to calibrate MYNTEYE by kalibr -=================================== - -Target ------------- - -* Calibrate the pose relationship between left and right camera - -* Calibrate the pose relationship left camera between and IMU - - -Preparation ------------- - -* **Install kalibr**:Refer to `kalibr wiki `_ and follow the steps to install - -* **Calibration board**： kalibr supports ``chessbord`` , ``circlegrid`` , ``aprilgrid`` ,choose ``aprilgrid`` here Calibration board file can be directly `download `_ , Or you can also generate calibration board by Kalibr tool. - -.. code-block:: bash - - $ kalibr_create_target_pdf --type 'apriltag' --nx 6 --ny 6 --tsize 0.08 --tspace 0.3 - -View parameters' meanings by kalibr_create)target_pdf command: - -.. code-block:: bash - - $ kalibr_create_target_pdf --h - usage: - Example Aprilgrid: - kalibr_create_target_pdf --type apriltag --nx 6 --ny 6 --tsize 0.08 --tspace 0.3 - Example Checkerboard: - kalibr_create_target_pdf --type checkerboard --nx 6 --ny 6 -csx 0.05 --csy 0.1 - - - Generate PDFs of calibration patterns. - - optional arguments: - -h, --help show this help message and exit - - Output options: - output Output filename - --eps Also output an EPS file - - Generic grid options: - --type GRIDTYPE The grid pattern type. ('apriltag' or 'checkerboard') - --nx N_COLS The number of tags in x direction (default: 6) - --ny N_ROWS The number of tags in y direction (default: 7) - - Apriltag arguments: - --tsize TSIZE The size of one tag [m] (default: 0.08) - --tspace TAGSPACING The space between the tags in fraction of the edge size - [0..1] (default: 0.3) - --tfam TAGFAMILIY Familiy of April tags ['t16h5', 't25h9', 't25h7', - 't36h11'] (default: t36h11) - - Checkerboard arguments: - --csx CHESSSZX The size of one chessboard square in x direction [m] - (default: 0.05) - --csy CHESSSZY The size of one chessboard square in y direction [m] - (default: 0.05) - -* **Calibrate the intrinsic IMU parameters** ：kalibr requires imu data to be calibrated by intrinsic parameters by default.The intrinsic parameters calibration tool uses `imu-tk `_ . - -* **Count imu data parameter**： - * noise density - * bias random walk - -Using Allan analyzing tool `imu_utils `_, We can get the characteristics of above imu data,and to format the output as ``imu.yaml`` - -.. code-block:: bash - - #Accelerometers - accelerometer_noise_density: 0.02680146180736048 #Noise density (continuous-time) - accelerometer_random_walk: 0.0026296086159332804 #Bias random walk - - #Gyroscopes - gyroscope_noise_density: 0.008882328296710996 #Noise density (continuous-time) - gyroscope_random_walk: 0.00037956578292701033 #Bias random walk - - rostopic: /mynteye/imu/data_raw #the IMU ROS topic - update_rate: 200.0 #Hz (for discretization of the values above) - -Calibrate the pose relationship between left and right camera --------------------------------------------------------------- - -* Collect calibration images: kalibr supports the collection of the required calibration images through two ways:by ``rosbag`` or collect offline images . Using ``rosbag`` here for convenience,Reference `link `_ for collecting images. -* Method of collecting images by ``rosbag`` :fix mynteye camera,move ``aprilgrid`` calibration board in the camera field of view. -* To increase the calibration time,try to use image acquisition data with lower frame rate,kalibr recommends using ``4Hz`` frame rate,here uses ``10hz`` . -* MYNTEYE S series camera offers images at least 10Hz,You can use `topic_tools `_ to modify frequency,because using 10Hz requires more calibration time. -* Record ``static.bag`` : After fix the mynteye camera,start `wrapper `_, record the topic of the left and right images to ``static.bag`` . - -.. code-block:: bash - - $ source wrappers/ros/devel/setup.bash - $ roslaunch mynt_eye_ros_wrapper display.launch - $ cd ~ - $ mkdir -p bag - $ cd bag - $ rosbag record -O static_10hz /mynteye/left/image_raw /mynteye/right/image_raw #recommand use 10hz,you can also use topic_tools to publish 4hz. - -* kalibr calibration: - -.. code-block:: bash - - $ kalibr_calibrate_cameras --target aprilgrid.yaml --bag ~/bag/static_10hz.bag --models pinhole-radtan pinhole-radtan --topics /mynteye/left/image_raw /mynteye/right/image_raw - -View parameters' meanings by kalibr_calibrate_cameras command: - -.. code-block:: bash - - $ kalibr_calibrate_cameras --h - - Calibrate the intrinsics and extrinsics of a camera system with non-shared - overlapping field of view. - - usage: - Example usage to calibrate a camera system with two cameras using an aprilgrid. - - cam0: omnidirection model with radial-tangential distortion - cam1: pinhole model with equidistant distortion - - kalibr_calibrate_cameras --models omni-radtan pinhole-equi --target aprilgrid.yaml \ - --bag MYROSBAG.bag --topics /cam0/image_raw /cam1/image_raw - - example aprilgrid.yaml: - target_type: 'aprilgrid' - tagCols: 6 - tagRows: 6 - tagSize: 0.088 #m - tagSpacing: 0.3 #percent of tagSize - - optional arguments: - -h, --help show this help message and exit - --models MODELS [MODELS ...] - The camera model ['pinhole-radtan', 'pinhole-equi', - 'omni-radtan', 'pinhole-fov'] to estimate - - Data source: - --bag BAGFILE The bag file with the data - --topics TOPICS [TOPICS ...] - The list of image topics - --bag-from-to bag_from_to bag_from_to - Use the bag data starting from up to this time [s] - - Calibration target configuration: - --target TARGETYAML Calibration target configuration as yaml file - - Image synchronization: - --approx-sync MAX_DELTA_APPROXSYNC - Time tolerance for approximate image synchronization - [s] (default: 0.02) - - Calibrator settings: - --qr-tol QRTOL The tolerance on the factors of the QR decomposition - (default: 0.02) - --mi-tol MITOL The tolerance on the mutual information for adding an - image. Higher means fewer images will be added. Use -1 - to force all images. (default: 0.2) - --no-shuffle Do not shuffle the dataset processing order - - Outlier filtering options: - --no-outliers-removal - Disable corner outlier filtering - --no-final-filtering Disable filtering after all views have been processed. - --min-views-outlier MINVIEWOUTLIER - Number of raw views to initialize statistics (default: - 20) - --use-blakezisserman Enable the Blake-Zisserman m-estimator - --plot-outliers Plot the detect outliers during extraction (this could - be slow) - - Output options: - --verbose Enable (really) verbose output (disables plots) - --show-extraction Show the calibration target extraction. (disables - plots) - --plot Plot during calibration (this could be slow). - --dont-show-report Do not show the report on screen after calibration. - -Output the following three files after finish calibration: - - * ``camchain-homezhangsbagstatic_10hz.yaml`` - * ``report-cam-homezhangsbagstatic_10hz.pdf`` - * ``results-cam-homezhangsbagstatic_10hz.txt`` - -.. tip:: - - If you use camera parameters in Vins,it would be better to choose the pinhole-equi model or the omni-radtan model.If you use camera parameters in Maplab,please choose pinhole-equi model - -Calibrate the pose relationship between camera and IMU coordinate system -------------------------------------------------------------------------- - -* **Collect calibration data**：as calibrate the pose relationship of camera,Kalibr supports two ways to collect data,we still use ``rosbag`` here. - * Method of collecting image: fix ``apilgrid`` calibration board, move camera - * Make sure that the data collected is good:the brightness of the calibration board should be appropriate,too bright or too dark can't guarantee the quality of data,meanwhile do not shake too fast to avoid blurring of the image. - * Set the imu publishing frequency to 200Hz,image to 20Hz(recommended by kalibr) - * Fully motivate each axis of the imu,for example ,3 actions on each axis,then in the \"8-shaped\" motion - -* Record camera and imu as ``dynamic.bag``. - -.. code-block:: bash - - $ roslaunch mynt_eye_ros_wrapper display.launch - $ cd bag - $ rosbag record -O dynamic /mynteye/left/image_raw /mynteye/right/image_raw /mynteye/imu/data_raw #remember set image hz to 20hz, imu hz to 200hz - -* kalibr calibration: - -.. code-block:: bash - - $ kalibr_calibrate_imu_camera --cam camchain-homezhangsbagstatic_10hz.yaml --target aprilgrid.yaml --imu imu.yaml --time-calibration　--bag ~/bag/dynamic.bag - -View the parameters' meanings by kalibr_calibrate_imu_camera command - -.. code-block:: bash - - $ kalibr_calibrate_imu_camera --h - - Calibrate the spatial and temporal parameters of an IMU to a camera chain. - - usage: - Example usage to calibrate a camera system against an IMU using an aprilgrid - with temporal calibration enabled. - - kalibr_calibrate_imu_camera --bag MYROSBAG.bag --cam camchain.yaml --imu imu.yaml \ - --target aprilgrid.yaml --time-calibration - - camchain.yaml: is the camera-system calibration output of the multiple-camera - calibratin tool (kalibr_calibrate_cameras) - - example aprilgrid.yaml: | example imu.yaml: (ADIS16448) - target_type: 'aprilgrid' | accelerometer_noise_density: 0.006 - tagCols: 6 | accelerometer_random_walk: 0.0002 - tagRows: 6 | gyroscope_noise_density: 0.0004 - tagSize: 0.088 | gyroscope_random_walk: 4.0e-06 - tagSpacing: 0.3 | update_rate: 200.0 - - optional arguments: - -h, --help show this help message and exit - - Dataset source: - --bag BAGFILE Ros bag file containing image and imu data (rostopics - specified in the yamls) - --bag-from-to bag_from_to bag_from_to - Use the bag data starting from up to this time [s] - --perform-synchronization - Perform a clock synchronization according to 'Clock - synchronization algorithms for network measurements' - by Zhang et al. (2002). - - Camera system configuration: - --cams CHAIN_YAML Camera system configuration as yaml file - --recompute-camera-chain-extrinsics - Recompute the camera chain extrinsics. This option is - exclusively recommended for debugging purposes in - order to identify problems with the camera chain - extrinsics. - --reprojection-sigma REPROJECTION_SIGMA - Standard deviation of the distribution of reprojected - corner points [px]. (default: 1.0) - - IMU configuration: - --imu IMU_YAMLS [IMU_YAMLS ...] - Yaml files holding the IMU noise parameters. The first - IMU will be the reference IMU. - --imu-delay-by-correlation - Estimate the delay between multiple IMUs by - correlation. By default, no temporal calibration - between IMUs will be performed. - --imu-models IMU_MODELS [IMU_MODELS ...] - The IMU models to estimate. Currently supported are - 'calibrated', 'scale-misalignment' and 'scale- - misalignment-size-effect'. - - Calibration target: - --target TARGET_YAML Calibration target configuration as yaml file - - Optimization options: - --time-calibration Enable the temporal calibration - --max-iter MAX_ITER Max. iterations (default: 30) - --recover-covariance Recover the covariance of the design variables. - --timeoffset-padding TIMEOFFSET_PADDING - Maximum range in which the timeoffset may change - during estimation [s] (default: 0.01) - - Output options: - --show-extraction Show the calibration target extraction. (disables - plots) - --extraction-stepping - Show each image during calibration target extraction - (disables plots) - --verbose Verbose output (disables plots) - --dont-show-report Do not show the report on screen after calibration. - -Output the follwing 4 files after finish calibration: - * ``camchain-imucam-homezhangsbagdynamic.yaml`` - * ``imu-homezhangsbagdynamatic.yaml`` - * ``report-imucam-homezhangsbagdynamic.pdf`` - * ``results-imucam-homezhangsbagdynamic.yaml`` \ No newline at end of file diff --git a/docs/src/slam/orb_slam2.rst b/docs/src/slam/orb_slam2.rst index 46e1539..a26a83f 100644 --- a/docs/src/slam/orb_slam2.rst +++ b/docs/src/slam/orb_slam2.rst @@ -9,41 +9,38 @@ If you wanna run ORB_SLAM2 with MYNT EYE camera, please follow the steps: 1. Download `MYNT-EYE-S-SDK `_ and follow steps to install. 2. Follow the normal procedure to install ORB_SLAM2. -3. Update ``distortion_parameters`` and ``projection_parameters`` to ``/config/mynteye_*.yaml``. -4. Run examples by MYNT® EYE. +3. Run examples by MYNT® EYE. -Binocular camera sample ------------------------- - -* Calibrate a stereo camera with `ROS-StereoCalibration `_ or OpenCV, and then update parameters to ``/config/mynteye_s_stereo.yaml``. - -* Execute ``build.sh``: +Prerequisites +-------------- .. code-block:: bash - chmod +x build.sh - ./build.sh - -* Run stereo sample using the follow type: - -.. code-block:: bash - - ./Examples/Stereo/stereo_mynt_s ./Vocabulary/ORBvoc.txt ./config/mynteye_s_stereo.yaml true /mynteye/left/image_raw /mynteye/right/image_raw - + sudo apt-get -y install libglew-dev cmake + cd ~ + git clone https://github.com/stevenlovegrove/Pangolin.git + cd Pangolin + mkdir build + cd build + cmake .. + cmake --build . + sudo make install Building the nodes for mono and stereo (ROS) -------------------------------------------- -* Add the path including ``Examples/ROS/ORB_SLAM2`` to the ``ROS_PACKAGE_PATH`` environment variable. Open ``.bashrc`` file and add at the end the following line. Replace ``PATH`` by the folder where you cloned ORB_SLAM2: +* Add the path including ``Examples/ROS/ORB_SLAM2`` to the ``ROS_PACKAGE_PATH`` environment variable. Open ``.bashrc`` file and add at the end the following line. .. code-block:: bash - export ROS_PACKAGE_PATH=${ROS_PACKAGE_PATH}:PATH/ORB_SLAM2/Examples/ROS + export ROS_PACKAGE_PATH=${ROS_PACKAGE_PATH}:~/catkin_ws/src/MYNT-EYE-ORB-SLAM2-Sample * Execute `build_ros.sh`: .. code-block:: bash + chmod +x build.sh + ./build.sh chmod +x build_ros.sh ./build_ros.sh @@ -51,8 +48,6 @@ Building the nodes for mono and stereo (ROS) Stereo_ROS Example ~~~~~~~~~~~~~~~~~~~ - * Reference ``Get camera calibration parameters`` in :ref:`slam_okvis` to get ``distortion_parameters`` and ``projection_parameters`` , and update ``/config/mynteye_s_stereo.yaml`` . - * Launch ORB_SLAM2 ``Stereo_ROS`` 1. Launch mynteye node @@ -68,4 +63,4 @@ Stereo_ROS Example .. code-block:: bash - rosrun ORB_SLAM2 mynteye_s_stereo ./Vocabulary/ORBvoc.txt ./config/mynteye_s_stereo.yaml true /mynteye/left/image_raw /mynteye/right/image_raw + rosrun ORB_SLAM2 mynteye_s_stereo ./Vocabulary/ORBvoc.txt ./config/mynteye_s_stereo.yaml false /mynteye/left_rect/image_rect /mynteye/right_rect/image_rect diff --git a/docs/src/slam/vins.rst b/docs/src/slam/vins.rst index e6e907e..a3a0d82 100644 --- a/docs/src/slam/vins.rst +++ b/docs/src/slam/vins.rst @@ -9,8 +9,7 @@ If you wanna run VINS-Mono with MYNT EYE camera, please follow the steps: 1. Download `MYNT-EYE-S-SDK `_ and install mynt_eye_ros_wrapper. 2. Follow the normal procedure to install VINS-Mono. -3. Update ``distortion_parameters`` and ``projection_parameters`` to `here `_ . -4. Run mynt_eye_ros_wrapper and VINS-Mono. +3. Run mynt_eye_ros_wrapper and VINS-Mono. Install ROS Kinetic conveniently (if already installed, please ignore) ---------------------------------------------------------------------- @@ -21,6 +20,22 @@ Install ROS Kinetic conveniently (if already installed, please ignore) wget https://raw.githubusercontent.com/oroca/oroca-ros-pkg/master/ros_install.sh && \ chmod 755 ./ros_install.sh && bash ./ros_install.sh catkin_ws kinetic +Install Ceres +-------------- + +.. code-block:: bash + + cd ~ + git clone https://ceres-solver.googlesource.com/ceres-solver + sudo apt-get -y install cmake libgoogle-glog-dev libatlas-base-dev libeigen3-dev libsuitesparse-dev + sudo add-apt-repository ppa:bzindovic/suitesparse-bugfix-1319687 + sudo apt-get update && sudo apt-get install libsuitesparse-dev + mkdir ceres-bin + cd ceres-bin + cmake ../ceres-solver + make -j3 + sudo make install + Install MYNT-EYE-VINS-Sample ------------------------------ @@ -28,28 +43,14 @@ Install MYNT-EYE-VINS-Sample mkdir -p ~/catkin_ws/src cd ~/catkin_ws/src - git clone -b mynteye https://github.com/slightech/MYNT-EYE-VINS-Sample.git + git clone https://github.com/slightech/MYNT-EYE-VINS-Sample.git cd .. catkin_make source devel/setup.bash echo "source ~/catkin_ws/devel/setup.bash" >> ~/.bashrc source ~/.bashrc -Get image calibration parameters ---------------------------------- - -Use MYNT® EYE's left eye camera and IMU. By `MYNT-EYE-S-SDK `_ API ``GetIntrinsics()`` function and ``GetExtrinsics()`` function, you can "get the image calibration parameters of the current working device: - -.. code-block:: bash - - cd MYNT-EYE-S-SDK - ./samples/_output/bin/tutorials/get_img_params - -After running the above type, pinhole's ``distortion_parameters`` and ``projection_parameters`` is obtained , and then update to `here `_ . - -.. tip:: - - You can get the camera model of device when get camera calibration parameters, if model is equidistant you need calibrate pinhole model by yourself or reference :ref:`write_img_params` to write a default pinhole config file to your device. +(if you fail in this step, try to find another computer with clean system or reinstall Ubuntu and ROS) Run VINS-Mono with MYNT® EYE ----------------------------- @@ -68,7 +69,3 @@ Run VINS-Mono with MYNT® EYE cd ~/catkin_ws roslaunch vins_estimator mynteye_s.launch - -.. note:: - - If you want to use a fish-eye camera model, please click `here `_ . diff --git a/docs/src/slam/vins_fusion.rst b/docs/src/slam/vins_fusion.rst index 231c5d5..9bb13cf 100644 --- a/docs/src/slam/vins_fusion.rst +++ b/docs/src/slam/vins_fusion.rst @@ -11,13 +11,30 @@ If you wanna run VINS-Fusion with MYNT EYE camera, please follow the steps: 2. Follow the normal procedure to install VINS-Fusion. 3. Run mynt_eye_ros_wrapper and VINS-Fusion. +Install ROS Kinetic conveniently (if already installed, please ignore) +---------------------------------------------------------------------- -Prerequisites +.. code-block:: bash + + cd ~ + wget https://raw.githubusercontent.com/oroca/oroca-ros-pkg/master/ros_install.sh && \ + chmod 755 ./ros_install.sh && bash ./ros_install.sh catkin_ws kinetic + +Install Ceres -------------- -1. Install Ubuntu 64-bit 16.04 or 18.04. ROS Kinetic or Melodic.(if already installed, please ignore). `ROS Installation `_ -2. Install `Ceres `_ +.. code-block:: bash + cd ~ + git clone https://ceres-solver.googlesource.com/ceres-solver + sudo apt-get -y install cmake libgoogle-glog-dev libatlas-base-dev libeigen3-dev libsuitesparse-dev + sudo add-apt-repository ppa:bzindovic/suitesparse-bugfix-1319687 + sudo apt-get update && sudo apt-get install libsuitesparse-dev + mkdir ceres-bin + cd ceres-bin + cmake ../ceres-solver + make -j3 + sudo make install Install MYNT-EYE-VINS-FUSION-Samples ------------------------------------- @@ -26,7 +43,7 @@ Install MYNT-EYE-VINS-FUSION-Samples mkdir -p ~/catkin_ws/src cd ~/catkin_ws/src - git clone -b mynteye https://github.com/slightech/MYNT-EYE-VINS-FUSION-Samples.git + git clone https://github.com/slightech/MYNT-EYE-VINS-FUSION-Samples.git cd .. catkin_make source ~/catkin_ws/devel/setup.bash @@ -42,14 +59,15 @@ Run VINS-FUSION with MYNT® EYE cd (local path of MYNT-EYE-S-SDK) source ./wrappers/ros/devel/setup.bash - roslaunch mynt_eye_ros_wrapper mynteye.launch + roslaunch mynt_eye_ros_wrapper vins_fusion.launch 2. Open another terminal and run vins .. code-block:: bash - cd ~/catkin_ws - roslaunch vins mynteye-s-mono-imu.launch # mono+imu fusion - # roslaunch vins mynteye-s-stereo.launch # Stereo fusion / Stereo+imu fusion - # roslaunch vins mynteye-avarta-mono-imu.launch # mono+imu fusion with mynteye-avarta - # roslaunch vins mynteye-avarta-stereo.launch # Stereo fusion / Stereo+imu fusion with mynteye-avarta + cd ~/catkin_ws/src + source ./devel/setup.bash + roslaunch vins mynteye-s-stereo.launch # Stereo fusion / Stereo+imu fusion + # roslaunch vins mynteye-s-mono-imu.launch # mono+imu fusion + # roslaunch vins mynteye-s2100-mono-imu.launch # mono+imu fusion with mynteye-s2100 + # roslaunch vins mynteye-s2100-stereo.launch # Stereo fusion / Stereo+imu fusion with mynteye-s2100