Skip to content

Update the camera calibration app with the visualization of the lens distortion#1918

Open
s-trinh wants to merge 14 commits intolagadic:masterfrom
s-trinh:feat_calib_tools_displacement_map
Open

Update the camera calibration app with the visualization of the lens distortion#1918
s-trinh wants to merge 14 commits intolagadic:masterfrom
s-trinh:feat_calib_tools_displacement_map

Conversation

@s-trinh
Copy link
Contributor

@s-trinh s-trinh commented Feb 27, 2026

The chessboard calibration pattern is obsolete since the whole board must be detected and it does not allow to have points at the extremities of the image, which is crucial for correct lens distortion modelling.

This is what would give a classical images acquisition:

mosaics_reproj_err_with_dist_0000

But this is what is fed to the calibration procedure:

calibration_pattern_occupancy

With the first image in background, to check that the pattern occupancy is correct:

calibration_pattern_occupancy2

Some alternatives that are widely used are:

See also the DLR CalDe for advices about the procedure (e.g. oblique images are crucial, big calibration board in the image, ...):

image image image image

Previous PR: #1063

Following are some results on ViSP and additional calibration images found on the Net.


./visp-calibrate-camera default-chessboard.cfg --save

  • calibration pattern occupancy:
calibration_pattern_occupancy
  • effect of the distortion
distortion_displacement_map
Calibration with distortion in progress on 13 images...
Camera parameters for perspective projection with distortion:
  px = 536.9845596	 py = 537.4212583
  u0 = 344.111272	 v0 = 235.1745174
  kud = -0.2672795888
  kdu = 0.308119637

Global reprojection error: 0.4341517859
  • mosaic of undistorted images
mosaics_undist_0000

./visp-calibrate-camera default-circles.cfg --save

calibration_pattern_occupancy distortion_displacement_map
  • mosaics_reproj_err_with_dist_0000
    mosaics_reproj_err_with_dist_0000

Dataset 1

calibration_pattern_occupancy distortion_displacement_map
Calibration with distortion in progress on 21 images...
Camera parameters for perspective projection with distortion:
  px = 1205.671359	 py = 1204.805002
  u0 = 660.4616637	 v0 = 580.9800591
  kud = -0.1709858138
  kdu = 0.1929137239

Global reprojection error: 1.758919741
  • mosaics_reproj_err_with_dist_0000:

mosaics_reproj_err_with_dist_0000

Canon 6D Lens Canon 35mm

calibration_pattern_occupancy distortion_displacement_map
Calibration with distortion in progress on 23 images...
Camera parameters for perspective projection with distortion:
  px = 5414.713464	 py = 5414.149906
  u0 = 2713.074881	 v0 = 1819.078615
  kud = -0.08385360219
  kdu = 0.08659721482

Global reprojection error: 0.9372371689
  • mosaics_reproj_err_with_dist_0000
    mosaics_reproj_err_with_dist_0000

Canon 6D Lens Canon Ultrasonic Macro 24-105mm

calibration_pattern_occupancy distortion_displacement_map
Calibration with distortion in progress on 25 images...
Camera parameters for perspective projection with distortion:
  px = 3887.883503	 py = 3880.700626
  u0 = 2755.498706	 v0 = 1775.323829
  kud = -0.1084063088
  kdu = 0.1187173467

Global reprojection error: 3.474161422
  • mosaics_reproj_err_with_dist_0000
    mosaics_reproj_err_with_dist_0000

  • mosaics_undist_0000
    mosaics_undist_0000

Canon PowerShot SX130

Process frame: IMG_0025.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0026.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0027.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0028.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0029.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0030.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0031.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0032.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0033.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0034.JPG, grid detection status: 1, image used as input data
Process frame: IMG_0035.JPG, grid detection status: 1, image used as input data
Catch an exception: Error [4]:	Maximum number of iterations reached

Update: fixed by scaling the initial focal length by the current image resolution

Canon 40D Lens Canon 28-90mm

calibration_pattern_occupancy distortion_displacement_map
Calibration with distortion in progress on 24 images...
Camera parameters for perspective projection with distortion:
  px = 4931.720173	 py = 4938.034173
  u0 = 1963.024086	 v0 = 1296.726668
  kud = -0.15083377
  kdu = 0.1564686076

Global reprojection error: 1.418242249
  • mosaics_reproj_err_with_dist_0000
    mosaics_reproj_err_with_dist_0000

GoPro

More distortion coefficients are needed.

calibration_pattern_occupancy distortion_displacement_map
Calibration with distortion in progress on 25 images...
Camera parameters for perspective projection with distortion:
  px = 1763.05464	 py = 1775.163417
  u0 = 2101.378805	 v0 = 1465.66668
  kud = -0.1818221764
  kdu = 0.2877112244

Global reprojection error: 10.79785318
  • mosaics_reproj_err_with_dist_0000
    mosaics_reproj_err_with_dist_0000

  • mosaics_undist_0000
    mosaics_undist_0000

Nikon D3200 Lens Nikon DX 18-105mm

calibration_pattern_occupancy distortion_displacement_map
Calibration with distortion in progress on 24 images...
Camera parameters for perspective projection with distortion:
  px = 4760.930112	 py = 4763.277097
  u0 = 3040.149807	 v0 = 1995.427415
  kud = -0.1364586128
  kdu = 0.1489301149

Global reprojection error: 2.718639959
  • mosaics_reproj_err_with_dist_0000
    mosaics_reproj_err_with_dist_0000

  • mosaics_undist_0000
    mosaics_undist_0000

OnePlus X

Process frame: IMG_20160509_135147.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_135210.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_135302.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_135333.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_135410.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_135704.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_135748.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_135841.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_135922.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_140007.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_140125.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_140254.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_140354.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_143142.jpg, grid detection status: 1, image used as input data
Process frame: IMG_20160509_143800.jpg, grid detection status: 1, image used as input data
Catch an exception: Error [4]:	Maximum number of iterations reached

Update: fixed by scaling the initial focal length by the current image resolution

RaspberryPi

calibration_pattern_occupancy distortion_displacement_map
Calibration with distortion in progress on 25 images...
Camera parameters for perspective projection with distortion:
  px = 2577.308021	 py = 2576.000385
  u0 = 1305.299627	 v0 = 971.9074695
  kud = 0.01764096471
  kdu = -0.01753010016

Global reprojection error: 0.6449599556
  • mosaics_reproj_err_with_dist_0000
    mosaics_reproj_err_with_dist_0000

@codecov
Copy link

codecov bot commented Feb 27, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 48.41%. Comparing base (b03c98f) to head (9eb9fab).
⚠️ Report is 60 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1918      +/-   ##
==========================================
- Coverage   48.68%   48.41%   -0.27%     
==========================================
  Files         532      532              
  Lines       69293    69334      +41     
  Branches    32403    32441      +38     
==========================================
- Hits        33735    33569     -166     
- Misses      24976    31507    +6531     
+ Partials    10582     4258    -6324     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…e image center (in green) and the camera principal point (in red) in the distortion displacement map image.
@s-trinh s-trinh force-pushed the feat_calib_tools_displacement_map branch from d38bc19 to 971d67a Compare March 2, 2026 03:07
@s-trinh s-trinh force-pushed the feat_calib_tools_displacement_map branch from a65f1c6 to b965d26 Compare March 10, 2026 07:03
@s-trinh s-trinh marked this pull request as draft March 10, 2026 11:31
@s-trinh s-trinh force-pushed the feat_calib_tools_displacement_map branch from 4079323 to 85c785b Compare March 12, 2026 09:56
@s-trinh
Copy link
Contributor Author

s-trinh commented Mar 12, 2026

The PR is ready.


apps/calibration/intrinsic/visp-calibrate-camera.cpp now requires >= C++11.
It is too annoying to check for < C++11 and to check that every return codepath is correctly handled by a delete.
OpenCV 4 requires C++11.
Actually, the current code does not build with < C++11, see:

#if (VISP_CXX_STANDARD < VISP_CXX_STANDARD_11)
vpDisplay *d = nullptr;
#endif

The declared variable is d.

#if (VISP_CXX_STANDARD < VISP_CXX_STANDARD_11)
if (display != nullptr) {
delete display;
}
#endif

At multiple locations display is used while it should be d.


I have added a graph with uv-reprojection errors:

  • for all the images:
    img-reproj-error-graph

  • for a specific images:
    img-reproj-err-graph-chessboard-05

@s-trinh s-trinh marked this pull request as ready for review March 12, 2026 09:59
@fspindle
Copy link
Contributor

@s-trinh It makes sense to not support c++98

@fspindle
Copy link
Contributor

@s-trinh Can you add the 2 images that are missing in the doc:

visp/doc/tutorial/calibration/tutorial-calibration-intrinsic.dox:481: warning: image file img-reproj-error-graph.jpg is not found in IMAGE_PATH: assuming external image.
visp/doc/tutorial/calibration/tutorial-calibration-intrinsic.dox:485: warning: image file img-reproj-err-graph-chessboard-05.jpg is not found in IMAGE_PATH: assuming external image.

@s-trinh
Copy link
Contributor Author

s-trinh commented Mar 13, 2026

@fspindle I have added the missing images.


Also, by default now the initial camera focal length is scaled by the image size:

      double scale = I.getWidth() / 640.0;
      double px = opt_use_focal_cmd_line ? opt_init_focal : scale * cam_init.get_px();
      double py = opt_use_focal_cmd_line ? opt_init_focal : scale * cam_init.get_py();

@s-trinh s-trinh force-pushed the feat_calib_tools_displacement_map branch from f436431 to 7366c7c Compare March 14, 2026 01:47
@s-trinh s-trinh marked this pull request as draft March 15, 2026 07:49
@s-trinh s-trinh force-pushed the feat_calib_tools_displacement_map branch 2 times, most recently from 37dae4e to d4d6a74 Compare March 18, 2026 11:40
@s-trinh
Copy link
Contributor Author

s-trinh commented Mar 18, 2026

  • add possibility to calibrate also with OpenCV, this will allow using the different tools to assess the quality of the camera calibration process with OpenCV

  • add basic support for ChArUco board with the following limitation:

    • the full board needs to be visible in the image
      • some tweaks are needed to compute the corresponding object points using the marker id (instead of creating once the 3D object model) when the board is not fully visible (and to check that OpenCV allows detecting a non complete ChArUco board)
  • tested quickly using images from here, seems to work

  • new config file for CHARUCOBOARD (old config files are supported without modification):

# Number of inner corners per a item row and column. (square, circle)
BoardSize_Width: 18
BoardSize_Height: 9

# The size of a square in meters
Square_Size: 0.02

# The type of pattern used for camera calibration.
# One of: CHESSBOARD or CIRCLES_GRID
Calibrate_Pattern: CHARUCOBOARD

# The input image sequence to use for calibration
Input: pycalib/data/charuco/%08d.jpg

# Tempo in seconds between two images. If > 10 wait a click to continue
Tempo: 1

# ChArUco
Charuco_Marker_Size: 0.015
Charuco_Dictionary: DICT_4X4_250
Charuco_Legacy_Pattern: 0

calibration_pattern_occupancy

  • ViSP:
    reprojection_error_graph

mosaics_reproj_err_with_dist_0000

mosaics_undist_0000

  • OpenCV:

reprojection_error_graph

mosaics_reproj_err_with_dist_0000

mosaics_undist_0000

@s-trinh s-trinh marked this pull request as ready for review March 18, 2026 11:46
…ing limitation: the full board needs to be visible in the image.

Update the camera calibration tutorial with a section with comparison between OpenCV and ViSP calibration for wide-angle lens.
@s-trinh s-trinh force-pushed the feat_calib_tools_displacement_map branch from 0bb0ad5 to 9eb9fab Compare March 18, 2026 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants