Raspberry Pi:calibrate Camera And Undistortion Of Image

I want to remove fisheye effect from image, so I need camera matrix and distortion cofficient. That's why I created cal.py file. When I run that file it give me camera matrix and d

Solution 1:

I will do an answer to summarize all the problems solved in the comments (this way the future readers do not have to read all of them).

1. Follow the tutorial from here to have a better understanding of what you should do and which functions to use.

2. Check that your camera matrix have the correct form:

   [ f_x  s    c_x
     0    f_y  c_y
     0    0    1  ]

   the s is a skew value that I think in opencv it always gives 0.

3. Make sure in every step that the images are loading correctly and that they are doing exactly what is intended (use imshow function to debug)

4. This part of your code

   newcamera, roi = cv2.getOptimalNewCameraMatrix(K, d, (w,h), 0)
   newimg = cv2.undistort(img, K, d, None, newcamera)
   

is faulty. Why, well from the opencv documentation we have that:

cv2.getOptimalNewCameraMatrix(cameraMatrix, distCoeffs, imageSize, alpha[, newImgSize[, centerPrincipalPoint]]) → retval, validPixROI

and

The function computes and returns the optimal new camera matrix based on the free scaling parameter. By varying this parameter, you may retrieve only sensible pixels alpha=0 , keep all the original image pixels if there is valuable information in the corners alpha=1 , or get something in between. When alpha>0 , the undistortion result is likely to have some black pixels corresponding to “virtual” pixels outside of the captured distorted image. The original camera matrix, distortion coefficients, the computed new camera matrix, and newImageSize should be passed to initUndistortRectifyMap() to produce the maps for remap() .

This means that the new camera matrix also gives you a new valid size!! To be more specifically:

the computed new camera matrix, and newImageSize should be passed to initUndistortRectifyMap()

But the undistort function does the initUndistortRectifyMap automatically.... and it doesn't have a way to pass the newImageSize from this function. So basically you have 2 options.

1. Use your newcamera matrix, but instead of doing undistort, you should do everything manually.... this means that you must do initUndistortRectifyMap and remap using the new sizes and new camera matrix.
2. Use the original camera matrix obtained in the calibrateCamera function. This way it will not have this zoom effect, but you may have some extra black pixels representing the areas that you may not see due to rectification.

If not, it will always give you this zoom effect, because it will not show the invalid pixels (black pixels) of the undistorted areas.