c++ - Unwarping a set of rectangles to form a square grid in OpenCV -

i trying map image contains these points, drawn rectangles using top left point , bottom right points, though have points quadrilaterals

i trying map image above squares squares below:

i believe supposed obtain perspective transform each quad. here code wrote try , facilitate this:

cv::mat output = cv::mat::zeros(outputsize,cv_32f);       // uses size of quadoutput     std::vector<std::vector<cv::point2f>> quadoutputimage = quadralateralsfromimageoutput(imagetowarp.size().height);      cv::mat perspectivematrix = cv::mat(3,3,cv_32f);      cv::mat warp = cv::mat::zeros(outputsize,cv_32f);      (int = 0; < quadoutputimage.size(); ++) {              // mapping quadtriangles             perspectivematrix = cv::getperspectivetransform(quadcentroids[i],quadoutputimage[i]);              // perform warp current quadtriangles             cv::warpperspective(imagetowarp,warp,perspectivematrix,output.size());              // copy roi output      }      return warp; 

i quad quad , image not appear correct @ all. appears skewed.

edit

i tried finding homography matrix, ended odd result. here's code wrote

    cv::mat warpedimagegeneration(const cv::mat& imagetowarp,cv::size outputsize, std::vector<std::vector<cv::point2f>> quadcentroids) {              // uses size of quadoutput 50 50 squares     std::vector<std::vector<cv::point2f>> quadoutputimage = quadralateralsfromimageoutput(450);      // flatten matrices     std::vector<cv::point2f> flattenedquads = flattenmatrix(quadoutputimage);     std::vector<cv::point2f> flattenedcentroids = flattenmatrix(quadcentroids);      cv::mat perspectivematrix = cv::mat(3,3,cv_32f);      cv::mat warp = cv::mat::zeros(450,450,cv_32f);      perspectivematrix = cv::findhomography(flattenedcentroids, flattenedquads);      cv::warpperspective(imagetowarp, warp, perspectivematrix, warp.size());      return warp;  } 

on example image, unwarped result get:

i think issue vertices source out of quadrilateral order i.e top left, top right, bottom left , bottom right.

warping each of rectangles individually won't give best results via getperspectivetransform. instead, if know exact coordinates of each quad original image maps in warped, try defining one perspective transformation matrix encompasses of these correspondences together. specifically, take @ of corner points each quad in source image, determine should map in warped image , define 1 perspective transformation matrix describing of these points. way, when provide points source image, should map warped points in warped, , else should interpolate cleanly.

in terms of implementation, have 2 vector containers 1 container contains of source image points , vector container contains of warped image points. order matters, make sure each location in source vector container corresponds same location on warped vector container.

however, you're going have change gears , use findhomography instead. findhomography more general case of getperspectivetransform in determining transformation matrix best warps 1 set of points set of points least amount of error. getperspectivetransform allows specify 4 points. findhomography allows specify many points want.

if want modify current framework, have code contains vector of vector of points each respective quad both input , warped image. you're accessing each vector , specifying getperspectivetransform perspective transformation matrix per quad. specify 1 vector input , 1 vector warped image contains of quad points together, make single call findhomography. order important make sure each location of quad in input image matches output location of warped image.

also, make sure coordinate system correct. opencv's point structures assume x horizontal coordinate while y vertical coordinate. have placed edits suggested in original post, i'm going place them here completeness , make answer self-contained:

 cv::mat warpedimagegeneration(const cv::mat& imagetowarp,cv::size outputsize, std::vector<std::vector<cv::point2f>> quadcentroids) {      // uses size of quadoutput 50 50 squares     std::vector<std::vector<cv::point2f>> quadoutputimage = quadralateralsfromimageoutput(450);      // flatten matrices     std::vector<cv::point2f> flattenedquads = flattenmatrix(quadoutputimage);     std::vector<cv::point2f> flattenedcentroids = flattenmatrix(quadcentroids);      cv::mat perspectivematrix = cv::mat(3,3,cv_32f);      cv::mat warp = cv::mat::zeros(450,450,cv_32f);      perspectivematrix = cv::findhomography(flattenedcentroids, flattenedquads);      cv::warpperspective(imagetowarp, warp, perspectivematrix, warp.size());      return warp;  }