Mapping frames based on motion

Question

I need to create 3 intermediate frames between two frames (prevImg, nextImg), I have found out the motion of each pixel using calcOpticalFlowFarneback() function in opencv

  calcOpticalFlowFarneback(gray1, gray2, flow, pyrScale, pyrLevel, winSize,
                           iter, polyn, sigma, method);

then I have created 3 intermediate frames by calling the function createNewFrame(), with following shift (func. argument) values 0.25, 0.5, 0.75

void createNewFrame(Mat & frame, const Mat & flow, float shift, int & c, Mat & prev, Mat &next)
{
  for (int y = 0; y < mapX.rows; y++)
  {
    for (int x = 0; x < mapX.cols; x++)
    {
      Point2f f = flow.at<Point2f>(y, x);
      mapX.at<float>(y, x) =  x + f.x/shift;
      mapY.at<float>(y, x) =  y + f.y/shift;
    }
  }
  remap(frame, newFrame, mapX, mapY, INTER_LINEAR);
}

But I am not getting proper intermediate frames.. the transition from one frame to other is non smooth (flickering). What is the problem in my code ? What I need to do to get proper intermediate frames ? ?

Answer 1

I think you should be multiplying by your variable shift, not dividing by it.

Try replacing your double loop in function createNewFrame by the following one:

  for (int y = 0; y < mapX.rows; y++)
  {
    for (int x = 0; x < mapX.cols; x++)
    {
      Point2f f = flow.at<Point2f>(y, x);
      mapX.at<float>(y, x) =  x + f.x*shift;
      mapY.at<float>(y, x) =  y + f.y*shift;
    }
  }