Project 4 Results

Starter Images

Recovering Homographies

First, we need to find the parameters of the homography transformation, given by:

$$\mathbf{p'} = H\mathbf{p}$$

Here, \(H\) represents a 3 x 3 matrix containing 8 unknowns where the bottom right is a scale factor of 1, while \(\mathbf{p}\) and \(\mathbf{p'}\) denote the homogeneous coordinates of corresponding points in the source and target images.

Our goal is to determine a projective transformation that satisfies:

$$\begin{bmatrix} a & b & c \\ d & e & f \\ g & h & 1 \end{bmatrix} \begin{bmatrix} x \\ y \\ 1 \end{bmatrix} = \begin{bmatrix} wx' \\ wy' \\ w \end{bmatrix}$$

Expanding and substitution yields the following:

$$\begin{cases} ax + by + c = wx' \\ dx + ey + f = wy' \\ gx + hy + 1 = w \end{cases}$$

$$\begin{cases} ax + by + c = (gx + hy + 1)x' \\ dx + ey + f = (gx + hy + 1)y' \end{cases}$$

$$\implies \begin{bmatrix} x & y & 1 & 0 & 0 & 0 & -xx' & -yx' \\ 0 & 0 & 0 & x & y & 1 & -xy' & -yy' \end{bmatrix} \begin{bmatrix} a \\ b \\ c \\ d \\ e \\ f \\ g \\ h \end{bmatrix} = \begin{bmatrix} x' \\ y' \end{bmatrix}$$

With n=4 correspondences, the system can be solved standardly, but the homography will be more unstable and noisy, as will be shown below. With more correspondences, we have an overdetermined system and can solve using least squares by combining multiple correspondence points into an expanded matrix and vector form.

Rectification

For rectification, I selected 4 points corresponding with what should be a square at various angles, and then warped using a homography associated with a scaled square based on the size of the image as the im2_pts.

I think part of the reason the output is not a perfect square as it probably should be is due to a variety of reasons. The main one is as mentioned before, with 4 points and manual selection there is a lot of noise in the results. Additionally, the photo of the RSF was taken at a sharp angle and object to rectify may not be a perfect square, all of which could be contributing factors for the rectification result not being a perfect square.

Mosiac

For creating mosaics, I warped one image into the other's projection. I computed a low-pass and high-pass filter of the image and blended each. The high-pass was blended by comparing the distance transform in each image, and for low-pass it was blended with a weighted average with the distance transforms. Results are shown below:

Detecting Harris Corners

Using the provided sample code:

Adaptive Non-Maximal Suppression (ANMS)

Because Harris points are very clustered, we can run ANMS to select corner points that are more spread out, I use k=500 points here. Harris is on the left, ANMS on the right.

Feature Descriptor Extraction

To extract features for each point, select a 40x40 window scale down by a factor of 5 for 8x8 patches. Then normalize to have zero mean and variance 1. Each 8x8 patch can be flattneed into a vector for a feature descriptor.

Automatic Feature Matching

With the help of the provided dist function and using nearest neighbors, we try to match correspondences. Using Lowe's trick, outliers can be reduced by thresholding on the ratio between the nearest and second nearest neighbors, as good matches should correspond to having a very close nearest neighbor relative to all other neighbors.

RANSAC

In order to best handle outliers, RANSAC can be used, which randomly samples 4 pairs of points and computes the exact homography. The inliers are computed, and after all iterations we store the largest set of inliers, and work backwards using least-squares to find the associated homography.

Inliers are given by blue, outliers that are not used are shown in red