Image Interpolation - cs.toronto.edu

Image Interpolation

Problem: Given a sampled image Á Ò½ Ò¾ we wish to interpolate it at

   non-integer position, say Á Ò½ ½ Ò¾ ¾ , where ´ ½ ¾µÌ is a

real-valued vector.

Separable Approach: We use a separable filter kernel to reduce 2D
interpolation to several 1D interpolation problems.

n1 -1 n1 x

n2 -1 x (n1, n2)
n2 (n1 -d1, n2-d2)

y

Reuse: We reuse our 1D interpolation filters for the separable kernels
(see upSample.m). The Fourier properties of the 1D kernels carry di-
rectly over to this 2D approach.

Matlab: imageTutorial.m demonstrates simple image translations and
warps.

320: 2D Interpolation ­c A.D. Jepson, 2005 Page: 1

Example: Image Translation

Suppose we wish to translate an image Á Ò½ Ò¾ by a (constant) dis-
placement . That is, the new image Ê Ò½ Ò¾ is

   Á Ò½ (1)
Ê Ò½ Ò¾ ½ Ò¾ ¾

We assume the new image Ê is the same size as Á, with pixels for

   which ´Ò½ ½ Ò¾ ¾µ is out of bounds in the original image Á set to

some special value (eg. nan in Matlab). (Alternatively, the range for

the pixels in Ê might be enlarged to include the range over which the

   translated image Á Ò½ ½ Ò¾ ¾ is defined.)

Strictly speaking, Á Ò½ Ò¾ is only defined at integer-valued sample

   points. The term Á Ò½ ½ Ò¾ ¾ in (1) is short-hand for the sam-

ple of some interpolating image ´Ü ݵ at the intermediate location

    £´Ò½ ½ Ò¾ ¾µ. In general, we could define ´Ü ݵ Á, where

is some 2D interpolation kernel ´Ü ݵ.

We will use a separable interpolation kernel ´Ü ݵ ½´Üµ ¾´Ýµ. Here
can be any 1D interpolation kernel, such as the triangular filter, or

the Catmull-Rom filter considered previously.

320: 2D Interpolation Page: 2

Separable Implementation

For a separable kernel, we have

´Ü ݵ £ £ £Á ´ ¾ Ý ´ ½ Ü Áµµ´Ü ݵ (2)

£ £where Ü and Ý denotes convolution in the Ü and Ý directions, respec-

tively. Using a temporary image Ì ´Ü Ò¾µ we can write (2) in two steps,

Ì ´Ü Ò¾µ £´ ½ Ü Á µ´Ü Ò¾µ  ½´Ü ѽµÁ ѽ Ò¾

´Ü ݵ £´ ¾ Ý Ì µ´Ü ݵ ѽ

 ¾´Ý ѾµÌ ´Ü Ѿµ

Ѿ

   Evaluating this at the desired point ´Ü ݵ ´Ò½ ½ Ò¾ ¾µ, we find

 Ì ´Ò½ ½ Ò¾µ    ½´Ò½ (3)
½ ѽµÁ ѽ Ò¾

ѽ

   ´Ò½ ½ Ò¾ ¾µ      ¾´Ò¾ ½ Ѿµ (4)
¾ ѾµÌ ´Ò½

Ѿ

Notice equation (4) provides the desired translated image Ê Ò½ Ò¾

   ´Ò½ ½ Ò¾ ¾µ. By defining Ì Ò½ Ò¾  Ì ´Ò½ ½ Ò¾µ we can

rewrite equations (3) and (4) as discrete 1D convolutions.

320: 2D Interpolation Page: 3

Image Translation by Separable Convolution

Define the discrete filter kernels  ´ ½µ (5)
 ´ ¾µ (6)
½
¾

for integers , where ´Üµ is some 1D interpolation kernel (eg. Catmull-
Rom). Then equations (3-4) can be written

Ì Ò½ Ò¾ £½ Ü Á  ½´Ò½ ѽµÁ ѽ Ò¾ (7)
Ê´Ò½ Ò¾µ £¾ Ý Ì (8)
ѽ

 ¾´Ò¾ ѾµÌ ´Ò½ Ѿµ

Ѿ

These operations were sketched on page 1. First Á is interpolated along

 each row, and sampled at the intermediate locations Ò½ ½. This forms

the temporary image Ì . Then this temporary image is interpolated

 down each column, and sampled at the intermediate locations Ò¾ ¾.

Away from the image boundaries, if we swap the order of these two
convolutions then we obtain the same translated image (why?).

When ´Üµ is the triangular (piecewise linear) interpolation filter this
is called bilinear interpolation. For piecewise cubic ´Üµ it is called
bicubic interpolation.

320: 2D Interpolation Page: 4

Image Warping

More general “rubber sheet” image deformations involve spatial dis-

placements which vary with the image position Ü. In particular, sup-

pose Ï ´Üµ is a warp function which takes a point Ü in the desired im-
age ʴܵ, to the corresponding point Ü Û Ï ´Üµ in the original image
Á´Ü Ûµ. Then the warped image is defined by

ÊÒ Á ´Ï ´Òµµ where Ò ´Ò½ Ò¾µÌ Ï ´Ï½ ϾµÌ (9)

In order to evaluate Á´Ï ´Òµµ, we again need to use 2D interpolation.

However, due to spatial variation in Ï ´Üµ, the Ü, Ý interpolation kernels

in (5-6) will in general depend on Ü,

½Ü    ´ ´Ü½ Ͻ´Üµµµ (10)
¾Ü    ´ ´Ü¾ Ͼ´Üµµµ (11)

Equations (7-8) become

Ì Ò½ Ѿ Ò  ½ Ò½ ½ Ò Á ½ Ѿ (12)

ÊÒ ½ (13)

 ¾ Ò¾ Ѿ Ò Ì ´Ò½ Ѿ Òµ

Ѿ

This is not simply convolution since the filter kernels depend on the im-

age location Ò. Finally, if the warp Ï ´Üµ involves a substantial shrink-

age, then we need to blur Á Ò before warping to avoid aliasing.

320: 2D Interpolation Page: 5