Intensity-Based Image Registration

Intensity-Based Image Registration Intensity-Based
Image
Gustavo K. Rohde
Registration
42-431 Intro. Biomedical Imaging and Image Analysis
42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

November 7, 2008

Overview Intensity-Based
Image
Intensity-based registration
Registration
Given digital images s(m, n) and t(m, n), find spatial
transformation f such that s(fx(m, n), fy(m, n)) and 42-431 Intro.
t(m, n) are similar. Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Overview Intensity-Based
Image
Intensity-based registration
Registration
Given digital images s(m, n) and t(m, n), find spatial
transformation f such that s(fx(m, n), fy(m, n)) and 42-431 Intro.
t(m, n) are similar. Biomedical
Imaging and
Optimization
Image Analysis
min Ψ(s(f ), t, f )
Overview
f ∈C
Least squares
f : Rd → Rd: spatial transformation model
C spatial transformation class Cross-
Ψ(·, ·, ·): difference measure correlation

Mutual
Information

Continuous image representation Intensity-Based
Image

Registration

s(x, y) = c[p, q]ϕ(x − p)ϕ(y − q) 42-431 Intro.
Biomedical
p∈Z q∈Z Imaging and

where ϕ(x) is the basis function (sinc function, B-splines), Image Analysis
and c are the the coefficients.
Overview

Least squares

Spatial transformation Cross-
correlation

Mutual
Information

s(fx(m, n), fy(m, n)) = c[p, q]ϕ(fx(m, n)−p)ϕ(fy(m, n)−q)

p∈Z q∈Z

Three main components Intensity-Based
Image

Registration

42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Least squares Intensity-Based
Image
Ψ(s(f ), t, f ) = s(f ) − t 2
Registration

42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Least squares Intensity-Based
Image
Ψ(s(f ), t, f ) = s(f ) − t 2
Registration
Example
42-431 Intro.
Affine transformations f (x) = Ax + a in 2D. Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Ψ(s(f ), t, f ) = (s (fx(m, n), fy(m, n)) − t(m, n))2

mn

with fx(m, n) = A11m + A12n + a1 and
fy(m, n) = A21m + A22n + a2.

Least squares Intensity-Based
Image
Ψ(s(f ), t, f ) = s(f ) − t 2
Registration
Example
42-431 Intro.
Affine transformations f (x) = Ax + a in 2D. Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Ψ(s(f ), t, f ) = (s (fx(m, n), fy(m, n)) − t(m, n))2

mn

with fx(m, n) = A11m + A12n + a1 and
fy(m, n) = A21m + A22n + a2.

Optimize using

pk+1 = pk − ∇Ψ(s(f ), t, f ).

Cross-correlation Intensity-Based
Image
Let s[n] and t[n] = s[n − a] be two ”nice” signals to be
aligned. The goal is then to find a. This can be done via Registration
cross correlation
42-431 Intro.
a = arg max s ∗ tv[n] Biomedical
Imaging and
n
Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

For larger signals, the computation above can be done
more efficiently using the FFT.

Example Intensity-Based
Image

Registration

42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Example Intensity-Based
Image

Registration

42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Normalized Cross-correlation Intensity-Based
Image
What if we have s[n] and t[n] = As[n − a], with unknown
A? Registration
Use normalized cross correlation:
42-431 Intro.
s[n], t[f (n)] Biomedical
Ψ(s, t, f ) = Imaging and

s[n] t(f [n]) Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Example Intensity-Based
Image

Registration

42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Example Intensity-Based
Image

Registration

42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Mutual Information Intensity-Based
Image
Key problem
Registration
Matching images of different modalities.
42-431 Intro.
Example Biomedical
Imaging and
Matching CT and MRI.
Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Intensity values not linearly related Intensity-Based
Image
Joint histogram: no linear relationship between intensity
value, even when the images are optimally aligned. This Registration
is generlly the case when matching images of different
modalities. 42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Intensity values not linearly related Intensity-Based
Image
Joint histogram: no linear relationship between intensity
value, even when the images are optimally aligned. This Registration
is generlly the case when matching images of different
modalities. 42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

As a consequence, sum of squared differences, cross
correlation, not likely to succeed.

Intensity-Based
Image

Registration

42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Intensity-Based
Image

Registration

42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

Mutual Information Intensity-Based
Image
Let Ps(f),t(s(f ), t) denote the joint PDF of images s(f )
and t. The mutual information is given by: Registration

I(s(f ), t) = Ps(f),t(s(f ), t) log Ps(f),t(s(f ), t) 42-431 Intro.
Ps(f)(s(f ))Pt(t) Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

s(f ),p

Mutual Information Intensity-Based
Image
Let Ps(f),t(s(f ), t) denote the joint PDF of images s(f )
and t. The mutual information is given by: Registration

I(s(f ), t) = Ps(f),t(s(f ), t) log Ps(f),t(s(f ), t) 42-431 Intro.
Ps(f)(s(f ))Pt(t) Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information

s(f ),p

Optimization

min −I(s(f ), t)

f

Example Intensity-Based
Image

Registration

42-431 Intro.
Biomedical
Imaging and

Image Analysis

Overview

Least squares

Cross-
correlation

Mutual
Information