CS4495/6495 Introduction to Computer Vision - Amazon S3

CS4495/6495
Introduction to Com

6B-L2 Dense flow: Lucas

mputer Vision

s and Kanade

Recall: Aperture prob

blem

Gradient component

0  It  I  u, v  or

Intuitively, what does this cons

• The component of the flow in th
determined

• The component of the flow para

gradient
(u

(u’,v’)
edge

t of flow

r Ixu  I yv  It  0

straint mean?

he gradient direction is

allel to an edge is unknown

u,v)
(u+u’,v+v’)

e

Solving the aperture

• Basic idea: Impose local
more equations for a pi

• E.g., assume that the flow

problem

l constraints to get
ixel

w field is smooth locally

Solving the aperture

• One method:
Pretend the pixel’s
neighbors have the
same (, )

• If we use a 5x5
window, that gives us
25 equations per
pixel!

problem

Lukas-Kanade flow

Problem: We have

more equations than
unknowns

( = )



Lukas-Kanade flow

Solution: Least

squares problem

(The summations are over all
pixels in the K x K window)



Lukas-Kanade flow

This technique was
first proposed by
Lukas & Kanade, 1981



Aperture Problem an

The gra

Defin

v

Normal Flow:

nd Normal Flow

adient constraint:

Ixu  Iyv  It  0
I·U  It  0

nes a line in the (u,v) space

u

Combining Local Con

v

u

nstraints

I1 U  It1
I 2 U  It2

I 3 U  It3

etc.

Conditions for solvab

When is This Solvable?

• should be invertible
• => So should be wel

should not be too large (

bility

e
ll-conditioned - 1/2
1= larger eigenvalue)

Conditions for solvab

When is This Solvable?

• Also should be solva
aperture problem

• Does this remind you of som

bility

able when there is no

mething???

Eigenvectors of ATA

• Recall the Harris corner
= is the se

• The eigenvectors and ei
to edge direction and m

r detector:
econd moment matrix

igenvalues of M relate
magnitude

Interpreting the eigen

Classification of image points usi



1 and 2 are small;
E is almost constant
in all directions

nvalues

ing eigenvalues of M:

2 “Edge” “Corner”
2 >> 1
1 and 2 are
large,

1 ~ 2;
E increases in all

directions

“Flat” “Edge”
region 1 >> 2

1

Low texture region

M  I (I )T Gra

adients have small magnitude
=> small 1, small 2

Edge

M  I (I )T Lar
=

rge gradients, all the same
=> large 1, small 2

High textured region

M   I (I )T Gra
=

adients different, large magnitudes
=> large 1, large 2

RGB version

• One method:
pretend the pixel’s
neighbors have the
same (u,v)

• If we use a 5x5x3
window, that gives us
75 equations per
pixel!



RGB version

Note that RGB alone at a
pixel is not enough to
disambiguate because
R, G & B are correlated.
Just provides better
gradient.



Errors in Lucas-Kanad

•Brightness constancy doe

• Do exhaustive neighborho
correlation - tracking featu
later….

de

es not hold

ood search with normalized
ures – maybe SIFT – more

Errors in Lucas-Kanad

•A point does not move lik

• Motion segmentation

de

ke its neighbors

Errors in Lucas-Kanad

•The motion is large (large
doesn’t hold

• Not-linear: Iterative refine
• Local minima: coarse-to-fi

de

er than a pixel) – Taylor

ement
ine estimation

Not tangent: Iterative

Iterative Lukas-Kanade A

1. Estimate velocity at each
Kanade equations

2. Warp towards +1usin

• Use image warping techn

3. Repeat until convergence

e Refinement

Algorithm

h pixel by solving Lucas-

ng the estimated flow field

niques

e

Optical Flow: Iterative

estimate
update

x0

(using d for displacemen

e Estimation

Initialization:
Estimate:

x

nt here instead of u)

Optical Flow: Iterative

estimate
update

x0

e Estimation

Initial guess:
Estimate:

x