MATLAB 6 Least Squares Approximation - Signal Processing(DFT)

MATLAB 6
Least Squares Approximation - Signal Processing(DFT)
Definition: Let the m x n matrix A have rank n. Then by the least squares solution of the system
Ax = b is meant the solution x of the corresponding normal system AT Ay AT b .
Note:
The normal system has a unique solution

x ( AT A) 1 AT b ;

The orthogonal projection p of b into Col(A) is given by p Ax .
If the column vectors of A form an orthonormal set of vectors, then AT A I and the

solution to the least squares problem is x AT b
*********************************************************************
Let S be a subspace of an inner product space V and let x V . Let {v1 , v 2 ,..., v n} be an

n

orthonormal basis for S. If p ci vi where ci x, vi , then p x S

i1

p is the element of S that is closest to x and is said to be the projection of x onto S.
Let S be a nonempty subspace of Rm and b Rm . If {v1 , v 2 ,..., v k } is an orthonormal basis
for S and V [v1 v 2 ... v k ] , then the projection p of b onto S is given by p VV T b
The matrix VV T is called the projection matrix corresponding to the subspace S of Rm .
The projection matrix is unique (Prove)

Example 1: Let S {[x, y,0]T | x, y are real}. Let w [5,3,4]T . If v1 [1,0,0]T & v 2 [0,1,0]T ,
then p VV T w [5,3,0]T .

Example 2: Find the least squares approximation to e x on the interval [0,1] by a linear function.

Example 3:

The trigonometric polynomial of degree n is the function of the form

tn (x) a0 n
2
(ak coskx bk sin kx)

k1

{ 1 , cos x, cos2x, ..., cosnx, sin x, sin 2x, ..., sin nx}forms an orthonormal set with respect
2

to the inner product f , g 1 f (x)g(x)dx .

The best least squares fit approximation to a continuous 2 periodic function f (x) by a
trigonometric polynomial of degree n or less has Fourier Coefficients given by:

a0 f ,1 , ak f , coskx , & bk f ,sin kx .

Let ck 1 (ak ibk ) and c k 1 (ak ibk ) , k 0 . Using these, it can be shown that
2 2

tn (x) n

ck eikx

kn

where

ck 1 f (x)e ikxdx
2

The Discrete Fourier Series (Page 267)

In what follows we assume that f (x) is a noisy signal (see figure 5.5.5 a in text).

We will choose [0,2 ] instead of [ , ].

n

We want to approximate f (x) by tn (x) ck eikx

kn

The trigonometric approximation represents the function by simple harmonics where the

the k-th harmonic is rk cos(kx k ) , where k is the angular frequency.
A signal is smooth if the coefficients ck 0 rapidly as k
If some of the coefficients corresponding to larger frequencies are not small, the graph is
noisy.
We can filter the signal by setting these coefficients to zero. (see figure 5.5.5 b)

In actual signal processing we do not have a mathematical formula for f (x) . Rather, the

signal is sampled over a sequence of times x j 2 j and the function is represented by
N

the N sample values y0 f (x0 ), y1 f (x1 ),..., yN 1 f (xN 1 ). [ y0 f (0) & yN f (2 )]

In this case, we cannot compute ck . Instead we use a numerical integration method, the

trapezoidal rule, to approximate the integral. The approximations to the Fourier

coefficients are given by

dk 1N 1 ikx j
Nj 0 f (x j )e

2i 1N 1
Nj 0yj
If N eN cos(2 ) i sin(2 ) we can write dk jk
N N N

The finite sequence { dk } is said to be the discrete Fourier transform of { yk } .

The Fourier Transform can be determined by a single matrix vector multiplication. For

example, if N = 4,

d0 1 ( y0 y1 y2 y3 )
4

1 4 y1 42 y2 3 y3 )
d1 4 ( y0 4 2 y1 4
43 y1
1 44 y2 46 y3 )
d2 4 (y0

d3 1 ( y 0 46 y2 49 y3 )
4

If z = (1/4)y = 1 y1 y2 y3 ]T , then the vector d = [d0 d1 d2 d3 ]T is determined by
4 [ y0

multiplying z by the matrix

11 1 1 11 1 1

1 23 1 i 1i
1 1 called the Fourier Matrix
F4 44 4
11 1
246
444

13 6 9 1i 1i
444

FN is the N N matrix whose ( j, k) element is f j,k .( j 1)(k 1)

N

DFT algorithm is the method of computing the discrete Fourier transform.

Exercises.
1. Derive the equation (5) on page 268.
2. Explain how the powers of 4 can be derived on the unit circle in the complex plane.
3. Determine all the powers of 8 by hand (using the unit circle) and write an exact form for
F8 (no decimal approximations)
4. Use MATLAB to compute the DFT { d 0 , d1 , d2 , d 3 , d4 , d 5 , d 6 , d 7 } for the following
data and plot the data
y0 0.324
y1 0.456
y2 0.112
y3 0.202
y4 0.654
y5 0.234
y6 0.678
y7 0.867
5. Plot the trigonometric polynomial tn (x) .
6. Compare the Exercise 8 with the built in MATLAB FFT (do an help fft)