Markov Approximation for Combinatorial Network Optimization

Markov Appro
Combinatorial Netw

Minghu

Joint wo
Soung‐Chang Liew, Ziyu Shao, Ca

oximation for 
work Optimization

ua Chen

ork with 
aihong Kai, and Shaoquan Zhang

Resource Alloca

ation is Critical

□ Utilize resource

– Efficiently
– Fairly
– Distributedly

□ A bottom‐up example

– Flow control (TCP)

2

Convex Networ
Popular an

□ Formulate resource alloc
maximization problem [K

□ Design distributed soluti

– Local decision, adapt to d

rk Optimization: 
nd Effective

cation as a utility 
Kelly 98, Low et. al. 99, …]

max X
Us(xs)
x≥0
s∈S
s.t.
Ax ≤ C

ions

dynamics

3

Example: Unde

max X 1
− Tr2xr
xr ≥0,r∈R Xr∈R
xs ≤ Cl, ∀
s.t.

s:l∈s

– End users: run TCP based
– Routers: drop packets ba

□ Local actions jointly solv

erstanding TCP

accessing efficiency + fairness 
[Mo‐Walrand 00]

∀l ∈ L incoming rate less 
than the link capacity

d on end‐to‐end measurement
ased on local information

ve global problem

4

Combinatorial Netw
Popular b

□ Joint routing and flow 

x

?

□ Many others: Wireless
channel assignment, to

work Optimization: 
but Hard

 control problem

max X
Us(xs)
x≥0,A
s∈S

s.t. Ax ≤ C,

A ∈ {feasible

routing matrices}

s utility maximization, 
opology control …

5

Observations a

Convex: solved

• Top‐down approach
• (mostly) convex problems
• Theory‐guided distributed 
solutions

□ This talk: Explore theory‐gu
solutions for combinatorial

and Messages

Combinatorial: open

• Top‐down approach
• Combinatorial problems
• ??

uided design for distributed
l network problems

6

Markov Approximati

Convex network problems

Formulation

Penalty/decomposition 
approach

Primal/dual/primal‐dual 
design

ion: Our Perspective

Combinatorial network 
problems

Formulation

Log‐sum‐exp approximation

Distributed Monte Carlo 
Markov Chain

7

Existing Solutions 
Optimi

□ Polynomial‐time approx

– Deterministic/randomize
– Centralized (in general)

□ Simulated annealing and

– No control of time‐comp
– Centralized (in general)

□ Our perspective: distribu

– Distributed simulated an
dynamics as special cases

 for Combinatorial 
ization

ximated solutions

ed solutions solutions

d Glauber dynamics

plexity tightly connect to 
statistical physics

uted solution

nnealing and Glauber
s

8

Generic Form of Com
Optimizatio

maxf ∈F

□ System settings:

– A set of user configurat
– System performance un

□ Goal: maximize networ
choosing configuration

mbinatorial Network 
on Problem

Wf .

tions, f =[f1, f2, …, f|R|]∈ F
nder f, Wf

rk‐performance by 
ns

9

Exam

□ Wireless network utilit

– Configuration f: indepe

□ Channel assignments in

– Configuration f: one co
channel assignments

□ Path selection and flow

– Configuration f: one co
selected paths 

□ Peering in Peer‐to‐Pee

mples

ty maximization New  

endent set perspective

n WiFi networks

ombination of 

w control New 
perspective 
ombination of 
and new 
er systems… solutions

10

Wireless Network U

max

z≥0,p≥

s.t.

□ zs: rate of user s
□ L: set of links, each with un
□ F : set of all independent s
□ pf: percentage of time f is a

Utility Maximization

x X Us(zs)

≥0 s∈SX

zs ≤ X pf , ∀l ∈ L

sX:l∈s,s∈S f :l∈f
pf = 1

f ∈F

Wireless link 

nit capacity capacity constraints
sets (configurations)
active

11

Wireless Network U

max X Wirele
Us(zs) capacity c
z≥0,p≥0
s∈SX zs ≤ X pf ,
s.t.

sX:l∈s,s∈S f :l∈f
pf = 1

f ∈F

□ zs: rate of user s
□ L: set of links, each with 

□ F : set of all independen
□ pf: percentage of time f i

Utility Maximization

ess link  L3 ∅
constraints L1
L2
∀l ∈ L L2

L1 L3

3‐links interference  L1L3

graph independent 

 unit capacity sets

nt sets (configurations)

is active

12

Scheduling Proble

XX
min max Us(zs) −

λ≥0 z≥0 sX∈S s∈

s.t. pf = 1.

f ∈F

□ An NP‐hard combinato

Independent Set probl
XX
max pf λl

p≥0 fX∈F l∈f

s.t. pf = 1.

f ∈F

em: Key Challenge

XX XX
zs λl + max pf λl

∈S l∈s p≥0 f ∈F l∈f

(scheduling)

orial Max Weighted 

lem max X λl

= f ∈F

l∈f

13

Related Work 

□ Wireless scheduling is NP‐h

□ It is recently shown that bo
scheduling problem approx

– [Wang‐Kar 05, Liew et. al. 0
Rajagopalan‐Shah 08, Liu‐Y
Srikant 09, …]

□ Our framework provides a 

– Note that our framework a
problems

 on Scheduling

hard [Lin‐Shroff‐Srikant 06, …]

ottom‐up CSMA can solve the 
ximately 

08, Jiang‐Walrand 08, 
Yi‐Proutiere‐Chiang‐Poor 09, Ni‐

 new top‐down perspective

applies to general combinatorial 

14

Step 1: Log‐sum‐e

max X λl 1
β
f ∈F ≈

l∈f

max(x1, x2) ≈ 1 log (
10

□ Approximation gap: 

□ The approximation be
approaches infinity 

exp Approximation

⎛ ⎛ ⎞⎞
X X
1 log ⎝ exp ⎝β λl⎠⎠
β
f ∈F l∈f

(exp (10x1) + exp (10x2))

1 log |F|
β

ecomes exact as β

15

Step 1: Log‐sum‐e

max X λl 1
β
f ∈F ≈

l∈f

max XX max
pf λl
p≥0 p≥0
fX∈F l∈f
s.t.
pf = 1.

f ∈F

exp Approximation

⎛ ⎛ ⎞⎞
X X
1 log ⎝ exp ⎝β λl⎠⎠
β
f ∈F l∈f

Log‐sum‐exp is a concave and 

closed function, double conjugate 

is itself 

X pf X − 1 X pf log pf
λl β

f ∈F Xl∈f f ∈F

s.t. pf = 1.

f ∈F

16

Big Picture After

□ The new max

z≥0,p≥

primal problem s.t.

□ Solution:

⎪⎪⎪⎧⎪⎪⎨⎪⎩zSλ˙˙scl h==edkαulslhehPUfs0s(f:loz∈rss),ps−f∈(SPβzλsl∈)−spλPelricl

r Approximation

≥0 X − 1X pf log pf
Us(zs) β
Xf ∈F
s∈SX pf , ∀l ∈ L
zs ≤

sX:l∈s,s∈R f :l∈f

pf = 1.

f∈F Distributed?

i+ TCP‐like ?

zs i+
l∈f pf (βλ) λl Local queue
centage of time.

17

Schedule by a Produ

X ⎛

max λl ≈ 1 log ⎝
β
f ∈F l∈f

pf (λ) pf

f ∈F
□ Computed by solving the Kar

the entropy‐approximated p

uct‐form Distribution

⎛ ⎛ ⎞⎞
XX

⎝ exp ⎝β λl⎠⎠

f ∈F l∈f

⎛⎞
X
f (λ) = 1 exp ⎝β λl⎠
C(βλ)
l∈f

f ∈F

rush‐Kuhn‐Tucker conditions to 
problem

18

Step 2: Achieving p

⎛ X

pf (λ) = 1 exp ⎝β
C(βλ)
l∈

f ∈F

□ Rcleagsas rodf  ptifm(λe)‐ raesv tehresi bstleeaM

– States: all the independe
– Transition rate: new desi
– Time‐reversible: detailed

pf(λ) Distributedly

⎞ f f0
X

λl⎠

∈f

f 000 f 00

pf (λ) qf,f0 = pf0 (λ) qf0,f

ady‐state distribution of a 
Markov Chains

ent sets f ∈ F
ign space
d balance equation holds

19

Design Space: Two D

f f0 f f0

f 000 f 00 f 000 f 00
(a) (b)

pf (λ) qf,f0 =

□ 1) Add or remove transit

– Stay connected
– Steady state distribution 

□ 2) Designing transition r

Degrees of Freedom

f f0 f f0

f 000 f 00 f 000 f 00
(c) (d)

= pf0 (λ) qf0,f

tion edge pairs

 remains unchanged

rate

20

Design Goal: Distribu

Implement a Markov chain
=

Realize the transitions

□ What leads to distribut

– Every transition involve
– Transition rates Involve

uted Implementation

n L1
∅ L1L3
L2

L3

ted implementation?

es only one link
e only local Information

21

Every Transition Invo

□ From f to f’ = f ∪ {Li}: L
□ From f’ = f ∪ {Li} to f: L

L3

L2 L2
L1 L3 starts
3‐links conflict graph

olves Only One Link

Li starts to send
Li stops transmission

L1

∅ L1L3

s/stops

L3 L1 starts/stops
Designed Markov chain

22

Transition Rates Involve 

□ Consider transition bet

□ λLi is the local queue le

□ Diffe³rePntq’s´ givedif
exp β l∈f λl ex

C(βλ) qf,f 0 =

⎛
X

exp ⎝ βλl −

l∈f 0

 Only Local Information

tween f and f’ = f ∪ {Li}

ength of link Li

ffer³enPtimple´mentation
xp β l∈f 0 λl
qf 0,f
C(βλ)

⎞1

X
− βλl⎠ = exp βλLi

l∈f

23

Distributed Im

□ Link Li counts down at 
exp(β λLi)

– Count down expires?
transmit

– Interference sensed? Fr
the count‐down, and 
continue afterwards

□ Reinvent CSMA using a
top‐down approach

mplementation

 rate  L1
∅ L1L3
L2

reeze L3

L3

a  L2

L1

24

The Overall Distr

⎨⎪⎪⎪⎧⎪⎪⎪⎩zλD˙˙slis==trikαblsuhhtPUeds0s(:Mlz∈ssC),s−M∈SCPzsal∈c−shλiPelvile+z∈ssfdpifst(

□ Converges to the optim
without time‐scale sep

– Proof utilizes Lyapunov
approximation, and mix

ributed Solution

Distributed?

TCP‐like

(βλ) i+

Local queue

λl

tribution pf (βλ). CSMA‐like

mal solution with or 
paration

v arguments, stochastic 
xing time bounds

25