Towards Efficient, Portable Application-Level Consistency

Towards Effic
Application-Lev

Thanumalayan Sankaranaraya
Joo-Young Hwang,Andrea C.Arpaci

cient, Portable
vel Consistency

ana Pillai,Vijay Chidambaram,
i-Dusseau, Remzi H.Arpaci-Dusseau

1

File Sy
Crash Co

ystem
onsistency

2

File System Cra

• What happens if there i
a file system update?
• File system crash consis
system‟s metadata is log
there is a crash
• Multiple techniques: FSC
Journaling, Copy-On-W

ash Consistency

is a system crash during

stency: Make sure file
gically consistent, even if

CK, Soft Updates,
Write …

3

Applicati
Crash Consis

ion-Level
stency (ALC)

4

Application-Level C

• What happens to user d
• Consistency of user dat
Crash Consistency (ALC
• This work – Study of w

Consistency (ALC)

data if there is a crash?
ta – Application-Level
C)
what happens to user data

5

Res

sult

6

Res

• State of the art: For ef
consistency, application
specific details of file s

• This is bad: Many file sys
ext3, ext4, btrfs, xfs, zfs

sult

ffective application-level
n developers depend on
system implementation

stems in use (Linux:
…)

7

Out

• Background:Application
• Goals, Methodology of S
• File System Behavior
• ALC Bugs
• ALC Performance
• Summary

tline

n-Level Consistency (ALC)
Study

8

Out

• Background:Application
• Goals, Methodology of S
• File System Behavior
• ALC Bugs
• ALC Performance
• Summary

tline

n-Level Consistency (ALC)
Study

9

Application-Level

• Modern applications sto
• Google Chrome initializ

– History
– Cookies
– Web page cache

l Data Structures

ore many data structures
zation: 500+ files

10

Application-Level C

• Applications impose inv

– Web page cache: Shoul
– Photo application:Thum

• Invariants should hold a

– Violation: application fa

• Requires complex imple

– eXplode [OSDI „06], Ea

Consistency (ALC)

variants on data

ld contain complete entries
mbnails match pictures

across system crashes

ailures, silent corruption

ementations

at My Data [Stewart Smith]

11

Example: Atom

File grub.conf (Original)

kernel vmlinuz
initrd initrd.img

• File should always be in

– Fully original state or fu

• File should never

– Contain garbage, or be

mic File Rewrite

File grub.conf (Updated)

print “Hello”
kernel vmlinuz
initrd initrd.img

either

ully updated state

e empty, or filled with zeroes

12

Atomic File Rewrite

fd = creat(“tem
write(fd)
fsync(fd)
rename(“temp”

e – Correct Protocol

mp”)
”, “grub.conf ”)

13

Atomic File Rewrite

fd = creat(“tem
write(fd)
fsync(fd)
rename(“temp”

Possible states

grub.conf (Original) grub.conf (

kernel vmlinuz print “Hel
initrd initrd.img kernel vm
initrd initr

e – Wrong Protocol

mp”) Occurs because file

systems can re-order

write() and rename()

”, “grub.conf ”)

after crash

(Updated) grub.conf (Zeroes)

llo” 000000000000000
mlinuz 000000000000000
rd.img 000000000000000

14

Atomic File Rewrite

• Wrong protocol is com

– Bug (invalid assumption
– Correctness sacrificied

• Works under most com

– Ext4, btrfs etc. explicitly
– Though not required b

• Observation:

– FS implementation affec

e – Depends on FS

mmonly used – why?

n)
d for performance

mmon file systems

y ensure correctness
by standard FS interface

cts applications

15

Out

• Background:Application
• Goals, Methodology of S
• File System Behavior
• ALC Bugs
• ALC Performance
• Summary

tline

n-Level Consistency (ALC)
Study

16

Go

• Study relationship of FS

– ALC correctness
– ALC performance

• Characterize common f

– Deduce high-level “pro

oals

S implementation with
file systems

operties” affecting ALC

17

Method

• Case study:Two applicat

– Find new bugs, analyze

• Manual system call trace

– Find any correctness-pe

• Extract FS implementati
• Convert details to high-
• Characterize file system

– Understanding source c

dology

tions (SQLite, LevelDB)

existing bugs

e analysis, Bugzilla

erformance tradeoffs

ion details affecting bugs
-level properties
ms

code

18

Out

• Background:Application
• Goals, Methodology of S
• File System Behavior
• ALC Bugs
• ALC Performance
• Summary

tline

n-Level Consistency (ALC)
Study

19

Post-Crash

Post-Crash Prope
Does a system call seq

given, desirable set o

1. fd = creat(“FileA.temp”)
2. write(fd)
3. fsync(fd)
4. rename(“FileA.temp”,“FileA”)

Safe-rename property

h Property

erty (True / False):

quence only result in a
of post-crash states

grub.conf (Updated) grub.conf (Garbage)
print “Hello”
kernel vmlinuz #!@$%#!@$%#!
initrd initrd.img @$%#!@$%#!@
$%#!@$%#!@$%
(or)
grub.conf (Original) (or)
grub.conf (Zeroes)
kernel vmlinuz
initrd initrd.img 000000000000000
000000000000000
000000000000000

20

File System C

Different configurations of e
Different versions of ext4 file

ext3 – ordered
ext3 – writeback
ext4 – ordered
ext4 – ordered – original version
btrfs

Comparison

ext3 file system
e system

Safe rename







21

Ordered

• Ordered appends proper

File LogA File Lo

0.00 Started 0.00 S
append(LogA) 1.00 M
File LogB
File L
0.00 Started
0.00 S

Appends

rty 1. Append(LogA)

2. Append(LogB)

ogA File LogA

Started 0.00 Started
Msg append(LogB) 1.00 Msg

LogB File LogB

Started 0.00 Started
2.00 FAULT

22

Ordered

• Ordered appends proper

File LogA File LogA

0.00 Started 0.00 Started (or
1.00 Msg
(or)
File LogB File LogB

0.00 Started 0.00 Started

Appends

rty 1. Append(LogA)

2. Append(LogB)

File LogA File LogA

0.00 Started 0.00 Started
1.00 Msg
File LogB
r)
File LogB 0.00 Started
2.00 FAULT
0.00 Started
2.00 FAULT

23

File System C

• Ordered appends:Appen
issued order

ext3 – ordered Safe re
ext3 – writeback
ext4 – ordered 
ext4 – original 
btrfs 



Comparison

nds get persisted in the

ename Ordered appends

 

 24



More Pro

• Ordered dir-ops: Director
unlink, rename …) get p
• Safe appends:When a fil
appended portion will n
• Safe new file:After fsync(
fsync() on the parent di

roperties

ry operations (creat,
persisted in issued order
le is appended, the
never contain garbage
() on a new file, another
irectory is not needed

25