principles of database management systems 7 crash recovery l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Principles of Database Management Systems 7: Crash Recovery PowerPoint Presentation
Download Presentation
Principles of Database Management Systems 7: Crash Recovery

Loading in 2 Seconds...

play fullscreen
1 / 54

Principles of Database Management Systems 7: Crash Recovery - PowerPoint PPT Presentation


  • 236 Views
  • Uploaded on

Principles of Database Management Systems 7: Crash Recovery. Pekka Kilpeläinen (after Stanford CS245 slide originals by Hector Garcia-Molina, Jeff Ullman and Jennifer Widom). PART II of Course. Major consern so far: How to manipulate database efficiently ? Now shift focus to:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Principles of Database Management Systems 7: Crash Recovery' - darice


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
principles of database management systems 7 crash recovery

Principles of Database Management Systems7: Crash Recovery

Pekka Kilpeläinen

(after Stanford CS245 slide originals by Hector Garcia-Molina, Jeff Ullman and Jennifer Widom)

Notes 7: Crash Recovery

part ii of course
PART II of Course
  • Major consern so far:
    • How to manipulate database efficiently?
  • Now shift focus to:
    • How to ensure correctness of database manipulation?

Notes 7: Crash Recovery

integrity or correctness of data
Integrity or correctness of data
  • Would like data to be “accurate” or “correct” at all times

EMP

Name

Age

White

Green

Gray

52

3421

1

Notes 7: Crash Recovery

integrity or consistency constraints
Integrity or consistency constraints
  • Predicates that the data must satisfy
  • Examples:

- x is key of relation R

- x  y holds in R

- Domain(x) = {Red, Blue, Green}

- no employee should make more than twice the average salary

Notes 7: Crash Recovery

definition
Definition:
  • Consistent state: satisfies all constraints
  • Consistent DB: DB in a consistent state

Notes 7: Crash Recovery

constraints as we use here may not capture full correctness
Constraints (as we use here) may not capture “full correctness”

Example 1Transaction constraints

  • When salary is updated,

new salary > old salary

  • When account record is deleted,

balance = 0

In general, various policies to ensure that the DB corresponds to "real world"

Notes 7: Crash Recovery

slide7

.

.

.

.

.

.

50

150

150

.

.

.

.

.

.

1000

1000

1100

Obs: DB cannot be consistent always!

Example: a1 + a2 +…. an = TOT (constraint)

Deposit $100 in a2: a2 a2 + 100

TOT  TOT + 100

a2

TOT

Notes 7: Crash Recovery

transaction collection of actions that preserve consistency process
Transaction: collection of actions that preserve consistency( process)

Consistent DB

Consistent DB’

T

SQL: each query or modification statement

Embedded SQL: Sequence of DB operations up to COMMIT or ROLLBACK ("abort")

Notes 7: Crash Recovery

big assumption
Big assumption:

If transaction T starts with consistent state +

executes in isolation

 T leaves consistent state

Correctness(informally)

  • If we stop running transactions, DB left consistent
  • Each transaction sees a consistent DB

Notes 7: Crash Recovery

how can constraints be violated
How can constraints be violated?
  • Transaction bug
  • DBMS bug
  • Hardware failure

e.g., disk crash alters balance of account

  • Simultaneous transactions accessing shared data

e.g.: T1: give 10% raise to programmers T2: change programmers  systems analysts

Notes 7: Crash Recovery

how can we prevent fix violations
How can we prevent/fix violations?
  • This lecture: due to system failures only[Chapter 8]
  • Chapter 9: due to concurrency only
  • Chapter 10: due to failures and concurrency

Notes 7: Crash Recovery

will not consider
Will not consider:
  • How to write correct transactions
  • How to write correct DBMS
  • Constraint checking & repair

That is, solutions studied here do not need

to know constraints

Notes 7: Crash Recovery

failure model
Failure Model

Events Desired

Undesired Expected

Unexpected

Notes 7: Crash Recovery

context of failure model
Context of failure model

processor

memory disk

- volatile, - nonvolatiledisappears when power off

CPU

D

M

Notes 7: Crash Recovery

slide15

that’s it!!

Undesired Unexpected: Everything else!

Desired events: see product manuals….

Undesired expected events:

System failures

- SW errors, power loss

-> memory and transaction state lost

Notes 7: Crash Recovery

slide16

Undesired Unexpected: Everything else!

Examples:

  • Disk data is lost
  • Memory lost without CPU halt
  • Aliens attack and destroy the building….

(can protect against this by archive backups)

Notes 7: Crash Recovery

is this model reasonable
Is this model reasonable?

Approach: Add low level checks + redundancy to increase probability that the model holds

E.g., Replicate disk storage (stable store)

Memory parity

CPU checks

Notes 7: Crash Recovery

interacting address spaces
Interacting Address Spaces:

Storage hierarchy

Memory Disk

Read(x,t)

Input(x)

t

x

x

Write(x,t)

Output(x)

Input (x): block with x from disk to buffer

Output (x): block with x from buffer to disk

Read (x,t): transaction variable t:= X; (Performs Input(x) if needed)

Write (x,t): X (in buffer) := t

Notes 7: Crash Recovery

note on database elements x
Note on "database elements" X:
  • Anything that can have value and be accessed of modified by transactions:
    • a relation (or extent of an object class)
    • a disk block
    • a record
  • Easiest to use blocks as database elements

Notes 7: Crash Recovery

key problem unfinished transaction
Key problem Unfinished transaction

Example Constraint: A=B (*)

T1: A  A  2

B  B  2

(*) simplification; a more realistic example: the sum of loan balances = the total debt of a bank

Notes 7: Crash Recovery

slide21

failure! -> A != B

16

16

16

T1: Read (A,t); t  t2

Write (A,t);

Read (B,t); t  t2

Write (B,t);

Output (A);

Output (B);

A: 8

B: 8

A: 8

B: 8

memory

disk

Notes 7: Crash Recovery

slide22
Need atomicity: either execute all actions of a transaction or none at all

One solution: undo logging (immediate

modification)

Log: sequence of log records, recording actions of transactions Ti:

<Ti, start>: transaction has begun

<Ti, commit>: completed succesfully

<Ti, abort>: could not complete succesfully

<Ti, X, v>: Ti has updated element X, whose old value was v

Notes 7: Crash Recovery

slide23

16

16

16

16

Undo logging (Immediate modification)

constraint: A=B

T1: Read (A,t); t  t2 Write (A,t);

Read (B,t); t  t2

Write (B,t);

Output (A);

Output (B);

<T1, start>

<T1, A, 8>

A:8

B:8

A:8

B:8

<T1, B, 8>

<T1, commit>

disk

memory

log

Notes 7: Crash Recovery

one complication

16

BAD STATE

# 1

One “complication”
  • Log is first written in memory
  • Not written to disk on every action

memory

DB

Log

A: 8

B: 8

A: 8 16

B: 8 16

Log:

<T1,start>

<T1, A, 8>

<T1, B, 8>

Notes 7: Crash Recovery

one complication25

16

BAD STATE

# 2

One “complication”
  • Log is first written in memory
  • Not written to disk on every action

memory

DB

Log

A: 8

B: 8

A: 8 16

B: 8 16

Log:

<T1,start>

<T1, A, 8>

<T1, B, 8>

<T1, commit>

...

<T1, B, 8>

<T1, commit>

Notes 7: Crash Recovery

undo logging rules
Undo logging rules

(1) Generate an undo log record for every update action (with the old value)

(2) Before modifying element X on disk, any log records for X must be flushed to disk (write-ahead logging)

(3) All WRITEs of transaction T must be OUTPUT to disk before <T, commit> is flushed to log

Notes 7: Crash Recovery

recovery rules undo logging
Recovery rules: Undo logging
  • For every Ti with <Ti, start> in log: - If <Ti,commit> or <Ti,abort> in log, do nothing - Else For all <Ti, X, v> in log:

write (X, v)

output (X )

Write <Ti, abort> to log

IS THIS CORRECT??

Notes 7: Crash Recovery

slide28

11

Recovery with Undo logging

constraint: A=B

T1: A  A+1; B  B+1;

T2: A  A+1; B  B+1;

log

DB on disk

A:10

B:10

<T1, start>

<T1, A, 10>

<T2, start>

<T2, A, 11>

CRASH => A should be restored to 10, not 11

Notes 7: Crash Recovery

undo logging recovery more precisely
Undo logging Recovery :(more precisely)

(1) Let S = set of transactions with <Ti, start> in log, but no

<Ti, commit> (or <Ti, abort>) record in log

(2) For each <Ti, X, v> in log,

in reverse order (latest  earliest) do:

- if Ti  S then - write (X, v)

- output (X)

(3) For each Ti  S do

- write <Ti, abort> to log

Notes 7: Crash Recovery

slide30
What if system fails during recovery?

No problem! Undo idempotent

- Can repeat (unfinished) undo, result remains the same

Notes 7: Crash Recovery

to discuss
To discuss:
  • Redo logging
  • Undo/redo logging, why both?
  • Checkpointing
  • Media failures

Notes 7: Crash Recovery

redo logging deferred modification

output

16

16

16

Redo logging (deferred modification)

T1: Read(A,t); t t2; write (A,t);

Read(B,t); t t2; write (B,t);

Output(A); Output(B)

<T1, start>

<T1, A, 16>

<T1, B, 16>

<T1, commit>

A: 8

B: 8

A: 8

B: 8

DB

memory

LOG

Notes 7: Crash Recovery

redo logging rules
Redo logging rules

(1) For every disk update, generate a redo log record (with new value)

(2) Before modifying DB element X on disk, all log records for the transaction that (including COMMIT) must be flushed to disk

Notes 7: Crash Recovery

recovery rules redo logging
Recovery rules: Redo logging
  • For every Ti with <Ti, commit> in log:
    • For all <Ti, X, v> in log:

Write(X, v)

Output(X)

This time, the last updates should remain in effect

Notes 7: Crash Recovery

recovery with redo logging more precisely
Recovery with Redo Logging:(more precisely)

(1) Let S = set of transactions with <Ti, commit> in log

(2) For each <Ti, X, v> in log, in forward order (earliest  latest) do:

- if Ti  S then Write(X, v)

Output(X)

Notes 7: Crash Recovery

recovery is very very slow
Recovery is very, very SLOW !

Redo log:

First T1 wrote A,B Last

Record Committed a year ago Record

(1 year ago) --> STILL, Need to redo after crash!!

...

...

...

Crash

Notes 7: Crash Recovery

solution checkpointing for redo logging simple version
Solution: Checkpointing (for redo-logging, simple version)

Periodically:

(1) Stop accepting new transactions

(2) Wait all transactions to finish (commit/abort)

(3) Flush all log records to disk (log)

(4) Flush all buffers to disk (DB)

(5) Write & flush a <CKPT> record on disk (log)

(6) Resume transaction processing

Notes 7: Crash Recovery

example what to do at recovery

Crash

<T1,A,16>

<T1,commit>

Checkpoint

<T2,B,17>

<T2,commit>

<T3,C,21>

...

...

...

...

...

...

Example: what to do at recovery?

Redo log (on disk):

Result of transactions completed prior to the checkpoint are stored permanently on disk

-> Redo write(B, 17) output(B)

Notes 7: Crash Recovery

drawbacks
Drawbacks:
  • Undo logging:
    • System forced to update disk at the end of transactions (may increase disk I/O)
    • cannot redo recent updates to refresh a backup copy of the DB
  • Redo logging:
    • need to keep all modified blocks in memory until commit (may lead to shortage of buffer pool)

Notes 7: Crash Recovery

solution undo redo logging
Solution: Undo/redo logging!

Update record

 <Ti, X, Old-X-value, New-X-value>

Provides more flexibility to order actions

Notes 7: Crash Recovery

rules for undo redo logging
Rules for undo/redo logging

(1) Log record <Ti, X, old, new> flushed to disk before writing X to disk

(2) Flush the log at commit

  • Element X can be written to disk either before or after the commit of Ti

Notes 7: Crash Recovery

undo redo recovery policy
Undo/Redo Recovery Policy
  • To recover from a crash,
    • redo updates by any committed transactions (in forward-order, earliest first)
    • undo updates by any incompletetransactions (in backward-order, latest first)

Notes 7: Crash Recovery

problem with simple checkpointing
Problem with Simple Checkpointing
  • Simple checkpointing stops the system from accepting new transactions -> system effectively shuts down
  • Solution: "nonquiescent" checkpointing
    • accepts new transactions during a checkpoint

Notes 7: Crash Recovery

nonquiescent checkpointing
Nonquiescent Checkpointing
  • Slightly different for various logging policies
  • Rules for undo/redo logging:
    • Write log record (and flush log) <START CKPT T1, …, Tk>, where T1, …, Tk are the active transactions
    • Flush to disk all dirty buffers which contain changed database elements (Corresponding log records first!) => Effect of writes prior to the chkpt made permanent
    • Write & flush log record <END CKPT>

Notes 7: Crash Recovery

non quiescent checkpoint
Non-quiescent checkpoint

<Start ckpt

Ti,T2,…>

<end

ckpt>

L

O

G

for

undo dirty buffer

pool pages

flushed

...

...

...

...

Notes 7: Crash Recovery

examples what to do at recovery time
Exampleswhat to do at recovery time?

no T1 commit

L

O

G

...

<T1, X,a, a'>

...

Ckpt

T1

...

Ckpt

end

...

<T1,Y, b, b'>

 Undo T1 (restore Y to b and X to a)

Notes 7: Crash Recovery

example
Example

L

O

G

...

T1

a

...

ckpt-s

T1

...

T1

b

...

ckpt-

end

...

T1

c

...

T1

cmt

...

 Redo T1: (redo update of elements to b and to c)

Notes 7: Crash Recovery

recovery process
Recovery process:
  • Backwards pass (end of log  latest checkpoint start)
    • construct set S of committed transactions
    • undo actions of transactions not in S
  • Undo pending transactions
    • follow undo chains for transactions in (checkpoint active list) - S
  • Forward pass (latest checkpoint start  end of log)
    • redo actions of transactions in set S

backward pass

start

check-

point

forward pass

Notes 7: Crash Recovery

media failure loss of non volatile storage

Disk crash!

Media failure (loss of non-volatile storage)

A: 16

Solution: Make copies of data!

Notes 7: Crash Recovery

solution a use mirrored or redundant disks
Solution a:Use mirrored or redundant disks

D1

D2

D3

  • Mirroring: copies on separate disks
    • Output(X) --> three outputs
    • Input(X) --> three inputs + vote

Notes 7: Crash Recovery

example 2 redundant writes single reads
Example #2 Redundant writes, Single reads
  • Keep N copies on separate disks
  • Output(X) --> N outputs
  • Input(X) --> Input one copy - if ok, done

- else try another one

 Assumes bad data can be detected

Notes 7: Crash Recovery

solution b archiving
Solution b: Archiving

backup

database

active

database

log

  • If active database is lost,
    • restore active database from backup (archive)
    • bring up-to-date using redo entries in log

Notes 7: Crash Recovery

when can we discard the log
When can we discard the log?

last

needed

undo

check-

point

db

dump

log

time

not needed for

media recovery

not needed for undo

after system failure

not needed for

redo after system failure

Notes 7: Crash Recovery

summary
Summary
  • Consistency of data
  • Recovery from system failures
    • Logging (undo/redo/undo-redo), checkpoints
    • Recovery: using log to reconstruct the DB
  • Recovery form media failures
    • reducing risk through redundancy
    • backups / archiving
  • Another source of problems:
    • Concurrent transactions that access shared DB elements .... NEXT

Notes 7: Crash Recovery