The Viterbi Algorithm

1. The Viterbi Algorithm • Application of Dynamic Programming-the Principle of Optimality • -Search of Citation Index -213 references since 1998 • Applications • Telecommunications • Convolutional codes-Trellis codes • Inter-symbol interference in Digital Transmission • Continuous phase transmission • Magnetic Recording-Partial Response Signaling- • Divers others • Image restoration • Rainfall prediction • Gene sequencing • Character recognition

2. Milestones • Viterbi (1967) decoding convolutional codes • Omura (1968) VA optimal • Kobayashi (1971) Magnetic recording • Forney (1973) Classic survey recognizing the generality of the VA • Rabiner (1989) Influential survey paper of hidden Markov chains

3. Professor X chooses an optimum path on his trip to lunch Find optimal path to each bridge Example-Principle of Optimality Perish Publish N 1.2 .2 .7 1.2 .5 N Faculty Club 1.2 .5 .3 EE Bld Optimal: 6 adds Brute force:8 adds N bridges Optimal: 4(N+1) adds Brute force: (N-1)2N adds .5 S 1.0 .8 .8 S .8

4. Digital Transmission with Convolutional Codes Information Source Convolutional Encoder BSC p p Information Sink Viterbi Algorithm

5. Maximum a Posteriori (MAP) Estimate Brute force = Exponential Growth with N

6. Convolutional codes-Encoding a sequence (output,input) efficiency=input/output Example(3,1) code Output 111 100 010 110 011 001 000 Input 110100 T T

7. Fig.2.14

8. Trellis Representation State output Next state 0 input s1s2 1 input 000 111 00 01 10 11 00 01 10 11 001 110 011 100 010 101

9. Iteration for Optimization

10. Key step! Redundant Linear growth in N

11. Deciding Previous State State i-1 State i 00 00 1 4 4 Search previous states 10 2 2

12. Viterbi Algorithm-shortest pathto detect sequence First step s0 s1 s2 s3 Trace though successive states shortest path-Hamming distance to s0 Trellis codes-Euclidean distance Optimum Sequence Detection

13. Inter-symbol Interference Transmitter Channel Equalizer VA Decisions

14. AWGN Channel-MAP Estimate

15. Viterbi Algorithm for ISI Magic Iteration State = number of symbols in memory

16. Magnetic Recording Magnetization pattern Magnetic flux passes over heads Differentiation of pulses Controlled ISI Same model applies to Partial Response signaling Output Sample

17. Continuous Phase FSK odd number ½ cycles Whole cycles

18. Merges and State Reduction Optimal paths through trellis All paths merge Force merges to reduce complexity Computations order of (No states)2 Carry only high probability states

19. Image Restoration Input Pixel Effect of Blurring Blurring Analogous to ISI

20. Row Scan VA for optimal row sequence Known state transitions And Decision Feed back Utilized for state reduction

21. Hidden Markov Chain • Data suggests Markovian structure • Estimate initial state probabilities • Estimate transition probabilities • VA used for estimation of Probabilities • Iteration

22. Rainfall Prediction Rainy wet Rainy dry Transition probabilities estimated from Rainfall data using VA. No rain Showery wet Showery dry Rainfall observations

23. DNA Sequencing • DNA-double helix • Sequences of four nucleotides, A,T,C and G • Pairing between strands • Bonding • Genes • Made up of Cordons, i.e. triplets of adjacent nucleotides • Overlapping of genes Nucleotide sequence CGGATTC Gene 1 Cordon A in three genes Gene 2 Gene 3

24. Hidden Markov ChainTracking genes S M1 P1 S-start first cordon of gene P1-4- +1,…,+4 from start G- Gene E-stop H-gap M1-4-1,…,-4 from start M2 P2 M3 P3 Initial and Transition Probabilities known M4 P4 . G H E

25. Text-line images Recognizing Handwritten Chinese Characters Estimate stroke width Set up m X n grid Estimate initial and transition probabilities Detect possible segmentation paths by VA Next Slide Results

26. Example Segmenting Handwritten Characters Eliminating Redundant Paths All possible segmentation paths Removal of Overlapping paths Discarding near paths