- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

#### Presentation Transcript

**1. **Establishing Convergence James Neel
June 8, 2004

**2. **Presentation Overview Convergent Sequences
Monotonic Sequences
Cauchy Sequences
Contraction Mappings
Pseudo-contraction Mappings
Block Contractions
Round Robin (Gauss-Seidel) Contractions

**3. **Convergent Sequence A sequence in a metric space X with point such that for every , there is an integer N such that implies
This can be equivalently written as
or

**4. **Example Convergent Sequence Given ?, choose N=1/ ?, p=0

**5. **Convergent Sequence Properties

**6. **Cauchy Sequence A sequence in a metric space X such that for every , there is an integer N such that if

**7. **Example Cauchy Sequence Given ?, choose N=2/?, p=0

**8. **Cauchy Sequences and Cauchy Sequences (Theorem 3.11 in Rudin)
(a) In any metric space X, every convergent sequence is a Cauchy sequence.
(b) If X is a compact metric space and if is a Cauchy sequence in X, then converges to some point of X.
(c) In , every Cauchy sequence converges.

**9. **Complete metric spaces A metric space in which every Cauchy sequence converges.
Examples of complete metric spaces:
All compact metric spaces
All Euclidean spaces
All closed subsets of complete metric spaces.

**10. **Monotonic Sequences A sequence is monotonically increasing if .
A sequence is monotonically decreasing if
(Note: some authors use the inclusion of the equals condition to define a sequence to be respectively monotonically nondecreasing or monotonically nonincreasing.). A sequence which is either monotonically increasing or monotonically decreasing is said to be monotonic.

**11. **Convergent Monotonic Sequences Suppose is a monotonic in X. Then converges iff X is bounded.
Note that also converges if X is compact.

**12. **Set Diameter Set Diameter
Let X be a metric space and . The diameter of E is given by
Note that if , is Cauchy iff

**13. **Example Set Diameters

**14. **Sequence of compact subsets Theorem 3.10 (b) Rudin
If Kn is a sequence of compact sets in X such that and if then
consists of exactly one point.
Note that this can be used to establish both the existence and uniqueness of a fixed point if is generated by iterative evaluation of a function of the form

**15. **Contraction Mapping Let be a metric space, is a contraction if there is a such that

**16. **Contraction Mapping Lemma (Blackwell’s Conditions) Let be any map that satisfies
(Monotonicity)
(Discounting) There is a such that for all
Then L is a contraction.

**17. **Another Common Contraction Mapping Condition Recall
If

**18. **Pseudo-contraction Let be a metric space and with fixed point . f is a pseudo-contraction if there is such that

**19. **Pseudo-Contraction Lemma Let be a complete metric space, and be a pseudo-contraction. Then f has a unique fixed point. (Proof in Bertsekas, proposition 1.2)
Note: not every pseudo-contraction is a contraction.

**20. **Convergence Rate Given a contraction or pseudo-contraction
with modulus ?, where . then

**21. **Cartesian Product Contractions Define X as where and are given ||xi||i for each Xi.
Define ||xi|| = max ||xi||i (over i). This is the block-maximum norm.
f: X?X is a block contraction if f is a contraction on the block maximum norm.
f: X?X is a block pseudo-contraction if f is a pseudo-contraction on the block maximum norm.

**22. **Round Robin Updating Also called Gauss-Seidel techniques
Define
Then define

**23. **Round Robin Block Contraction If f is a block contraction, then the round robin mapping, S, is also a block contraction. Further S converges to the same fixed point as f if X is closed (Bertsekas 3.1.4)
If f is a block pseudo-contraction, then the round robin mapping, S, is also a block pseudo-contraction. Further S converges to the same fixed point as f if X is closed (Bertsekas 3.1.5)

**24. **Block Contraction Rate If f is a block contraction and X closed, then the round robin mapping, S, and f converge geometrically (? t )

**25. **Continuous Differentiable Contractions Suppose X is convex. If f:?n? ?n is continuously differentiable and there exists a scalar ? ?[0,1) such that
where Gi is an invertible symmetric matrix and ? is a nonzero scalar, then
is a block contraction.
Note if ni = 1, then can choose Gi = 1. The bigger ? is, the faster it converges.
(Bertsekas 3.1.10)

**26. **Continuous Variant Suppose ni = 1 for all i, X is convex, f:?n? ?n
if a) b)
Then T(x)=x-? f(x) is a block contraction provided that 0<?<1/K
An application of Bertsekas 3.1.10
Similar to Numerical Analysis’s Fixed Point Iteration Criterion

**27. **More convergence Descent algorithms
Monotone mappings
Asynchronous algorithms
Next time?