Download Presentation
## Online Algorithms

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Online Algorithms**Advanced Seminar A Supervisor: Matya Katz Ran Taig, Achiya Elyasaf December, 2009**Overview-part A**• Introduction • What is Online Algorithm? • Evaluating the Online Algorithm • The Online Paging Problem & Algorithm • Deterministic Online Algorithms For Paging • CA ≥ k Theorem • Adversary Models • Randomize Online Paging Algorithm • Oblivious Adversary • Adaptive Adversary**IntroductionWhat is Online Algorithm?**• So far, all algorithms received their entire inputs at one time • Online algorithms (OA), receive and process the input in partial amounts. • A sequence of requests are received and the OA must service each request before it receives the next one.**IntroductionWhat is Online Algorithm? (Cont.)**• In servicing each request, a several alternatives with an associated cost are possible. • The alternative chosen may influence the costs of alternatives on future requests. • Examples: • data-structuring • resource-allocation in operating systems • finance • distributed computing**IntroductionEvaluating the Online Algorithm**• It is often meaningless to have an absolute performance measure for an algorithm. • The algorithm can be forced to incur an unbounded cost by appropriately choosing the input sequence • It is difficult, if not impossible, to perform a comparison of competing strategies. • Q: How to evaluate the algorithm? • A: We compare the total cost of the OA on a sequence of requests, to the total cost of an offline algorithm • We refer to such an analysis as a competitive analysis**IntroductionThe Paging Problem**Some definitions: • Memory item - a page of virtual memory • Cache memory – a fast memory of size of k memory items • Main memory - a slower memory that can potentially hold an infinite number of items • A paging algorithm - decides which k items to retain in the cache at each point in timeWe have a sequence of requests, each of which specifies a memory item. • A hit – the requested item is in the cache. There is no cost • A miss – the item must be fetched from the main memory. There is a ‘unit’ cost and one of the kitems in the cache must be evicted • Paging – The replacement of one page with another in the cache is called paging or page fault**IntroductionThe Paging Algorithm**• The cost measure is the number of misses on a sequence of requests • This cost depends on the algorithm that decides which k items to retain in the cache at each point in time • When a page fault happens, the paging algorithm invoke an eviction rule for deciding which item currently in the cache should be evicted to make room for the new item • Intuitively, items that will be requested again in the near future should not be evicted**IntroductionOffline Algorithm**• The offline algorithm (aka the MIN algorithm): on a miss, evict that item whose next request occurs furthest in the future • The worst-case number of misses on a request sequence of length N is N/k.**IntroductionDeterministic Online Algorithms For Paging**• Least Recently Used (LRU): evict the item in the cache whose most recent request occurred furthest in the past. • First-in, First-out (FIFO): evict the item that has been in the cache for the longest period • Least Frequently Used (LFU): evict the item in the cache that has been requested least often**IntroductionDeterministic Online Algorithms For Paging**(Cont.) • Let p1, p2,...,pn be a request sequence presented to an online paging algorithm A • Let fA(p1, p2,...,pn) denote the number of times A misses on p1, p2,...,pn • Let f0(p1, p2,...,pn) denote the minimum number of misses on p1, p2,...,pn**IntroductionDeterministic Online Algorithms For Paging**(Cont.) • A deterministic online paging algorithm A is said to be C-Competitive if there exists a constant b such that on every sequence of requests p1, p2,...,pn: fA(p1, p2,...,pn) - C∙f0(p1, p2,...,pn)≤ bwhere the constant b must be independent of N but may depend on k • Competitiveness measures the performance of an OA in terms of the worst-case ratio of its cost to that of the optimal offline algorithm**IntroductionDeterministic Online Algorithms For Paging**(Cont.) • The competitiveness coefficient of A, denoted CA, is the infimum of C such that A is C-competitive • Since the worst case of offline algorithm is N/k, no deterministic online paging algorithm has competitiveness coefficient smaller than k • LRU, FIFO are known to be k-competitive • No deterministic online paging algorithm has competitiveness coefficient smaller than k (proof will follow soon…) • Thereby, LRU and FIFO are optimal deterministic OA**IntroductionPaging Algorithm - Formally**• A paging algorithm consists of an automaton with a finite set S of states • The automaton response is defined by function • The cache after the request is serviced must include the requested item curr. state curr. cache new item curr. state curr. cache**IntroductionCA ≥ k Theorem**Let A be a deterministic online algorithm for paging then CA ≥ k Proof: Initialization - • Both A & the offline algorithm are managing different caches for the same request sequence • They both has the same k-items in the cache • First request is to an item not in either cache, and the algorithms incur a miss • Let S be the set of k+1 items consisting of the initially k items together with the new item • From then on, every request is for the unique item in S not in A's cache • Thus A misses on every request**IntroductionCA ≥ k Theorem (cont.)**• A round is a maximal sequence of requests in which at most k distinct items are requested; each of these items may be requested any number of times and in any order • A round ends when, after k distinct items have been requested, a new item p is requested, and p then becomes the first request of the next round • Since the round contains at least k requests and A misses on every one of them, it misses at least k times during the round**IntroductionCA ≥ k Theorem (cont.)**• There is an offline algorithm that misses only once during a round, in fact on the first request of the round • We denote p as first request of the following round • When the offline algorithm misses on the first request, it evicts p and thereby ensures that there are no further misses in that round (as expected from a MIN algorithm)**IntroductionCA ≥ k Theorem (cont.)**• The offline algorithm knows A’s initial cache, the entire request sequence in advance and the identity of p for every round • At the end of each round, both the online algorithm and the offline algorithm have the same set of items in their caches • This construction can be repeated many times, proving that there are arbitrarily long sequences on which A has k times as many misses as the offline algorithm.**IntroductionConclusions**• The proof uses only the fact that the OA doesn’t know future requests • Thus the lower bound applies to any deterministic OA without any regard for its use of computational resources such as time or space • This is a negative result for the online algorithms • The offline algorithm may be view as an adversary who is not only managing a cache, but is also generating the request sequence**Overview-part A**• Introduction • What is Online Algorithm? • Evaluating the Online Algorithm • The Online Paging Problem & Algorithm • Deterministic Online Algorithms For Paging • CA ≥ k Theorem • Adversary Models • Randomize Online Paging Algorithm • Oblivious Adversary • Adaptive Adversary**Adversary ModelsRandomize Online Paging Algorithm**• The Adversary Models, where in collusion with a reference algorithm that is the yardstick against which the competitiveness of the given online algorithm is being measured • The adversary's goal is to increase the cost to the given online algorithm, while keeping it down for the reference algorithm**Adversary ModelsRandomize Online Paging Algorithm**• 2 definitions: • R- a randomize online paging algorithm (=ROA) • fR(p1, p2,...,pn) – a random variable, denotes the number of times that R misses on the sequence • Evaluating the ROA • We still study the behavior of R when the sequence of requests is generated by an adversary • However, there is no longer a unique notion of an "adversary" for a randomized online algorithm**Adversary Models Oblivious Adversary**• The weakest adversary knows R in advance, but has no knowledge of the random choices made by R • This adversary calculates the “worst case” request sequence for R, regardless of the actual execution of R • The fixed cost of this sequence is not a random variable and is denoted byf0(p1, p2,...,pn) • We call such an adversary an oblivious adversary, reflecting that the adversary is oblivious to the random choices made by R**Adversary Models Oblivious Adversary**• R is C-competitive against the oblivious adversary if for every sequence of requests p1, p2,...,pn: for a constant b independent of N. • The oblivious competitiveness coefficient of R, denoted by , is the infimum of C such that R is C-competitive**Adversary ModelsAdaptive Adversary**• The Adaptive Adversary chooses pi+1 after observing the responses of the ROA to p1, p2,...,pi • To define the cost of the optimal algorithm it might help to think of the adaptive adversary and the optimal algorithm as working in collusion**Adversary ModelsAdaptive Adversary**• First scenario: • The adversary generates the optimal strategy adaptively by learning p1, p2,...,pi • We refer to this as the adaptive offline adversary • The request sequence is a random sequence. Thus, f0(p1, p2,...,pn) and fR(p1, p2,...,pn) are random variables**Adversary ModelsAdaptive Adversary**• Second scenario: • The adversary works as before, but also required to concurrently manage a cache online • Meaning, the adversary generates pi+1 based on the responses of R to p1,p2,...,pi, and immediately exhibits its own response to pi+1 • Again both f0(p1, p2,...,pn) and fR(p1, p2,...,pn) are random variables • We refer to such an adversary as an adaptive online adversary**Adversary ModelsAdaptive Adversary**• We say that R is C-competitive against the adaptive offline adversary if for a constant b independent of N: • The adaptive offline competitiveness coefficient of R, , is the infimum of C such that R is C-competitive • Likewise, is the adaptive online competitiveness coefficient of R**Adversary ModelsAdaptive Adversary**• Clearly, we can define the following proportion – • Let be the lowest oblivious competitive coefficient of any randomized paging algorithm • Similarly we define • Finally we define to be the lowest competitive coefficient of any deterministic paging algorithm • Then we have**Overview-part B**• Introduction • Paging against an oblivious – Definitions. • The players on this section – What is the goal? • Yao’s minimax theorem. • Proof of a Theorem on CR : • Preparations. • Behavior of the offline algorithm. • Behavior of a deterministic algorithm. • Results. • The marker algorithm • Description. • Proof of it’s competitiveness coefficient. • Summary**Paging against an oblivious adversary**Remember that in the proof of the result : CA ≥ k we based our analysis on the fact that the request sequence was determined at each step according to the behavior (the cache contents) of the deterministic online algorithm When talking about randomized algorithm it’s intuitive that this ability of the offline algorithm won’t be so helpful**Our goal now**We now want to prove that against an oblivious adversary a randomized online algorithm can do much better in terms of competitiveness against the result (K) we saw when we talked about deterministic online algorithm. We will see that under the above assumptions . where is the k’th harmonic number which is clearly smaller then K.**So… who against who?**Our players now are an oblivious adversary which specifies all the request sequence in advance and never changing it later on Then, an optimal offline algorithm is activated on this sequence and announcing it’s result Only then – the sequence is given to the random online algorithm we want to test – each request at a time.**A reminder : Yao’s MiniMax principle**Yao's minimax principle states that given any arbitrarily chosen input distribution P: the expected cost of the optimal deterministic algorithm under this distribution is a lower bound on the expected cost of the optimal randomized (Las Vegas) algorithm.**How to use this principle for our purposes?**The principle gives us the ability to deal only with deterministic algorithms in order to give a lower bound for the randomized algorithm. All we need to do is to choose a probability distribution for the inputs and then give a lower bound for the best (in terms of expected cost) deterministic online algorithm under this distribution.**Overview-part B**• Introduction • Paging against an oblivious – Definitions. • The players on this section – What is the goal? • Yao’s minimax theorem. • Proof of a Theorem on CR : • Preparations. • Behavior of the offline algorithm. • Behavior of a deterministic algorithm. • Results. • The marker algorithm • Description. • Proof of it’s competitiveness coefficient. • Summary**Some preparations:**The probability distribution on the inputs can be taught as the probability to choose the i’th memory item on the sequence – this probability can depend in the older requests on the chosen sequence; Given a deterministic online paging algorithm A we define it’s competitiveness under a distribution ρ to be the infimum of C such that:**Cont.**Formally – the MiniMax principle gives us: Meaning of the left member: The competitiveness co efficient of the best random algorithm (Vs. an oblivious adversary). Meaning of the Right member: The competitiveness coefficient of the best deterministic online Alg. Under a “worst-case” input distribution probability.**Since we are looking for a lower bound we can look at a very**simple case We will look on a world where there are only K+1 memory items: I = {I1,……….,Ik+1} Since the cache can consist at most K items at once only one of the k + 1 items is out at each step So, we can say the algorithm decides which of the items will be out at each step. Theorem: Let R be a randomized algorithm for paging then**Choosing a “worst-case” distribution**First we should give P – an input probability distribution : The sequences to be chosen will be of length N – we assume N>>K. Choose the first request p1 uniformly from all the items in I. Choose the i’th request pi uniformly from the set : I\{pi-1}. i>1 Intuitive explanation: We never request an item that we are sure about it’s status – this is why this is a worst-case distribution.**Demonstration**I = {I1,I2,I3,I4,I5,I6,I7,I8,I9,I10,I11} current choice set = {I1,I2,I3,I5,I6,I7,I8,I9,I10,I11}**Now we need to find a lower bound for A – an optimal**det’ online algorithm We will use again the notion of rounds that Achiya has mentioned: The first round starts with the first request and ends when, in the first time all the k+1 items in I has been requested at least once. The next round starts with the next request and ends just before the request to the (k+1)th distinct item since the start of the current round. All next rounds acts the same.**How the offline algorithm behaves on each round?**The offline algorithm will use MIN on each round – the result will be that the item left out will be the one requested only at the last request of the round This means that the offline algorithm will miss only once during a round – at the last request of a round when it adds the missed item to the cache it puts out the last requested item of the next round**Demonstration**I3 I2 I2 I4 I4 I2 I3 I1 I2 I I I4 I4 I1 1 1 2 2 3 3 1 1 1 1 2 2 2 2 3 3 3 3 I = {I1,I2,I3,I4} Requests= {I1,I2,I1,I4,I3,I2,I4,I1,I3,I1,I3,I2,I4,I2,I4,I3}**How often does the offline algorithm miss?**We saw that the answer to that question depends the length of each round ( since it misses once in a round). We can ask an equivalent question: what is the expected number of random choices we need to do until we choose all items in a group of order K+1? Let’s look at it as a random walk on a complete graph with k+1 vertices – the expected number of steps to achieve full cover of that graph was approved to be : K*Hk**How a deterministic online algorithm behaves on each round?**Since the request sequence is random we can’t expect any regularity in the sequence. Since the policy of the known deterministic algorithms is based on some kind of regularity we can’t expect any of these policies to be better then the others. So – the probability for a miss is totally random.**Cont.**the probability that any request falls on the item that A leaves out at same time-step is exactly 1/k explanation: remember there is no item that will be requested twice consecutively so the next request is for one of the other k items that A might have choose to leave out. Given the expected length of a round we conclude that the expected number of misses by A during a round will be: (K*Hk)*(1/k) = Hk**Let’s finish the proof**We found that the any of the deterministic online algorithms misses (expected) Hk times the number of (expected) misses by an optimal offline algorithm This result is under the worst case input distribution we defined So we found Hk to be the competitiveness coefficient of A and by the MinMax principle we conclude this is also the competitiveness coefficient of an optimal randomized algorithm for our problem**Overview-part B**• Introduction • Paging against an oblivious – Definitions. • The players on this section – What is the goal? • Yao’s minimax theorem. • Proof of a Theorem on CR : • Preparations. • Behavior of the offline algorithm. • Behavior of a deterministic algorithm. • Results. • The marker algorithm • Description. • Proof of it’s competitiveness coefficient. • Summary**The Marker Algorithm**After we proved that there are randomized online algorithms with a competitiveness coefficient of Hk we will now see an example for a randomized online algorithm with a very close achievement. The algorithm require K bits (marker bits) – one for each cache location At The beginning all these bits are set to 0.**The Algorithm**Given a memory request: 1.1 if the requested Item is currently in cache – mark it’s location (set mark bit to 1). 1.2 else bring the Item into memory and evict an item by the following policy: 1.2.1 choose uniformly at random an unmarked memory location. 1.2.2 mark this location, replace the item in it with the currently requested item. 2. If all memory locations are marked - unmark all locations when the next request is given.