algorithms and datastructures n.
Skip this Video
Loading SlideShow in 5 Seconds..
Algorithms ( and Datastructures ) PowerPoint Presentation
Download Presentation
Algorithms ( and Datastructures )

Loading in 2 Seconds...

play fullscreen
1 / 26

Algorithms ( and Datastructures ) - PowerPoint PPT Presentation

  • Uploaded on

Algorithms ( and Datastructures ). Lecture 1 MAS 714 part 2 Hartmut Klauck. Algorithms. An algorithm is a procedure for performing a computation Algorithms consist of elementary steps/instructions Elementary steps depend on the model of computation

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Algorithms ( and Datastructures )' - havyn

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
algorithms and datastructures

Algorithms (andDatastructures)

Lecture 1

MAS 714 part 2

Hartmut Klauck

  • An algorithm is a procedure for performing a computation
  • Algorithmsconsist of elementary steps/instructions
  • Elementarystepsdepend on the model ofcomputation
    • Example: Turing machine: transitionfunction
    • Example: C++ commands
  • Algorithmsarenamed after Al-Khwārizmī(AbūʿAbdallāhMuḥammadibnMūsā al-Khwārizmī)c. 780-850 cePersianmathematicianandastronomer
  • (Algebra is also named after hiswork)
  • His worksbroughtthepositionalsystemofnumberstotheattentionofEuropeans
algorithms an example
Algorithms: an example
  • Addition via theschoolmethod:
    • Write numbersundereachother
    • Add numberpositionbypositionmoving a „carry“ forward
  • Elementaryoperations:
    • Add twonumbersbetween 0 and 9(memorized)
    • Read and Write
  • Can deal witharbitrarilylongnumbers!
  • The addition algorithm uses (implicitly) an array as datastructure
    • An array is a fixed length vector of cells that can each store a number
    • Note thatwhenweadd x and y thenx+yisatmost 1 digitlongerthanmax{x,y}
    • So theresult will bestored in an arrayoflength n+1
  • The schoolmultiplicationalgorithmis an exampleof a reduction
  • First we learn how to add n numbers with n digits each
  • To multiply x and y we generate n numbers xi¢ y¢2i and add them up
  • Reduction from Multiplication to Addition
  • Weusuallyanalyzealgorithmsto grade theperformance
  • The mostimportant (but not theonly) parametersare time andspace
  • Time referstothenumberofelementarysteps
  • Space referstothestorageneededduringthecomputation
example addition
Example: addition
  • Assumeweaddtwonumbersx,ywith n decimaldigits
  • Clearlythenumberofelementarysteps (addingdigitsetc) growslinearlywith n
  • Space is also O(n)
  • Typical: asymptoticanalysis
example multiplication
Example: multiplication
  • Wegenerate n numberswithatmost 2n digits, addthem
  • Number of steps is O(n2)
  • Space is O(n2)
  • Much fasteralgorithmsexist
    • (but not easy to do withpenandpaper)
  • Question: Is multiplicationharderthanaddition?
    • Answer: wedon‘tknow...
types of analysis
Typesof Analysis
  • Usuallyweuseasymptoticanalysis
    • Reason: nextyear‘scomputer will befaster, so constantfactorsdon‘t matter (usually)
    • Understand theinherentcomplexityof a problem (canyoumultiply in linear time?)
    • Usuallygivestherightanswer in practice
  • Worstcase
    • The running time of an algorithmisthemaximum time over all inputs
    • On thesafesidefor all inputs…
types of analysis1
  • Averagecase:
    • Averageunderwhichdistribution?
    • Oftenthe uniform distribution, but maybeunrealistic
  • Amortizedanalysis
    • Sometimes after somecostlypreparationswecansolvemanyprobleminstancescheaply
    • Count theaveragecostof an instance (preparationcostsarespreadbetweeninstances)
    • Often used in datastructure analysis
our model of computation
Our model ofcomputation
  • In princplewecoulduse Turing machines...
  • We will consideralgorithms in a richer model, formally a RAM
  • RAM:
    • randomaccessmachine
  • Basicallywe will just usepseudocode/informal language
  • Random Access Machine
    • Storage is made of registers that can hold a number (an unlimited amount of registers is available)
    • The machine is controlled by a finite program
    • Instructions are from a finite set that includes
      • Fetching data from a register into a special register
      • Arithmetic operations on registers
      • Writing into a register
      • Indirect addressing
  • Random Access Machines and Turing Machines can simulate each other
  • There is a universal RAM
  • RAM’s are very similar to actual computers
    • machine language
computation costs
Computation Costs
  • The time cost of a RAM step involving a register is the logarithm of the number stored
    • logarithmic cost measure
    • adding numbers with n bits takes time n etc.
  • The time cost of a RAM program is the sum of the time costs of its steps
  • Space is the sum (over all registers ever used) of the logarithms of the maximum numbers stored in the register
  • Computers spend a lot of time sorting!
  • Assume we have a list of numbers x1,…,xn from a universe U
  • For simplicity assume the xi are distinct
  • The goal is to compute a permutation ¼ such thatx ¼(1)< x¼(2) <  < x¼(n)
  • Think of a deck ofcards
insertion sort
Insertion Sort
  • An intuitive sorting algorithm
  • The input is provided in A[1…n]
  • Code:for i = 2:n, for (k = i; k > 1 and A[k] < A[k-1]; k--) swap A[k,k-1]→ invariant: A[1..i] is sortedend
  • Clear: O(n2) comparisons and O(n2) swaps
  • Algorithm works in place, i.e., uses linear space
  • By induction
  • Base case: n=2:We have one conditional swap operation, hence the output is sorted
  • Induction Hypothesis:After iteration i the elements A[1]…A[i] are sorted
  • Induction Step:Consider Step i+1. A[1]…A[i] are sorted already. Inner loop starts from A[i+1] and moves it to the correct position. After this A[1]…A[i+1] are sorted.
best case
Best Case
  • In the worst case Insertion Sort takes time (n2)
  • If the input sequence is already sorted the algorithms takes time O(n)
  • The same istrueiftheinputisalmostsorted
    • important in practice
  • Algorithmis simple and fast forsmall n
worst case
Worst Case
  • On someinputs Insertion Sorttakes(n2) steps
  • Proof: consider a sequencethatisdecreasing, e.g., n,n-1,n-2,…,2,1
  • Eachelementismovedfromposition i toposition 1
  • Hencetherunning time isat leasti=1,…,n i = (n2)
can we do better
Can we do better?
  • Attempt 1:Searchingforthepositiontoinsertthenextelementisinefficient, employbinarysearch
  • Orderedsearch:
    • Given an array A with n numbers in a sorted sequence, and a number x, find the smallest i such that A[i]¸ x
  • Use A[n+1]=1
linear search
Linear Search
  • Simplestwaytosearch
  • Run through A (from 1 to n) andcompare A[i] with x until A[i]¸ x is found, output i
  • Time: £(n)
  • Can also be used to search unsorted Arrays
binary search
Binary Search
  • If the array is sorted already, we can find an item much faster!
  • Assume we search for a x among A[1]<...<A[n]
  • Algorithm (to be called with l=1 and r=n):
    • BinSearch(x,A,l,r]
      • If r-l=0 test if A[l]=x, end
    • Compare A[(r-l)/2+l] with x
    • If A[(r-l)/2+l]=x output (r-l)/2+l, end
    • If A[(r-l)/2+l]> x BinSearch(x,A,l,(r-l)/2+l)
    • If A[(r-l)/2+l]<x Bin Search(x,a,(r-l)/2+l,r)
time of binary search
Time of Binary Search
  • Define T(n) as the time/number of comparisons needed on Arrays of length n
  • T(2)=1
  • T(n)=T(n/2)+1
  • Solution: T(n)=log(n)
  • All logs have basis 2
  • We just described an algorithm via recursion:a procedure that calls itself
  • This is often convenient but we must make sure that the recursion eventually terminates
    • Have a base case (here r=l)
    • Reduce some parameter in each call (here r-l)
binary insertion sort
Binary Insertion Sort
  • Using binary search in InsertionSort reduces the number of comparisons to O(n log n)
    • The outer loop is executed n times, each inner loop now uses log n comparisons
  • Unfortunatelythenumberofswapsdoes not decrease:
    • To insert an element we need to shift the remaining array to the right!