CS252 Graduate Computer Architecture Lecture 1 Review of Technology Trends and Cost/Performance

1. CS252Graduate Computer ArchitectureLecture 1Review of Technology Trends and Cost/Performance August 25, 2003 Prof. John Kubiatowicz http://www.cs.berkeley.edu/~kubitron/cs252-F03

2. Original Big Fishes Eating Little Fishes

3. 1988 Computer Food Chain Mainframe Work- station PC Mini- computer Mini- supercomputer Supercomputer Massively Parallel Processors

4. 1998 Computer Food Chain Mini- supercomputer Mini- computer Massively Parallel Processors Mainframe Work- station PC Server Now who is eating whom? Supercomputer

5. Why Such Change in 10 years? • Performance • Technology Advances • CMOS VLSI dominates older technologies (TTL, ECL) in cost AND performance • Computer architecture advances improves low-end • RISC, superscalar, RAID, … • Price: Lower costs due to … • Simpler development • CMOS VLSI: smaller systems, fewer components • Higher volumes • CMOS VLSI : same dev. cost 10,000 vs. 10,000,000 units • Lower margins by class of computer, due to fewer services • Function • Rise of networking/local interconnection technology

6. Amazing Underlying Technology Change • “Cramming More Components onto Integrated Circuits” • Gordon Moore, Electronics, 1965

7. Technology Trends: Microprocessor Capacity Pentium 4: 55 million Alpha 21264: 15 million Pentium Pro: 5.5 million PowerPC 620: 6.9 million Alpha 21164: 9.3 million Sparc Ultra: 5.2 million Moore’s Law • CMOS improvements: • Die size: 2X every 3 yrs • Line width: halve / 7 yrs

8. Memory Capacity (Single Chip DRAM) year size(Mb) cyc time 1980 0.0625 250 ns 1983 0.25 220 ns 1986 1 190 ns 1989 4 165 ns 1992 16 145 ns 1996 64 120 ns 2000 256 100 ns 2003 1024 60 ns

9. Technology  dramatic change • Processor • logic capacity: about 30% per year • clock rate: about 20% per year • Memory • DRAM capacity: about 60% per year (4x every 3 years) • Memory speed: about 10% per year • Cost per bit: improves about 25% per year • Disk • capacity: about 60% per year • Total use of data: 100% per 9 months! • Network Bandwidth • Bandwidth increasing more than 100% per year!

10. Computers in the News: New IBM Transistor • Announced 12/10/02 • 6nm gate length!!! • Details: Still to be announced

11. Processor PerformanceTrends 1000 Supercomputers 100 Mainframes 10 Minicomputers Microprocessors 1 0.1 1965 1970 1975 1980 1985 1990 1995 2000 Year

12. Processor Performance(1.35X before, 1.55X now) 1.54X/yr

13. Computer Architecture Is … the attributes of a [computing] system as seen by the programmer, i.e., the conceptual structure and functional behavior, as distinct from the organization of the data flows and controls the logic design, and the physical implementation. Amdahl, Blaaw, and Brooks, 1964 SOFTWARE

14. Computer Architecture’s Changing Definition • 1950s to 1960s: Computer Architecture Course: Computer Arithmetic • 1970s to mid 1980s: Computer Architecture Course: Instruction Set Design, especially ISA appropriate for compilers • 1990s: Computer Architecture Course:Design of CPU, memory system, I/O system, Multiprocessors, Networks • 2010s: Computer Architecture Course: Self adapting systems? Self organizing structures?DNA Systems/Quantum Computing?

15. Instruction Set Architecture (ISA) software instruction set hardware

16. Evolution of Instruction Sets Single Accumulator (EDSAC 1950) Accumulator + Index Registers (Manchester Mark I, IBM 700 series 1953) Separation of Programming Model from Implementation High-level Language Based Concept of a Family (B5000 1963) (IBM 360 1964) General Purpose Register Machines Complex Instruction Sets Load/Store Architecture (CDC 6600, Cray 1 1963-76) (Vax, Intel 432 1977-80) RISC (Mips,Sparc,HP-PA,IBM RS6000, . . .1987)

17. Interface Design • A good interface: • Lasts through many implementations (portability, compatibility) • Is used in many differeny ways (generality) • Provides convenient functionality to higher levels • Permits an efficient implementation at lower levels use time imp 1 Interface use imp 2 use imp 3

18. Virtualization:One of the lessons of RISC • Integrated Systems Approach • What really matters is the functioning of the complete system, I.e. hardware, runtime system, compiler, and operating system • In networking, this is called the “End to End argument” • Programmers care about high-level languages, debuggers, source-level object-oriented programming • Computer architecture is not just about transistors, individual instructions, or particular implementations • Original RISC projects replaced complex instructions with a compiler + simple instructions • Logical Extension => Genetically adaptive runtime systems enhanced by dynamic compilation running on reconfigurable hardware? Perhaps.

19. Computer Architecture Topics Input/Output and Storage Disks, WORM, Tape RAID Emerging Technologies Interleaving Bus protocols DRAM Coherence, Bandwidth, Latency Memory Hierarchy L2 Cache Network Communication Other Processors L1 Cache Addressing, Protection, Exception Handling VLSI Instruction Set Architecture Pipelining and Instruction Level Parallelism Pipelining, Hazard Resolution, Superscalar, Reordering, Prediction, Speculation, Vector, Dynamic Compilation

20. Proc Caches Busses adapters Memory Controllers Disks Displays Keyboards I/O Devices: Networks Sample Organization: It’s all about communication Pentium III Chipset

21. Computer Architecture Topics Shared Memory, Message Passing, Data Parallelism P M P M P M P M ° ° ° Network Interfaces S Interconnection Network Processor-Memory-Switch Topologies, Routing, Bandwidth, Latency, Reliability Multiprocessors Networks and Interconnections

22. CS 252 Course Focus Understanding the design techniques, machine structures, technology factors, evaluation methods that will determine the form of computers in 21st Century Parallelism Technology Programming Languages Applications Interface Design (ISA) Computer Architecture: • Instruction Set Design • Organization • Hardware/Software Boundary Compilers Operating Measurement & Evaluation History Systems

23. Topic Coverage Textbook: Hennessy and Patterson, Computer Architecture: A Quantitative Approach, 3rd Ed., 2002. Research Papers -- Handed out in class • 1.5 weeks Review: Fundamentals of Computer Architecture (Ch. 1), Instruction Set Architecture (Ch. 2), Pipelining (Ch. 3) • 2.5 weeks: Pipelining, Interrupts, and Instructional Level Parallelism (Ch. 4), Vector Processors (Appendix B). • 1.5 weeks: Dynamic Compilation. Data Speculation (papers). Complexity, design via genetic algorithms • 1 week: Memory Hierarchy (Chapter 5) • 1.5 weeks: Fault Tolerance, Input/Output and Storage (Ch. 6) • 1.5 weeks: Networks and Interconnection Technology (Ch. 7) • 1.5 weeks: Multiprocessors (Ch. 8 + Research papers + Culler book draft Chapter 1) • 1 week: Quantum Computing, DNA Computing

24. CS252: Information Instructor:Prof John D. Kubiatowicz Office: 673 Soda Hall, 643-6817 kubitron@cs Office Hours: Wed 3:30 - 5:00 or by appt. (Contact Veronique Richards, 642-4334, nicou@cs, 676 Soda) T. A: TBA Class: Mon/Wed, 1:00 - 2:30pm 310 Soda Hall Text: Computer Architecture: A Quantitative Approach, Third Edition (2002) Web page: http://www.cs/~kubitron/courses/cs252-F03/ Lectures available online <11:30AM day of lecture Newsgroup: ucb.class.cs252 Email: cs252@kubi.cs.berkeley.edu

25. Lecture style • 1-Minute Review • 20-Minute Lecture/Discussion • 5- Minute Administrative Matters • 25-Minute Lecture/Discussion • 5-Minute Break (water, stretch) • 25-Minute Lecture/Discussion • Instructor will come to class early & stay after to answer questions Attention 20 min. Break “In Conclusion, ...” Time

26. Grading • 10% Homeworks (work in pairs) • 40% Examinations (2 Midterms) • 40% Research Project (work in pairs) • Transition from undergrad to grad student • Berkeley wants you to succeed, but you need to show initiative • pick topic • meet 3 times with faculty/TA to see progress • give oral presentation • give poster session • written report like conference paper • 3 weeks work full time for 2 people • Opportunity to do “research in the small” to help make transition from good student to research colleague • 10% Class Participation

27. Quizes • Reduce the pressure of taking quizes • Only 2 Graded Quizes: Tentative: Wed Oct 13th and Wed Dec 1st • Our goal: test knowledge vs. speed writing • 3 hrs to take 1.5-hr test (5:30-8:30 PM, TBA location) • Both mid-term quizes can bring summary sheet • Transfer ideas from book to paper • Last chance Q&A: during class time day of exam • Students/Staff meet over free pizza/drinks at La Vals: Wed Oct 13th (8:30 PM) and Wed Dec 1st (8:30 PM)

28. Research Paper Reading • As graduate students, you are now researchers. • Most information of importance to you will be in research papers. • Ability to rapidly scan and understand research papers is key to your success. • So: you will read lots of papers in this course! • Quick 1 paragraph summaries will be due in class • Important supplement to book. • Will discuss papers in class • Papers will be scanned and on web page.

29. More Course Info • Everything is on the course Web page: www.cs.berkeley.edu/~kubitron/courses/cs252-F03 • Notes: • Not sure what the state of textbooks at Student Center. • The course Web page includes a pointer to last term’s 152 home page. The “handout” page includes pointers to old 152 quizes. • Schedule: • 2 Graded Quizes: Mon Oct 13th and Mon Dec 1st • Veteran’s Day: Friday Nov 5th • Thanksgiving Vacation: Thur Nov 27th - Sun Nov 28th • Oral Presentations: Tue/Wed Dec 9/10th • 252 Last lecture: Fri Dec 3rd • 252 Poster Session: ??? • Project Papers/URLs due: Fri Dec 12th • Project Suggestions: TBA

30. Basic knowledge of the organization of a computer is assumed! Related Courses Strong Prerequisite CS 152 CS 252 CS 258 Why, Analysis, Evaluation Parallel Architectures, Languages, Systems How to build it Implementation details CS 250 Integrated Circuit Technology from a computer-organization viewpoint

31. Coping with CS 252 • Too many students with too varied background? • Next Wednesday - Prequisite exam • Limiting Number of Students • First priority is CS/ EECS grad students taking prelims • Second priority is N-th year CS/ EECS grad students (breadth) • Third priority is College of Engineering grad students • Fourth priority is CS/EECS undergraduate seniors (Note: 1 graduate course unit = 2 undergraduate course units) • All other categories • If not this semester, 252 is offered regularly

32. Coping with CS 252 • Students with too varied background? • In past, CS grad students took written prelim exams on undergraduate material in hardware, software, and theory • 1st 5 weeks reviewed background, helped 252, 262, 270 • Prelims were dropped => some unprepared for CS 252? • In class exam on Wednesday September 3rd • Doesn’t affect grade, only admission into class • 2 grades: Admitted or audit/take CS 152 1st • Improve your experience if recapture common background • Review: Chapters 1-3, CS 152 home page, maybe “Computer Organization and Design (COD)2/e” • Chapters 1 to 8 of COD if never took prerequisite • If took a class, be sure COD Chapters 2, 6, 7 are familiar • Copies in Bechtel Library on 2-hour reserve • Last exam on previous-year’s web site (~kubitron/courses/cs252-F00)

33. Building Hardwarethat Computes

34. 1 Alpha/ 0 Beta/ 1 1 0 0 0 106 0 1 0 1 0 Delta/ 2 0 1 2 2 1 Mod 3 1 1 Finite State Machines: • System state is explicit in representation • Transitions between states represented as arrows with inputs on arcs. • Output may be either part of state or on arcs “Mod 3 Machine” Input (MSB first) 1 1 1 0

35. 1/0 “Moore Machine” Latch Alpha/ 0 Beta/ 1 1/1 Combinational Logic 0/0 0/1 0/0 Delta/ 2 “Mealey Machine” 1/1 Implementation as Combinational logic + Latch

36. State w/ Address Instruction Branch Combinational Logic/ Controlled Machine 0: forw 35 xxx 1: b_no_obstacles 000 2: back 10 xxx 3: rotate 90 xxx 4: goto 001 Control ROM (Instructions) Addr Branch PC + 1 Next Address MUX Microprogrammed Controllers • State machine in which part of state is a “micro-pc”. • Explicit circuitry for incrementing or changing PC • Includes a ROM with “microinstructions”. • Controlled logic implements at least branches and jumps

37. Instruction Fetch Instruction Decode Operand Fetch Execute Result Store Next Instruction Execution Cycle Obtain instruction from program storage Determine required actions and instruction size Locate and obtain operand data Compute result value or status Deposit results in storage for later use Determine successor instruction

38. What’s a Clock Cycle? Latch or register combinational logic • Old days: 10 levels of gates • Today: determined by numerous time-of-flight issues + gate delays • clock propagation, wire lengths, drivers

39. Next Instruction NI NI NI NI NI IF IF IF IF IF D D D D D Instruction Fetch E E E E E W W W W W Decode & Operand Fetch Execute Store Results Pipelined Instruction Interpretation Instruction Address Instruction Register Time Operand Registers Result Registers Registers or Mem

40. A B C D Sequential Laundry 6 PM Midnight 7 8 9 11 10 Time • Sequential laundry takes 6 hours for 4 loads • If they learned pipelining, how long would laundry take? 30 40 20 30 40 20 30 40 20 30 40 20 T a s k O r d e r

41. 30 40 40 40 40 20 A B C D Pipelined LaundryStart work ASAP 6 PM Midnight 7 8 9 11 10 • Pipelined laundry takes 3.5 hours for 4 loads Time T a s k O r d e r

42. 30 40 40 40 40 20 A B C D Pipelining Lessons 6 PM 7 8 9 • Pipelining doesn’t help latency of single task, it helps throughput of entire workload • Pipeline rate limited by slowest pipeline stage • Multiple tasks operating simultaneously • Potential speedup = Number pipe stages • Unbalanced lengths of pipe stages reduces speedup • Time to “fill” pipeline and time to “drain” it reduces speedup Time T a s k O r d e r

43. The Process of Design • Architecture is an iterative process: • Searching the space of possible designs • At all levels of computer systems Creativity Cost / Performance Analysis Good Ideas Mediocre Ideas Bad Ideas

44. Measurement Tools • Benchmarks, Traces, Mixes • Hardware: Cost, delay, area, power estimation • Simulation (many levels) • ISA, RT, Gate, Circuit • Queuing Theory • Rules of Thumb • Fundamental “Laws”/Principles

45. DC to Paris Speed Passengers Throughput (pmph) 6.5 hours 610 mph 470 286,700 3 hours 1350 mph 132 178,200 The Bottom Line: Performance (and Cost) Plane Boeing 747 BAD/Sud Concodre • Time to run the task (ExTime) • Execution time, response time, latency • Tasks per day, hour, week, sec, ns … (Performance) • Throughput, bandwidth

46. performance(x) = 1 execution_time(x) Performance(X) Execution_time(Y) n = = Performance(Y) Execution_time(Y) Definitions • Performance is in units of things per sec • bigger is better • If we are primarily concerned with response time " X is n times faster than Y" means

47. Amdahl’s Law Best you could ever hope to do:

48. Metrics of Performance Application Answers per month Operations per second Programming Language Compiler (millions) of Instructions per second: MIPS (millions) of (FP) operations per second: MFLOP/s ISA Datapath Megabytes per second Control Function Units Cycles per second (clock rate) Transistors Wires Pins

49. CPU time = Seconds = Instructions x Cycles x Seconds Program Program Instruction Cycle CPI Computer Performance inst count Cycle time Inst Count CPI Clock Rate Program X Compiler X (X) Inst. Set. X X Organization X X Technology X

50. Cycles Per Instruction(Throughput) “Average Cycles per Instruction” • CPI = (CPU Time * Clock Rate) / Instruction Count • = Cycles / Instruction Count “Instruction Frequency”