Put Everything
Download
1 / 23

- PowerPoint PPT Presentation


  • 131 Views
  • Uploaded on

Put Everything in Future (Disk) Controllers (it’s not “if”, it’s “when?”) Jim Gray http://www.research.Microsoft.com/~Gray Acknowledgements : Dave Patterson explained this to me a year ago Kim Keeton Erik Riedel Catharine Van Ingen. Helped me sharpen these arguments. Remember Your Roots.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '' - emma


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Put Everything in Future (Disk) Controllers(it’s not “if”, it’s “when?”)Jim Grayhttp://www.research.Microsoft.com/~GrayAcknowledgements:Dave Patterson explained this to me a year agoKim KeetonErik RiedelCatharine Van Ingen

Helped me sharpen

these arguments



Technology drivers disks l.jpg

Kilo

Mega

Giga

Tera

Peta

Exa

Zetta

Yotta

Technology Drivers: Disks

  • Disks on track

  • 100x in 10 years 2 TB 3.5” drive

  • Shrink to 1” is 200GB

  • Disk replaces tape?

  • Disk is super computer!


Data gravity processing moves to transducers l.jpg
Data GravityProcessing Moves to Transducers

  • Move Processing to data sources

  • Move to where the power (and sheet metal) is

  • Processor in

    • Modem

    • Display

    • Microphones (speech recognition) & cameras (vision)

    • Storage: Data storage and analysis


It s already true of printers peripheral cyberbrick l.jpg
It’s Already True of PrintersPeripheral = CyberBrick

  • You buy a printer

  • You get a

    • several network interfaces

    • A Postscript engine

      • cpu,

      • memory,

      • software,

      • a spooler (soon)

    • and… a print engine.


All device controllers will be cray 1 s l.jpg
All Device Controllers will be Cray 1’s

Central

Processor & Memory

  • TODAY

    • Disk controller is 10 mips risc engine with 2MB DRAM

    • NIC is similar power

  • SOON

    • Will become 100 mips systems with 100 MB DRAM.

  • They are nodes in a federation(can run Oracle on NT in disk controller).

  • Advantages

    • Uniform programming model

    • Great tools

    • Security

    • economics (CyberBricks)

    • Move computation to data (minimize traffic)

Tera Byte

Backplane


Basic argument for x disks l.jpg
Basic Argument for x-Disks

  • Future disk controller is a super-computer.

    • 1 bips processor

    • 128 MB dram

    • 100 GB disk plus one arm

  • Connects to SAN via high-level protocols

    • RPC, HTTP, DCOM, Kerberos, Directory Services,….

    • Commands are RPCs

    • management, security,….

    • Services file/web/db/… requests

    • Managed by general-purpose OS with good dev environment

  • Move apps to disk to save data movement

    • need programming environment in controller


The slippery slope l.jpg
The Slippery Slope

Nothing =

Sector Server

  • If you add function to server

  • Then you add more function to server

  • Function gravitates to data.

Something =

Fixed App Server

Everything =

App Server


Why not a sector server let s get physical l.jpg
Why Not a Sector Server?(let’s get physical!)

  • Good idea, that’s what we have today.

  • But

    • cache added for performance

    • Sector remap added for fault tolerance

    • error reporting and diagnostics added

    • SCSI commends (reserve,.. are growing)

    • Sharing problematic (space mgmt, security,…)

  • Slipping down the slope to a 2-D block server


Why not a 1 d block server put a little on the disk server l.jpg
Why Not a 1-D Block Server?Put A LITTLE on the Disk Server

  • Tried and true design

    • HSC - VAX cluster

    • EMC

    • IBM Sysplex (3980?)

  • But look inside

    • Has a cache

    • Has space management

    • Has error reporting & management

    • Has RAID 0, 1, 2, 3, 4, 5, 10, 50,…

    • Has locking

    • Has remote replication

    • Has an OS

    • Security is problematic

    • Low-level interface moves too many bytes


Why not a 2 d block server put a little on the disk server l.jpg
Why Not a 2-D Block Server?Put A LITTLE on the Disk Server

  • Tried and true design

    • Cedar -> NFS

    • file server, cache, space,..

    • Open file is many fewer msgs

  • Grows to have

    • Directories + Naming

    • Authentication + access control

    • RAID 0, 1, 2, 3, 4, 5, 10, 50,…

    • Locking

    • Backup/restore/admin

    • Cooperative caching with client

  • File Servers are a BIG hit: NetWare™

    • SNAP! is my favorite today


Why not a file server put a little on the disk server l.jpg
Why Not a File Server?Put a Little on the Disk Server

  • Tried and true design

    • Auspex, NetApp, ...

    • Netware

  • Yes, but look at NetWare

    • File interface gives you app invocation interface

    • Became an app server

      • Mail, DB, Web,….

    • Netware had a primitive OS

      • Hard to program, so optimized wrong thing


Why not everything allow everything on disk server thin client s l.jpg
Why Not Everything?Allow Everything on Disk Server(thin client’s)

  • Tried and true design

    • Mainframes, Minis, ...

    • Web servers,…

    • Encapsulates data

    • Minimizes data moves

    • Scaleable

  • It is where everyone ends up.

  • All the arguments against are short-term.


The slippery slope14 l.jpg
The Slippery Slope

Nothing =

Sector Server

  • If you add function to server

  • Then you add more function to server

  • Function gravitates to data.

Something =

Fixed App Server

Everything =

App Server


Disk node l.jpg
Disk = Node

  • has magnetic storage (100 GB?)

  • has processor & DRAM

  • has SAN attachment

  • has execution environment

Applications

Services

DBMS

RPC, ...

File System

SAN driver

Disk driver

OS Kernel


Technology drivers system on a chip l.jpg
Technology Drivers: System on a Chip

  • Integrate Processing with memory on one chip

    • chip is 75% memory now

    • 1MB cache >> 1960 supercomputers

    • 256 Mb memory chip is 32 MB!

    • IRAM, CRAM, PIM,… projects abound

  • Integrate Networking with processing on one chip

    • system bus is a kind of network

    • ATM, FiberChannel, Ethernet,.. Logic on chip.

    • Direct IO (no intermediate bus)

  • Functionally specialized cards shrink to a chip.


Technology drivers what if networking was as cheap as disk io l.jpg

TCP/IP

Unix/NT 100% cpu @ 40MBps

Disk

Unix/NT 8% cpu @ 40MBps

Why the Difference?

Host does

TCP/IP packetizing,

checksum,…

flow control

small buffers

Host Bus Adapter does

SCSI packetizing,

checksum,…

flow control

DMA

Technology Drivers: What if Networking Was as Cheap As Disk IO?


Technology drivers the promise of san via 10x in 2 years http www viarch org l.jpg
Technology Drivers: The Promise of SAN/VIA:10x in 2 yearshttp://www.viarch.org/

  • Today:

    • wires are 10 MBps (100 Mbps Ethernet)

    • ~20 MBps tcp/ip saturates 2 cpus

    • round-trip latency is ~300 us

  • In the lab

    • Wires are 10x faster Myrinet, Gbps Ethernet, ServerNet,…

    • Fast user-level communication

      • tcp/ip ~ 100 MBps 10% of each processor

      • round-trip latency is 15 us


San standard interconnect l.jpg

RIP

FDDI

RIP

ATM

RIP

FC

RIP

SCI

RIP

?

RIP

SCSI

SAN: Standard Interconnect

Gbps Ethernet: 110 MBps

  • LAN faster than memory bus?

  • 1 GBps links in lab.

  • 100$ port cost soon

  • Port is computer

PCI: 70 MBps

UW Scsi: 40 MBps

FW scsi: 20 MBps

scsi: 5 MBps


Technology drivers 100 gbps ethernet replaces scsi l.jpg
Technology Drivers:100 GBps Ethernet replaces SCSI

  • Why I love SCSI

    • Its fast (40MBps)

    • The protocol uses little processor power

  • Why I hate SCSI

    • Wires must be short

    • Cables are pricy

    • pins bend


Functionally specialized cards l.jpg
Functionally Specialized Cards

P mips processor

Today:

P=50 mips

M= 2 MB

ASIC

  • Storage

  • Network

  • Display

M MB DRAM

In a few years

P= 200 mips

M= 64 MB

ASIC

ASIC


Technology drivers plug play software l.jpg
Technology DriversPlug & Play Software

  • RPC is standardizing: (DCOM, IIOP, HTTP)

    • Gives huge TOOL LEVERAGE

    • Solves the hard problems for you:

      • naming,

      • security,

      • directory service,

      • operations,...

  • Commoditized programming environments

    • FreeBSD, Linix, Solaris,…+ tools

    • NetWare + tools

    • WinCE, WinNT,…+ tools

    • JavaOS + tools

  • Apps gravitate to data.

  • General purpose OS on dedicated ctlr can run apps.


Basic argument for x disks23 l.jpg
Basic Argument for x-Disks

  • Future disk controller is a super-computer.

    • 1 bips processor

    • 128 MB dram

    • 100 GB disk plus one arm

  • Connects to SAN via high-level protocols

    • RPC, HTTP, DCOM, Kerberos, Directory Services,….

    • Commands are RPCs

    • management, security,….

    • Services file/web/db/… requests

    • Managed by general-purpose OS with good dev environment

  • Move apps to disk to save data movement

    • need programming environment in controller


ad