Intel research @ berkeley and extreme networked systems www intel research net berkeley
Download
1 / 39

Intel Research Berkeley and Extreme Networked Systems intel-research - PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on

Intel Research @ Berkeley and Extreme Networked Systems www.intel-research.net/berkeley. David Culler 8/12/2002. Where this presentation might go. aka Outline new models of industry/academic research collaboration vast networks of tiny devices in the physical world

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Intel Research Berkeley and Extreme Networked Systems intel-research' - derick


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Intel research @ berkeley and extreme networked systems www intel research net berkeley

Intel Research @ Berkeleyand Extreme Networked Systemswww.intel-research.net/berkeley

David Culler

8/12/2002


Where this presentation might go
Where this presentation might go...

aka Outline

  • new models of industry/academic research collaboration

  • vast networks of tiny devices in the physical world

  • open infrastructure for emerging planetary-scale services

IRB/XIS


New model for ind acad collaboration
New model for ind/acad collaboration

  • Key challenges ahead in EECS are fundamentally problems of scale

    • require level of investigation and engineering beyond what is sustainable within the university and beyond what a company can commit outside product scope

    • industry possesses key technology and expertise

    • requires insights from many perspectives

  • A new lab stucture built around deep research collaboration and intimate ties to the EECS department

    • industry contributes substantial effort of high quality

    • projects span boundaries

    • faculty co-direct lab

    • student / faculty cycles drive the continuous motion

  • Operate in uniquely open fashion

IRB/XIS


Intel network of lablets concept
Intel Network of Lablets Concept

  • Network of small labs working closely with top computer science departments around the world on deeply collaborative projects.

    • Berkeley – extreme network systems

    • Washington – HCI

    • CMU – distributed storage

    • Cambridge

  • Complement the corporate labs

    • explore off the roadmap, long range, high risk

  • Complement the external-research council

    • drive projects of significant scale and impact

  • Expand the channel

    • Bi-directional transfer of people, ideas, technology

IRB/XIS


Lablet mission
lablet mission

  • Leadership role in emerging and important areas

  • Combining the unique strengths of Intel and Univ.

  • Bi-directional exchange of breakthough ideas, technology and people

University

Advance of the

research ecosystem

Lablet

SRPs

Novel component

technology

Advanced

Applications

Intel Labs

IRB/XIS


Berkeley emphasis
Berkeley Emphasis

  • Cross-cutting problems of scale.

  • Extreme Interconnected Systems

  • “endonets”

    • dense, fine-grain networked systems deeply embedded in or interacting with physical environment

    • sensor networks

    • ubiquitous computing architectures

    • computational fabrics, surfaces, structures

  • “exonets”

    • broad coverage networked systems at societal scale

    • world-wide storage systems

    • composable infrastructure services

    • massive servers for millions of users

IRB/XIS


Scale and structure
Scale and structure

Active day-to-day involvement

  • ~20 full-time Intel Researchers and Engineers

    • currently 13

  • ~5 part-time Intel folks

  • 20 faculty, students, visitors, research consultants

    Two-in-a-box co-directors

  • University Director + Intel Director

  • Report to David Tennenhouse, VP Research

    Project focused

  • ~6-year projects starting about every two years

IRB/XIS


Two major lab projects
Two Major Lab Projects

  • Define and Develop complete ‘network system stack’ for deeply embedded sensor/effector networks

    • enabling technology

    • create the community

    • core architecture, OS, networking, service foundations

    • demonstrate revolutionary applications

  • Create an Open Laboratory for Widely-distributed “Planetary Scale” Services to explore architecture, services and applications

    • enabling resource catalyzes community

    • distributed development effort

    • foundations: scalable, secure slice-able platform

    • infra and service design trade-offs (DHT, Dist-storage)

IRB/XIS


Open collaborative research agreement
Open Collaborative Research Agreement

  • Master Agreement states

    • intent: Open

    • terms, conditions (IP addendum)

  • Research Project Descriptions

    • what, who, where

  • scope of work defines boundary of openness!

    • an openness agreement is all about defining reach-through

IRB/XIS



Bridging the technology appln gap

Monitoring & Managing Spaces and Things

application

service

data

mgmt

prog / data model

network

system

architecture

technology

Bridging the Technology-Appln Gap

mgmt / diag / debug

algorithm / theory

IRB/XIS


Deeply embedded networks
Deeply Embedded Networks

  • # nodes >> # people

  • sensor/actuator data stream

  • unattended

  • inaccessible

  • prolonged deployment

  • energy constrained

  • operate in aggregate

  • in-network processing necessary

  • what they do changes over time

    => must be programmed over the network

IRB/XIS


Project activities
Project Activities

  • Core Platform

    • architecture, TinyOS, Networking

    • simulation and debugging tools

  • Programming Support

    • NesC (TinyOS modularity and concurrency)

    • Cooperating FSMs, atomicity

    • Macroprogramming

  • Sensor-Network databases

    • streaming, noisy data, with in-network query processing

  • Delay Tolerant Networking

    • overlay for diverse, challenged internets

  • Interactive Environments and Things

    • ambient displays, remote physical communication

    • context-aware tools for the handicapped

  • Habitat and Environmental Monitoring

    • dense sensor networks in the hands of life scientists

  • Generic Sensor Kit

IRB/XIS


Platform architecture
Platform Architecture

  • Goal

    • create a small wireless device that would enable us to explore the system design space, applns to be attempted, and a new research community

    • develop the architecture in response to observed system design

  • Approach

    • joined in the series of UCB COTS mote designs

      • WeC -> Rene -> iDot -> MICA

    • look to silicon for full architecture

  • New ideas

    • rich interfaces allow radical system optimizations

      • analog wake-up, Tx-Rx time synch

    • federation of accelerators, not dedicate protocol proc.

    • HW/SW multithreading for low power, passive vigilance

application

service

data

mgmt

network

system

architecture

technology

IRB/XIS



Tinyos application graph
TinyOS Application Graph

Route map

router

sensor appln

application

Active Messages

Serial Packet

Radio Packet

packet

Temp

photo

SW

Example: self-organized ad- hoc, multi-hop routing of photo sensor readings

HW

UART

Radio byte

ADC

byte

3450 B code

226 B data

clocks

RFM

bit

Graph of cooperating

state machines

on shared stack

IRB/XIS


It is a noisy world after all
It is a noisy world after all...

  • Get to rethink each of the layers in a new context

    • coding, framing

    • mac

    • routing

    • transport,

    • rate control

    • discovery

    • multicast

    • aggregation

    • naming

    • security

    • ...

  • Resource constrained, power aware, highly variable, ...

  • Every node is also a router

  • No entrenched ‘dusty packets’

probability of reception from center node vs xmit strength

IRB/XIS



Habitat monitoring

LAN

WAN

(satcast)

Acadia National Park

Mt. Desert Island, ME

sensor nets

Great Duck Island

Nature Conservancy

Ongoing research

Habitat Monitoring

http://www.greatduckisland.net

IRB/XIS


Cross cutting issues
Cross-cutting issues?

  • Programming environments

  • Deep & scalable simulation

  • Algorithm behavior at scale

  • Operating on prob. distributions

  • Fine-Grain Inverse problems

  • Pseudo-imaging

  • Constructive foundations of self-organization

application

service

data mgmt

prog / data model

network

mgmt / diag / debug

algorithm / theory

system

architecture

technology

IRB/XIS


The other extreme planetary scale services
The Other Extreme - Planetary Scale Services

www.planet-lab.org

IRB/XIS


Motivation
Motivation

  • A new class of services & applications is emerging that spread over a sizable fraction of the web

    • CDNs as the first examples

    • Peer-to-peer, ...

  • Architectural components are beginning to emerge

    • Distributed hash tables to provide scalable translation

    • Distributed storage, caching, instrumentation, mapping, events ...

  • The next internet will be created as an overlay on the current one

    • as did the last one

    • it will be defined by its services, not its transport

      • translation, storage, caching, event notification, management

  • There will soon be vehicle to try out the next n great ideas in this area

IRB/XIS


Confluence of technologies
Confluence of Technologies

  • Cluster-based scalable distribution, remote execution, management, monitoring tools

    • UCB Millennium, OSCAR, ..., Utah Emulab, ModelNet...

  • CDNS and P2Ps

    • Gnutella, Kazaa, ... ,Pastry, Chord, CAN, Tapestry

  • Proxies routine

  • Virtual machines & Sandboxing

    • VMWare, Janos, Denali,... web-host slices (EnSim)

  • Overlay networks becoming ubiquitous

    • XBONE, RON, Detour... Akamai, Digital Island, ....

  • Service Composition Frameworks

    • yahoo, ninja, .net, websphere, Eliza

  • Established internet ‘crossroads’ – colos

  • Web Services / Utility Computing

  • Grid authentication infrastructure

  • Packet processing,

    • Anets, .... layer 7 switches, NATs, firewalls

  • Internet instrumentation

The Time is NOW

IRB/XIS


Guidelines 1
Guidelines (1)

  • Thousand viewpoints on “the cloud” is what matters

    • not the thousand servers

    • not the routers, per se

    • not the pipes, per se

IRB/XIS


Guidelines 2
Guidelines (2)

  • and you miust have the vantage points of the crossroads

    • primarily co-location centers

IRB/XIS


Guidelines 3
Guidelines (3)

  • Each service needs an overlay covering many points

    • logically isolated

  • Many concurrent services and applications

    • must be able to slice nodes => VM per service

    • service has a slice across large subset

  • Must be able to run each service / app over long period to build meaningful workload

    • traffic capture/generator must be part of facility

  • Consensus on “a node” more important than “which node”

IRB/XIS


Guidelines 4
Guidelines (4)

  • Test-lab as a whole must be up a lot

    • global remote administration and management

      • mission control

    • redundancy within

  • Each service will require its own remote management capability

  • Testlab nodes cannot “bring down” their site

    • generally not on main forwarding path

    • proxy path

    • must be able to extend overlay out to user nodes?

  • Relationship to firewalls and proxies is key

Management, Management, Management

IRB/XIS


Guidelines 5
Guidelines (5)

  • Storage has to be a part of it

    • edge nodes have significant capacity

  • Needs a basic well-managed capability

    • but growing to the [email protected] model should be considered at some stage

    • may be essential for some services

IRB/XIS


Initial researchers mar 02

http://www.planet-lab.org/

Initial Researchers (mar 02)

  • Intel Research

    • David Culler

    • Timothy Roscoe

    • Sylvia Ratnasamy

    • Gaetano Borriello

    • Satya

    • Milan Milenkovic

  • Duke

    • Amin Vadat

    • Jeff Chase

  • Princeton

    • Larry Peterson

    • Randy Wang

    • Vivek Pai

Washington

Tom Anderson

Steven Gribble

David Wetherall

MIT

Frans Kaashoek

Hari Balakrishnan

Robert Morris

David Anderson

Berkeley

Ion Stoica

Joe Helerstein

Eric Brewer

John Kubi

  • Rice

    • Peter Druschel

  • Utah

    • Jay Lepreau

  • CMU

    • Srini Seshan

    • Hui Zhang

  • UCSD

    • Stefan Savage

  • Columbia

    • Andrew Campbell

  • ICIR

    • Scott Shenker

    • Mark Handley

    • Eddie Kohler

IRB/XIS


Initial planet lab candidate sites
Initial Planet-Lab Candidate Sites

Uppsala

UBC

Copenhagen

UW

Cambridge

WI

Chicago

UPenn

Amsterdam

Harvard

Utah

Intel Seattle

Karlsruhe

Tokyo

Intel

MIT

Intel OR

Beijing

Barcelona

Intel Berkeley

Cornell

CMU

ICIR

Princeton

UCB

St. Louis

Columbia

Duke

UCSB

Washu

KY

UCLA

GIT

Rice

UCSD

UT

ISI

Melbourne

IRB/XIS


Approach service centric virtualization
Approach:Service-Centric Virtualization

  • Virtual Machine Technology has re-emerged for hosting complete desktop environments on non-native OS’s and potentially on machine monitors.

    • ex. VMWare, ...

  • Sandboxing has emerged to emulate multiple virtual machines per server with limited /bin, (no /dev)

    • ex. ENSim web hosting

  • Network Services require fundamentally simpler virtual machines, can be made far more scalable (VMs per PM), focused on service requirements

    • ex. Jail, Denali, scalable and fast, but no full legacy OS

    • access to overlays (controlled access to raw sockets)

    • allocation & isolation

      • proportional scheduling across resource container - CPU, net, disk

    • foundation of security model

    • fast packet/flow processing puts specific design pressures

  • Instrumentation and management are additional virtualized ‘slices’

    • distributed workload generation, data collection

IRB/XIS


Hard problems challenges
Hard problems/challenges

  • “Slice-ability” – multiple experimental services deployed over many nodes

    • Distributed Virtualization

    • Isolation & Resource Containment

    • Proportional Scheduling

    • Scalability

  • Security & Integrity - remotely accessed and fully exposed

    • Authentication / Key Infrastructure proven, if only systems were bug free

    • Build secure scalable platform for distributed services

      • Narrow API vs. Tiny Machine Monitor

  • Management

    • Resource Discovery, Provisioning, Overlay->IP

    • Create management services (not people) and environment for innovation in management

      • Deal with many as if one

  • Building Blocks and Primitives

    • Ubiquitous overlays

  • Instrumentation

IRB/XIS


Emerging extreme internet

Wide-Area Broad-Coverage Services

Emerging Extreme Internet

Deeply-

Embedded

Networks

Traditional pt-pt Internet

IRB/XIS


Backup
backup

IRB/XIS


Mission for the network of labs
Mission for the Network of Labs

  • Bold new form of Industry-University collaboration that reflects the changing nature of the information age.

  • Conduct the highest quality research in emerging, important areas of CS and IT.

  • Join the unique strengths of Universities and the company in concurrent, collaborative efforts that are both broad in scope and deeply penetrating in exploration.

  • Operate in a uniquely open fashion, promoting a powerful, bidirectional exchange of groundbreaking ideas, technology, and people.

  • Leadership role in the creation of new research ecosystems spanning the continuum from academic study to product development.

  • Labs will be project-focused with an active, constantly evolving agenda involving Intel researchers, University researchers, and members of the larger research community

IRB/XIS


Berkeley focus
Berkeley Focus

Extreme Interconnected Systems

  • Invent, develop, explore, analyze, and understand highly interconnected systems at the extremes of the computing and networking spectrum - the very large, the very small, and the very numerous

  • Do leading-edge Computer Science on problems of scale, cutting across traditional areas of architecture, operating systems, networks, and languages to enable a wide range of explorations in ubiquitous computing, both embedded in the environment or carried easily on moving objects and people

IRB/XIS


Current research team

Hans Mulder – co-director, IA64

Kevin Fall: UCSD, ISI, UCB, NetBoost, Intel

high speed ip networking

Alan Mainwaring: TMC, UCB, Sun, Intel

virtual networks, deep scalable network systems

Anind Dey: Georgia Tech, aware house

framework for context aware applns, ubicom

David Gay: UCB

Prog. Lang. design/Imp for novel comm. layers

Wei Hong, UCB, Illustra, Cohera, PeopleSoft

Federated databases

Su Ping: Intel

Software Engineering, embedded systems

Eric Paulos: UCB

HCI, robotics, ubicomp

Timothy Roscoe: Cambridge, Sprint

Operating systems, Distributed Computing, Infrastructure Services

Brent Chun: UCB, CIT

cluster systems, resource management

Matt Welsh, UCB (Post Doc)

Operating Systems, internet service design

Phil Buonodonna, UCB (abd intern)

Storage Area Networks, networks

Silvia Ratnasamy, UCB/ICSI (abd)

Networking, P2P

Justin Tomilson, Part Time

optimization, IEOR PhD Student

Earl Hines – operations mgr

Current Research Team

IRB/XIS


Additional researchers
Additional Researchers

  • Joe Hellerstein, Faculty Consultant (next AD)

    • streaming database, sensor database, P2P

  • Eric Brewer, Faculty Consultant

    • systems, language design

  • Larry Peterson, Consultant/Sabattical

  • Deborah Estrin, Faculty consultant

    • internet, multicast, rsvp,...sensor nets

  • Paul Wright, Former Faculty consultant

    • infopad, BWRC, cybercut

IRB/XIS


Current faculty research associates
Current Faculty Research Associates

  • James Demmel large-scale comp. sci

  • Michael Franklin Sensor Databases

  • Steven Glaser structural dynamics

  • Joe Hellerstein Streaming Databases

  • John Kubiatowicz planetary storage

  • James Landay HCI

  • David A Patterson Architecture

  • Kris Pister MEMS, Smart Dust

  • Jan Rabaey Low power systems

  • Satish Rao Distr. Systems Theory

  • Ion Stoika Networking

  • Vivek Subramanian Disposable devices

  • David Wagner Security

  • Kathy Yelick Parallel Languages

  • Jennifer Mankoff HCI

  • Shankar Sastry Distributed Robotics

IRB/XIS


ad