cnn technology for brain like spatial temporal sensory computing present and future l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
CNN Technology for Brain-like Spatial-temporal Sensory Computing –Present and Future PowerPoint Presentation
Download Presentation
CNN Technology for Brain-like Spatial-temporal Sensory Computing –Present and Future

Loading in 2 Seconds...

play fullscreen
1 / 66

CNN Technology for Brain-like Spatial-temporal Sensory Computing –Present and Future - PowerPoint PPT Presentation


  • 224 Views
  • Uploaded on

CNN Technology for Brain-like Spatial-temporal Sensory Computing –Present and Future. ISCAS-2004 Plenary Lecture Vancouver, May 2004 Tamás ROSKA Pázmány University and the Hungarian Academy of Sciences Budapest, Hungary. Acknowledgements.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'CNN Technology for Brain-like Spatial-temporal Sensory Computing –Present and Future' - benjamin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
cnn technology for brain like spatial temporal sensory computing present and future
CNN Technology for Brain-like Spatial-temporal Sensory Computing –Present and Future

ISCAS-2004 Plenary Lecture

Vancouver, May 2004

Tamás ROSKA

Pázmány University and the Hungarian Academy of Sciences

Budapest, Hungary

acknowledgements
Acknowledgements
  • Berkeley-Budapest-Seville (12 years) and Notre Dame-Harvard research groups
  • Office of Naval Research (ONR)
  • Future and Emerging Technologies Division of the EU R&D Directorate
  • Human Frontiers of Science Program
  • Hungarian National Research Fund
  • Spanish National Research Council
  • Pázmány University, Budapest
  • Hungarian Academy of Sciences
table of contents
Table of contents
  • Scenario: new vistas of complexity in circuits, systems and computers
  • A new framework for sensory-computing- activating circuits and systems: Cellular Wave Computers and Wave-Logic
  • Various physical implementations: towards (topographic) visual microprocessors
  • Bio-inspiration- sensor fusion and proactive systems – multichannel retina models and cross-modalities
contents cont
Contents (cont.)

5. Wave-algorithms – a new kind of software with embedded sensors

6. Applications

– „CNN technology”

- Semantic embedding

7. CNN principles in nanotechnology

1 scenario new vistas of complexity
1 Scenario: new vistas of complexity
  • New directions in micro- and nano- technologies – a must for cellular
  • The sensory revolution and its impact on data – a move to image flows and multimodal sensor fusion
  • Complexity: billion devices and interconnect problems – bio-inspired architectures
  • Mind inspired and brain inspired computing
  • Sensory understanding and inferencing with embedded spatial-temporal semantics-Events are patterns
2 cellular wave computers and wave logic
2. Cellular Wave Computers and Wave-Logic
  • Data,
  • Instructions,
  • Subroutines,
  • Events, and
  • Algorithms

are different!

Non-Boolean logic

a first departure from the all digital logic computing paradigm
A first departure from the all digital-logic computing paradigm
  • Turing – von-Neumann framework:

data are bit streams, time is discrete,

elementary operations on bits, and

STORED PROGRAMMABLE

  • Blum-Schub-Smale departure:

data are reals, the role of accuracy and problem parameter in comp.complexity

Newton machine, the role of algebra&nonlinearity

a drastic departure
A drastic departure
  • Data are multivariable flows, continuous in time and signal value on a finite time interval
  • Events are patterns ,

in space, in space-time, or in multivariable synchronization

  • Elementary instructions: the solution of a nonlinear wave equation plus local and global binary logic
  • Architecture and algorithms => ?? Universality??
slide10

Brain-like: - analog signal array - several 2D strata of analog processors - mainlylocal and sparse global interconnections with variable delays - spatial-temporal active waves

  • Mind-like:

- logical sequences

- algorithmic

  • Hint: Left-brain – Right-brain
  • Univesral Machine on Flows
how to form a generic spatial temporal machine
How to form a generic spatial temporal machine?
  • Take the simplest dynamical system, a cell
  • Take the simplest spatial grid for placing the cells (2D sheets)
  • Introduce the simplest spatial interactions between dynamic cells

 CNN: Cellular Nonlinear Network;

archetype:Turing-morphogenesis

  • Place the CNN dynamics into the simplest stored programmable computing machine

 CNN Universal Machine on Flows

slide13

1

-1

1

-1

Cell Dynamics

introduction to cnn dynamics
Introduction to CNN Dynamics

j

The Cellular Neural/nonlinear

Network(CNN) is:

  • an analog processor array
  • on a rectangular grid
  • with space invariant
  • local interactions.

i

local interconnection pattern cloning template

A

B

Local interconnection pattern:cloning TEMPLATE

xij- state/ yij - output

z - bias

uij- input

z= -0.5

Template - the program of the network:[A Bz]

pde formulation of reaction diffusion processes
PDE formulation of reaction-diffusion processes

Reaction-diffusion type nonlinear PDE:

If the first diffusion term is the Laplacian, and there are only two variables in , we get the simple reaction diffusion equation of Turing - morphogenesis

deriving coupled odes from pdes
Deriving coupled ODEs from PDEs

Reaction-diffusion type nonlinear ODE:

Templates (symmetric-isotropic class):

computing with diffusion and waves derived from reaction diffusion systems
Computing with diffusion and waves derived from reaction-diffusion systems

Examples:

Linear diffusion

Trigger-waves

Pattern formation

a generalization of turing s morphogenesis and von neumann s vision on analytic theory of computing
A generalization of Turing’s morphogenesis and Von Neumann’s vision on analytic theory of computing
  • Generating a plethora of different
    • waves and
    • patterns
  • A framework of modeling chemical, physical, biological and abstract

complex systems

  • Introducing algorithms on waves
make a computing machine with cnn elementary instructions
Make a computing machine with CNN elementary instructions
  • Extend each cell with
    • A few local memory units (LAM, LLM)
    • A local Communication and Control Unit
    • And possibly local logic and arithmetic units
    • Add a Global analogic Programming Unit (GAPU) with

- Analog and logic Registers and

- A machine code storage (GACU)

cnn universal machine cnn um
CNN Universal Machine (CNN-UM)

LCCU

L

L

M

CNN

nucleus

L

A

M

LAOU LLU

G A P U

LAM: Local Analog Memory

LLM: Local Logic Memory

LCCU:Local Communication and Control Unit

LAOU:Local Analog Output Unit

LLU: Local Logic Unit

GAPU: Global Analogic

Programming Unit

APR: Analog Instruction Register

LPR: Logic Program Register

SCR: Switch Configuration Register

GACU:Global Analogic Control Unit

[A1 B1 z1], [A2 B2 z2], . . .

<=Analogic (analog+logic) algorithm

cellular active wave computer on image flows
Cellular Active Wave Computer on image Flows

Universal Machine on Flows (UMF)

Data: Image Flow

 (t):i j (t) , t  T= 0, td

1  i  m 1  j  n

at t = t,  (t) is an m x n Picture P

if P is binary it is a Mask M

if t = t0, t0+ t, t0+ 2t, …… t0+ k t then:image sequence or video stream.

operators on image flows
Operators on Image Flows
  • The protagonist elementary instruction, also called wave instruction, is defined as

output (t):=  (input(t), Po, ); t  T=0, tdwhere

: an array function on image flows or image sequences

Po: a picture defining initial state (0) and/or bias map

: boundary conditions (a frame),  (t) is a boundary input might also be connected to all cells in a row

  • A scalar functional on an image flow:

q: =  ( input(t), Po, );

algorithms on image flows
Algorithms on image flows
  • α-recursive function.

·initial settings of image flows, pictures, masks, and boundary values:  (0), Po, M, ;

·equilibrium and non-equilibriumsolutions of partial differential difference equations (PDDE) via canonical CNN equations on (t) ;

· global (and local) minimization on the above;

·memoryless (arithmetic) and logic combina-tions on the above results,

·comparisons (thresholding) and logic condi-tions in branchings, via scalarglobal functionals

·recursions on the above operations

properties of the universal machine on flows
Properties of the Universal Machine on Flows
  • Universality in Turing sense and as a spatial-temporal nonlinear operator
  • Active waves in the region of edge of chaos within the locally active regime
  • Stored programmability is the key for practical applications
  • Combining analog spatial-temporal waves with local and global logic– analog-and-logic
  • Native operators for Programmable Sensor-Computing and in Nanotechnology
3 physical implementation towards visual microprocessors
3. Physical implementation: towards visual microprocessors
  • Mixed-signal CMOS
  • Mixed signal BiCMOS
  • Emulated digital CMOS
  • Optical
  • FPGA
  • Integration of topographic sensors
  • SoC
  • Software and development systems
  • Self contained units e.g. Bi-i
comparison between an ibm cellular supercomputer and an analogic processor

64.

1.

32.

32.

1.

Comparison between an IBM Cellular Supercomputer and an analogic processor

128 x 128 processor with optical input

65536 (32*32*64) Power PC

An analog-and-logicic CNN

supercomputer

IBM Cellular

Supercomputer 2002

Computing Power ~ 12 * 1012(TeraFLOPS)

Computing Power ~ 12 * 1012 (TeraOPS)equivalent

A = 65536 x 1.06 cm2 = 6.9468 m2

P = 491 kW

A = 1.4 cm2P = 4.5 W

cnn technology roadmap
CNN technology roadmap

ACE16k

Complexity/resolution

XENON*

ACE4k

ACE400

20x22,

bin I/O,

optical input

50 000 fr/sec

128x96,

gray I/O,

optical input

10 000 fr/sec

embedded Digital

Microprocessor

A-D cells

128x128,

gray I/O,

optical input

50 000 fr/sec

64x64, gray I/O,

par. optical input,

1000 fr/sec

time

1995-96 1998-99 2003 2004

The ACE CNN chip family has been designed at IMSE-CNM and AnaFocus Ltd.,Seville Spain.

The XENON chip was designed by ANCLab at the Hungarian Academy of Sciences

and AnaLogic Computers Ltd., Budapest, Hungary *under fabrication

slide35

First prize and Product of the year at Vision 2003, Stuttgart

Bi-i: a standalone visual system

  • Standalone
  • Compact
  • Embedded 128x128 ACE16k* chip (1 or 2)
  • Above 5000 Fps
  • Embedded 250MHz DSP
  • Embedded 1.3M CMOS imager with ROI
  • Ethernet 100MBit/s
  • USB

*ACE16k chip was designed at IMSE-CNM Seville Spain

slide36

analog synapses

analogmemory

base cell

globális control signals

analog I/O bus

analog and logical operations

opt.input

logmem

logicalcircuitry

analog synapses

digital I/O bus

CNN Universal Machine (CNN-UM) Chip

slide38

Standard image processing functions

Separation of the stationary and moving parts

Original image Moving parts Stationary parts

high dynamic range integration time is indicated
High dynamic range(integration time is indicated)

24ms 200ms 1.6ms 6.4ms

12.8ms 25.6ms 51.2ms 102.4ms

high frame rate and processing
High Frame rate AND processing

1,000 to 50, 000 frames per second

and processing during the

20-1000 microsec window

High speed diffusion

5 grids in 100 nsec

4 biological relevance
4 Biological Relevance
  • Retinotopic Visual Pathway
  • New discovery in 2001 (Nature):

Multichannel Mammalian Retina Model

  • Multilayer CNN dynamics with programmable space and time constants
  • Towards a programmable vision prosthesis

– the first 5 human retinal implants (at USC)

  • Multichannel tactile (haptic) model
  • Non-synaptic neural signal transmission
  • Immune response inspired algorithms
retina structure
Retina structure

Cones

OPL

IPL

Ganglions

ganglion cell types
Ganglion cell types
  • Off Brisk L
  • Off Brisk Tr
  • On Brisk TrN
  • On Brisk Tr
  • On Beta
  • On Sluggish
  • Bistratified
  • Local Edge Detector

IPL Strata

Botond Roska and FrankS.Werblin Nature, March 29, 2001

Parallel space-time features

measurement
Measurement

Excitation

x

t

Inhibition

x

t

Spiking

x

On Beta ganglion cell

t

complex r unit decomposition

A11

b1

b2

t1

A11

b1

b2

t10

t1

t10

A22

t2

A11

A11

b1

b1

b2

b2

A22

t1

t1

t2

OPL

t10

t10

A11

b1

b2

t1

t10

A22

A22

t2

t2

A22

t2

IRE-A

IRE-D

IRI

GRB

Complex R-Unit decomposition

General block

  • input flow sampling
  • output sample/hold
  • spike generation
model simulations
Model Simulations

Stimulus and 3 different output together

Local Edge Detector

Off Brisk TrL cell

Spiking

transfer to the brain

Excitation

Inhibition

3 layer prototype r unit
3-Layer Prototype R-unit
  • mutually coupled 1st order „RC” cells, space constants
  • double time-scale property
  • separate inputs and initial states

Inputs

A11

b1

b2

t1

A22

t2

Output

5 wave algorithms a new kind of software with embedded sensors
5. Wave-algorithms– a new kind of software with embedded sensors
  • Are programmed complex nonlinear ~waves implementable on Silicon?
  • Are ~spatial-temporal signatures significant in coding some shapes?
  • Can we combine these waves algorithmically embedding in

~proactive adaptive systems?

five bio inspired algorithmic wave computing principles
Five bio-inspired algorithmic wave computing principles:
  • ·thetwin waveprinciple
  • ·thepush-pullprinciple
  • ·the multi-channel(e.g.color)opponentprinciple (center channel – surround channel)
  • ·the programmable first action(proactive) principle
  • ·the detection by emerging dynamicsprinciple
twin wave principle illustration
Twin wave principle illustration

All sad

Inhibition wave

(large lateral inhibition)

Excitation wave

(small lateral excitation)

Contours

Concave curves

from the bottom

(sad mouth)

No,

He is laughing!

Combination

example contour detection of a spiral wave
Example: Contour Detection of a Spiral-wave

Original image

CNN input

( noisy, compressed original image )

CNN initial state

(a patch in the spiral-wave region)

strategies to control a trigger wave i
Strategies to Control a Trigger-wave - I.

Case 1: pixel-wise adaptation

strategies to control a trigger wave ii
Strategies to Control a Trigger-wave - II.

Case 2: adaptation through an optimal reconstruction filter + bias control

tracking experiments in echocardiography
Tracking Experiments inEchocardiography

Motivation:

Feature extraction from echocardiographic images

Major importance for both quantitative and qualitative analysis of the heart function

Experiment:

Endocardial (inner) contour de-tection of the left ventricle from a sequence of echocardio-graphic images

LV

RV

LV - left ventricle

LA - left atrium

RV - right ventricle

RA - right atrium

RA

LA

Apical four-chamber view of the human heart

active contour tracking based on trigger waves
Active contour tracking based on trigger-waves

(k-1)-th result

(k-1)-th frame

k-th frame

CNN-UM

chip results

(ACE4K):

PDE formalism:

~30 sec/fr

~60 sec/fr

~160 sec/fr

 ~ 250 sec/fr

k-th result

6 applications
6 Applications
  • Very high frame rate real-time detection : ~ 10-50 k frame per second
  • Proactive, adaptive, topographic sensory computing – with locally tuned sensors
  • Very high computing power for complex, wavetype algorithms: Tera OPS
  • Very high number of targets and pattern matching templates – immune response inspired CNN algorithms
  • Embedded semantics

- handwriting (geometrical features)

- multimodal (vision and tactile)

virtual action closing a hole i
Virtual action closing a hole I

StarflexR septal occluder 3D modellje

virtual action closing a hole ii
Virtual action closing a hole II

AmplatzerR septal occluder 3D modellje

virtual action closing a hole iii
Virtual action closing a hole III.

Interventional closing

Virtualis ASD zárás AmplatzerR septal occluderrel

virtual action closing a hole iv
Virtual action closing a hole IV.

Interventional Closing in 3D

Virtualis ASD zárás AmplatzerR septal occluderrel

7 towards nanotechnology
7 Towards Nanotechnology
  • Starting from nano-friendly device modesl
  • Evolutionary and revolutionary Nanotechnology (>100 nm)
  • Integrating sensing and computing
  • CNN and Crossnet /CMOL technologies
slide62

Integrated sensing - computing Cellular Nano Architecture

Cellular Wave Computer Chip

with 1000x1000 processors

nano antenna

multiple sensor array

Projected capability:

10 PetaOps speed

100,000 frame/sec

  • Enabling technology for
  • Ultra high speed multiple target detection
  • Fusion reactor control
  • Intelligent surveillance
slide63

Processing via Unconventional Nano –friendly devices and Nano Systems

Start from Nano-friendly devices:

easy to implement and interconnect => CNN

Function-in-layout or non-transistor-based

Analog signals and logic (e.g. CMOL/BiCWAS)

I/O via radiation

- add sensing arrays and optical tranceivers

lithographically defined nanoantennas
Lithographically-Defined Nanoantennas

Dipole antenna with MOM diode, which functions at THz frequencies

Bowtie antenna with MOM diode, which operates in the visible

C. Fumeaux, J. Alda, and G. D. Boreman, “Lithographic Antennas at Visible Frequencies,” Optics Lett.24, 1629-1631 (1999).

I. Wilke, W. Herrmann, F. K. Kneubuhl, “Integrated Nanostrip Dipole Antennas for Coherent 30 THz Infrared Radiation,” Appl. Phys. B58(2), pp. 87-95 (1994).

slide65

1.5 μm

100 nm

Ni

Maybe

200 nm

NiO

35 Å

Ni

Ni

200 nm

Antenna

1.5 μm

SiO2

Si

1.5 μm

SiO2

Detector Layout for 30 THz

a) small enough ?

YES

b) sensitive enough ?

c) fast enough ?

YES

d) sufficient spectral purity ?

YES

e) dynamically controllable ?

Limited yes (Bias)

references
References
  • L.O.Chua and T. Roska, Cellular Neural Networks and Visual Computing, Cambridge Univesrity Press, Cambridge, 2002
  • T.Roska and Á.Rodríguez Vázquez, „Towards Visual Microprocessors”, Proc. IEEE, July, 2002
  • http://lab.analogic.sztaki.hu: Bibliography
  • Special Issues:
    • IEEE Trans. CAS-I, May 2004
    • Int.J. Bifurcation and Chaos, February, 2004
    • J. Circuits, Systems and Computers, Nos.4 and 6, 2003
    • Int.J.Circuit Theory and Applications, Nos. 1 and 2, 2002