- 206 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about '' - benjamin

Download Now**An Image/Link below is provided (as is) to download presentation**
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

Download Now

Presentation Transcript

CNN Technology for Brain-like Spatial-temporal Sensory Computing –Present and Future

ISCAS-2004 Plenary Lecture

Vancouver, May 2004

Tamás ROSKA

Pázmány University and the Hungarian Academy of Sciences

Budapest, Hungary

Acknowledgements

- Berkeley-Budapest-Seville (12 years) and Notre Dame-Harvard research groups
- Office of Naval Research (ONR)
- Future and Emerging Technologies Division of the EU R&D Directorate
- Human Frontiers of Science Program
- Hungarian National Research Fund
- Spanish National Research Council
- Pázmány University, Budapest
- Hungarian Academy of Sciences

Table of contents

- Scenario: new vistas of complexity in circuits, systems and computers
- A new framework for sensory-computing- activating circuits and systems: Cellular Wave Computers and Wave-Logic
- Various physical implementations: towards (topographic) visual microprocessors
- Bio-inspiration- sensor fusion and proactive systems – multichannel retina models and cross-modalities

Contents (cont.)

5. Wave-algorithms – a new kind of software with embedded sensors

6. Applications

– „CNN technology”

- Semantic embedding

7. CNN principles in nanotechnology

1 Scenario: new vistas of complexity

- New directions in micro- and nano- technologies – a must for cellular
- The sensory revolution and its impact on data – a move to image flows and multimodal sensor fusion
- Complexity: billion devices and interconnect problems – bio-inspired architectures
- Mind inspired and brain inspired computing
- Sensory understanding and inferencing with embedded spatial-temporal semantics-Events are patterns

2. Cellular Wave Computers and Wave-Logic

- Data,
- Instructions,
- Subroutines,
- Events, and
- Algorithms

are different!

Non-Boolean logic

A first departure from the all digital-logic computing paradigm

- Turing – von-Neumann framework:

data are bit streams, time is discrete,

elementary operations on bits, and

STORED PROGRAMMABLE

- Blum-Schub-Smale departure:

data are reals, the role of accuracy and problem parameter in comp.complexity

Newton machine, the role of algebra&nonlinearity

A drastic departure

- Data are multivariable flows, continuous in time and signal value on a finite time interval
- Events are patterns ,

in space, in space-time, or in multivariable synchronization

- Elementary instructions: the solution of a nonlinear wave equation plus local and global binary logic
- Architecture and algorithms => ?? Universality??

Brain-like: - analog signal array - several 2D strata of analog processors - mainlylocal and sparse global interconnections with variable delays - spatial-temporal active waves

- Mind-like:

- logical sequences

- algorithmic

- Hint: Left-brain – Right-brain
- Univesral Machine on Flows

How to form a generic spatial temporal machine?

- Take the simplest dynamical system, a cell
- Take the simplest spatial grid for placing the cells (2D sheets)
- Introduce the simplest spatial interactions between dynamic cells

CNN: Cellular Nonlinear Network;

archetype:Turing-morphogenesis

- Place the CNN dynamics into the simplest stored programmable computing machine

CNN Universal Machine on Flows

Introduction to CNN Dynamics

j

The Cellular Neural/nonlinear

Network(CNN) is:

- an analog processor array
- on a rectangular grid
- with space invariant
- local interactions.

i

B

Local interconnection pattern:cloning TEMPLATExij- state/ yij - output

z - bias

uij- input

z= -0.5

Template - the program of the network:[A Bz]

PDE formulation of reaction-diffusion processes

Reaction-diffusion type nonlinear PDE:

If the first diffusion term is the Laplacian, and there are only two variables in , we get the simple reaction diffusion equation of Turing - morphogenesis

Deriving coupled ODEs from PDEs

Reaction-diffusion type nonlinear ODE:

Templates (symmetric-isotropic class):

Computing with diffusion and waves derived from reaction-diffusion systems

Examples:

Linear diffusion

Trigger-waves

Pattern formation

A generalization of Turing’s morphogenesis and Von Neumann’s vision on analytic theory of computing

- Generating a plethora of different
- waves and
- patterns
- A framework of modeling chemical, physical, biological and abstract

complex systems

- Introducing algorithms on waves

Make a computing machine with CNN elementary instructions

- Extend each cell with
- A few local memory units (LAM, LLM)
- A local Communication and Control Unit
- And possibly local logic and arithmetic units
- Add a Global analogic Programming Unit (GAPU) with

- Analog and logic Registers and

- A machine code storage (GACU)

CNN Universal Machine (CNN-UM)

LCCU

L

L

M

CNN

nucleus

L

A

M

LAOU LLU

G A P U

LAM: Local Analog Memory

LLM: Local Logic Memory

LCCU:Local Communication and Control Unit

LAOU:Local Analog Output Unit

LLU: Local Logic Unit

GAPU: Global Analogic

Programming Unit

APR: Analog Instruction Register

LPR: Logic Program Register

SCR: Switch Configuration Register

GACU:Global Analogic Control Unit

[A1 B1 z1], [A2 B2 z2], . . .

<=Analogic (analog+logic) algorithm

Cellular Active Wave Computer on image Flows

Universal Machine on Flows (UMF)

Data: Image Flow

(t):i j (t) , t T= 0, td

1 i m 1 j n

at t = t, (t) is an m x n Picture P

if P is binary it is a Mask M

if t = t0, t0+ t, t0+ 2t, …… t0+ k t then:image sequence or video stream.

Operators on Image Flows

- The protagonist elementary instruction, also called wave instruction, is defined as

output (t):= (input(t), Po, ); t T=0, tdwhere

: an array function on image flows or image sequences

Po: a picture defining initial state (0) and/or bias map

: boundary conditions (a frame), (t) is a boundary input might also be connected to all cells in a row

- A scalar functional on an image flow:

q: = ( input(t), Po, );

Algorithms on image flows

- α-recursive function.

·initial settings of image flows, pictures, masks, and boundary values: (0), Po, M, ;

·equilibrium and non-equilibriumsolutions of partial differential difference equations (PDDE) via canonical CNN equations on (t) ;

· global (and local) minimization on the above;

·memoryless (arithmetic) and logic combina-tions on the above results,

·comparisons (thresholding) and logic condi-tions in branchings, via scalarglobal functionals

·recursions on the above operations

Properties of the Universal Machine on Flows

- Universality in Turing sense and as a spatial-temporal nonlinear operator
- Active waves in the region of edge of chaos within the locally active regime
- Stored programmability is the key for practical applications
- Combining analog spatial-temporal waves with local and global logic– analog-and-logic
- Native operators for Programmable Sensor-Computing and in Nanotechnology

3. Physical implementation: towards visual microprocessors

- Mixed-signal CMOS
- Mixed signal BiCMOS
- Emulated digital CMOS
- Optical
- FPGA
- Integration of topographic sensors
- SoC
- Software and development systems
- Self contained units e.g. Bi-i

1.

32.

32.

1.

Comparison between an IBM Cellular Supercomputer and an analogic processor128 x 128 processor with optical input

65536 (32*32*64) Power PC

An analog-and-logicic CNN

supercomputer

IBM Cellular

Supercomputer 2002

Computing Power ~ 12 * 1012(TeraFLOPS)

Computing Power ~ 12 * 1012 (TeraOPS)equivalent

A = 65536 x 1.06 cm2 = 6.9468 m2

P = 491 kW

A = 1.4 cm2P = 4.5 W

CNN technology roadmap

ACE16k

Complexity/resolution

XENON*

ACE4k

ACE400

20x22,

bin I/O,

optical input

50 000 fr/sec

128x96,

gray I/O,

optical input

10 000 fr/sec

embedded Digital

Microprocessor

A-D cells

128x128,

gray I/O,

optical input

50 000 fr/sec

64x64, gray I/O,

par. optical input,

1000 fr/sec

time

1995-96 1998-99 2003 2004

The ACE CNN chip family has been designed at IMSE-CNM and AnaFocus Ltd.,Seville Spain.

The XENON chip was designed by ANCLab at the Hungarian Academy of Sciences

and AnaLogic Computers Ltd., Budapest, Hungary *under fabrication

First prize and Product of the year at Vision 2003, Stuttgart

Bi-i: a standalone visual system

- Standalone
- Compact
- Embedded 128x128 ACE16k* chip (1 or 2)
- Above 5000 Fps
- Embedded 250MHz DSP
- Embedded 1.3M CMOS imager with ROI
- Ethernet 100MBit/s
- USB

*ACE16k chip was designed at IMSE-CNM Seville Spain

analogmemory

base cell

globális control signals

analog I/O bus

analog and logical operations

opt.input

logmem

logicalcircuitry

analog synapses

digital I/O bus

CNN Universal Machine (CNN-UM) Chip

Standard image processing functions

Separation of the stationary and moving parts

Original image Moving parts Stationary parts

High dynamic range(integration time is indicated)

24ms 200ms 1.6ms 6.4ms

12.8ms 25.6ms 51.2ms 102.4ms

High Frame rate AND processing

1,000 to 50, 000 frames per second

and processing during the

20-1000 microsec window

High speed diffusion

5 grids in 100 nsec

4 Biological Relevance

- Retinotopic Visual Pathway
- New discovery in 2001 (Nature):

Multichannel Mammalian Retina Model

- Multilayer CNN dynamics with programmable space and time constants
- Towards a programmable vision prosthesis

– the first 5 human retinal implants (at USC)

- Multichannel tactile (haptic) model
- Non-synaptic neural signal transmission
- Immune response inspired algorithms

Ganglion cell types

- Off Brisk L
- Off Brisk Tr
- On Brisk TrN
- On Brisk Tr
- On Beta
- On Sluggish
- Bistratified
- Local Edge Detector

IPL Strata

Botond Roska and FrankS.Werblin Nature, March 29, 2001

Parallel space-time features

b1

b2

t1

A11

b1

b2

t10

t1

t10

A22

t2

A11

A11

b1

b1

b2

b2

A22

t1

t1

t2

OPL

t10

t10

A11

b1

b2

t1

t10

A22

A22

t2

t2

A22

t2

IRE-A

IRE-D

IRI

GRB

Complex R-Unit decompositionGeneral block

- input flow sampling
- output sample/hold
- spike generation

Model Simulations

Stimulus and 3 different output together

Local Edge Detector

Off Brisk TrL cell

Spiking

transfer to the brain

Excitation

Inhibition

3-Layer Prototype R-unit

- mutually coupled 1st order „RC” cells, space constants
- double time-scale property
- separate inputs and initial states

Inputs

A11

b1

b2

t1

A22

t2

Output

5. Wave-algorithms– a new kind of software with embedded sensors

- Are programmed complex nonlinear ~waves implementable on Silicon?
- Are ~spatial-temporal signatures significant in coding some shapes?
- Can we combine these waves algorithmically embedding in

~proactive adaptive systems?

Five bio-inspired algorithmic wave computing principles:

- ·thetwin waveprinciple
- ·thepush-pullprinciple
- ·the multi-channel(e.g.color)opponentprinciple (center channel – surround channel)
- ·the programmable first action(proactive) principle
- ·the detection by emerging dynamicsprinciple

Twin wave principle illustration

All sad

Inhibition wave

(large lateral inhibition)

Excitation wave

(small lateral excitation)

Contours

Concave curves

from the bottom

(sad mouth)

No,

He is laughing!

Combination

Example: Contour Detection of a Spiral-wave

Original image

CNN input

( noisy, compressed original image )

CNN initial state

(a patch in the spiral-wave region)

Strategies to Control a Trigger-wave - I.

Case 1: pixel-wise adaptation

Strategies to Control a Trigger-wave - II.

Case 2: adaptation through an optimal reconstruction filter + bias control

Tracking Experiments inEchocardiography

Motivation:

Feature extraction from echocardiographic images

Major importance for both quantitative and qualitative analysis of the heart function

Experiment:

Endocardial (inner) contour de-tection of the left ventricle from a sequence of echocardio-graphic images

LV

RV

LV - left ventricle

LA - left atrium

RV - right ventricle

RA - right atrium

RA

LA

Apical four-chamber view of the human heart

Active contour tracking based on trigger-waves

(k-1)-th result

(k-1)-th frame

k-th frame

CNN-UM

chip results

(ACE4K):

PDE formalism:

~30 sec/fr

~60 sec/fr

~160 sec/fr

~ 250 sec/fr

k-th result

6 Applications

- Very high frame rate real-time detection : ~ 10-50 k frame per second
- Proactive, adaptive, topographic sensory computing – with locally tuned sensors
- Very high computing power for complex, wavetype algorithms: Tera OPS
- Very high number of targets and pattern matching templates – immune response inspired CNN algorithms
- Embedded semantics

- handwriting (geometrical features)

- multimodal (vision and tactile)

Virtual action closing a hole I

StarflexR septal occluder 3D modellje

Virtual action closing a hole II

AmplatzerR septal occluder 3D modellje

Virtual action closing a hole III.

Interventional closing

Virtualis ASD zárás AmplatzerR septal occluderrel

Virtual action closing a hole IV.

Interventional Closing in 3D

Virtualis ASD zárás AmplatzerR septal occluderrel

7 Towards Nanotechnology

- Starting from nano-friendly device modesl
- Evolutionary and revolutionary Nanotechnology (>100 nm)
- Integrating sensing and computing
- CNN and Crossnet /CMOL technologies

Integrated sensing - computing Cellular Nano Architecture

Cellular Wave Computer Chip

with 1000x1000 processors

nano antenna

multiple sensor array

Projected capability:

10 PetaOps speed

100,000 frame/sec

- Enabling technology for
- Ultra high speed multiple target detection
- Fusion reactor control
- Intelligent surveillance

Processing via Unconventional Nano –friendly devices and Nano Systems

Start from Nano-friendly devices:

easy to implement and interconnect => CNN

Function-in-layout or non-transistor-based

Analog signals and logic (e.g. CMOL/BiCWAS)

I/O via radiation

- add sensing arrays and optical tranceivers

Lithographically-Defined Nanoantennas

Dipole antenna with MOM diode, which functions at THz frequencies

Bowtie antenna with MOM diode, which operates in the visible

C. Fumeaux, J. Alda, and G. D. Boreman, “Lithographic Antennas at Visible Frequencies,” Optics Lett.24, 1629-1631 (1999).

I. Wilke, W. Herrmann, F. K. Kneubuhl, “Integrated Nanostrip Dipole Antennas for Coherent 30 THz Infrared Radiation,” Appl. Phys. B58(2), pp. 87-95 (1994).

100 nm

Ni

Maybe

200 nm

NiO

35 Å

Ni

Ni

200 nm

Antenna

1.5 μm

SiO2

Si

1.5 μm

SiO2

Detector Layout for 30 THz

a) small enough ?

YES

b) sensitive enough ?

c) fast enough ?

YES

d) sufficient spectral purity ?

YES

e) dynamically controllable ?

Limited yes (Bias)

References

- L.O.Chua and T. Roska, Cellular Neural Networks and Visual Computing, Cambridge Univesrity Press, Cambridge, 2002
- T.Roska and Á.Rodríguez Vázquez, „Towards Visual Microprocessors”, Proc. IEEE, July, 2002
- http://lab.analogic.sztaki.hu: Bibliography
- Special Issues:
- IEEE Trans. CAS-I, May 2004
- Int.J. Bifurcation and Chaos, February, 2004
- J. Circuits, Systems and Computers, Nos.4 and 6, 2003
- Int.J.Circuit Theory and Applications, Nos. 1 and 2, 2002

Download Presentation

Connecting to Server..