PGP Project

1. PGP Project Viktor Yarmolenko Lewis Mackenzie Paul Cockshott Ewan Borland

2. Contents • Background • Motivation • Proposed Solution • Status • Questions

3. Background Setup

4. Background Algorithm View from 3 cameras

5. Background Algorithm X warp Y warp Correlation

6. Background Algorithm Reconstructed model viewed from a Virtual perspective position.

7. Background The problem Highly parallelizable problem – the following can be done independently: • 8 Matching jobs per group • N process groups per sequence • ~1min on Matching • ~1min on Building (2GHz)

8. Motivation Dynamic data graphs S M 1 S M 1 T

9. Motivation Dynamic data graphs S M 1 S M B 1 1 T

10. Motivation Dynamic data graphs S M 2 M M 2 1 S M B 1 1 T

11. Motivation Dynamic data graphs S M 2 M 2 S B 1 T

12. Motivation The search for silver bullet Our Aim: • To develop a highly parallel system for 3D computer vision algorithms, • Which also can be a general framework for distributed processing. Primary Requirements: • Allows dynamic creation of processes (arbitrary code) • Allows creation of communications channels between processes. • Allows channels to be dynamically reconfigurable.

13. Motivation The search for silver bullet GRID+Dynamic+Flexible+Java+...= • Use of GRID as a transport layer sounds good • The requirements could not be readily met by existing GRID protocols • No current API (MPI, RMI, PVM, DSM) can readily meet the requirements • We need a conceptually new parallel architecture J p =

14. Solution What is J? J is a Java interface loosely modelled on the primitives of the -calculus (Milner) to be used as a substratum for GRID based parallel computing. The key concepts taken from Milner’s calculus are the ability to dynamically create processes and communication channels and to transmit communication channels along other channels. (Milner) • "The Polyadic -calculus, a tutorial“, Milner, R., (1991) • “A calculus of mobile processes”, Milner, R., Parrow, J., Walker, D., Information and Computation, 100:1-77, (1992)

15. HDD Solution J primitives JPieTask – implements Runnable JPieFunnel – extends OutputStream JPieTap – extends InputStream JPiePipe – contains connected JPieTap and JPieFunnel 1000km

16. 2 1 1 Solution J example HDD VM VM VM

17. ILS model Initiator Locator Servent Locator – currently at EPCC Runs web services Maintains MySQL database of servents Reply with Available servents Register With locator Servent runs JPie daemon Query locator Initiator which Starts job Any servent can also become an initiator during the course of a computation and spawn more tasks

18. ILS model Initiator Locator Servent Locator keeps track of available memory and performance of servents Reply with list of suitalble servents Register performance and memory Servent Queries locator For machines of Above performance Level X Java Grande Benchmark used To rate Servent performance Initiator Starts job

19. Status Need Your Thoughts • Performed network tests using the data demanding part of the algorithm. Data transfer is not a bottleneck in this problem. • Completed the multi processor implementation of J, using sockets. • Currently installing resource locator using Web services to find free cpus. • Would like to implement J transport using a GRID layer. Acknowledgements: NeSC – Sponsors of the project EPCC – Discussions

20. Examples Conformed sequence

21. Examples Conformed mesh

22. Examples Landmark grid

23. Examples Textured

24. Examples Textured + mesh

25. 2 1 1 Stream down the stream in pictures HDD VM VM VM

26. Host 1 Thread 1 HDD Thread 2 Client 1 Host 2 Thread 3 Host 3 Thread n Host 4 Host 5 Thread 1 HDD Thread 2 Host 6 Client 4 Thread 3 Host 7 Thread n Host N Preliminary Tests • A part of algorithm was used • Total data transfer is over 5GB • Processed 12 sec of 3D video • That is 3600 images (122543) • The bandwidth at it’s bottleneck 100Mbits • Virtually theoretical speedup • Can be improved by using J