Lecture 12: Cloud Computing-B

Lecture 12: Cloud Computing-B • Cloud storage • Cloud vs. cluster computing • RPC & RMI • Map/Reduce • Hadoop

Cloud Storage • So we can get computers… • …but they’re in unknown locations • Geographic location • IP address • How do I get my data to them? • Preload all machines • Cloud storage!

Cloud Storage • Goal is to have a server hold our files • Distribute them as-needed • NFS, CIFS, SMB, etc • FTP, SCP, etc • Custom software • Various GIS servers, etc

Cloud Storage • 1 Think it’s linked to iTunes, which requires card • 2 Every referral adds 500MB, up to 17GB total • 3 Nothing is free

Issues • Privacy • Can 3rd party people see your files? • Will cloud storage admins look? • Integrity • Are your files unchanged?

Cloud Computing • So now I have computers… • …AND I have data. • What’s the next step?

Cloud Computing • So now I have computers… • …AND I have data. • What’s the next step? • Are the computers aware of each other?

Cloud Computing • So now I have computers… • …AND I have data. • What’s the next step? • Are the computers aware of each other? No • Can we do that?

Cloud Computing • So now I have computers… • …AND I have data. • What’s the next step? • Are the computers aware of each other? No • Can we do that? Yes!

Cluster Computing • Sometimes, we need our computers to “be aware” of each other • E.g., neural networks • When they are, they are said to be “a cluster” • Typically 10+ machines (subjective) • Typically geographically close • High-speed network benefits

Remote Computing • This is easiest explained by example • What is Siri?

Remote Computing • This is easiest explained by example • What is Siri? • Software that hears audio • Translates it into “plain language” • Translates that into something it understands • Handles the “native tongue” request • Gives you the response

Remote Computing • Can it do that locally? • Yes! • “Set my alarm for 8AM” • “Remind me to teach class tomorrow” • “Play songs by Sirenia” • These are voice control and have a relatively rigid structure

Remote Computing • Can it do that locally? • Yes! • “Set my alarm for 8AM” • “Remind me to teach class tomorrow” • “Play songs by Sirenia” • These are voice control and have a relatively rigid structure • No! • “What is the meaning of life?” • “What is the opening day of the Dragons in 2013, and who do they play?” • These can be computationally complex • It may also not have the required data

Remote Computing • So, the idea is: • Get some input • Send it somewhere else (servers/cloud) • Do the heavy number-crunching there • Get back the result • Eases the burden on the local computer • Requires ability to send/receive data

Remote Computing • RPC • Remote procedure call • C/C++ • RMI • Remote method invocation • Java • These are what make the function call “transparent” (mostly) to the programmer

Map/Reduce • Now I have computers • I know how to talk to them • What should I be doing with them? • Parallel things! • Image processing, protein folding, etc • How should I divide up the work? • Generally, we follow “map and reduce” techniques

Map/Reduce • Map • Assign segments of the task to different processors • i.e., “map” a sub-task to a processor • Each of the sub-tasks can run simultaneously • Reduce • Collect the partial-results into a final result • i.e., “reduce” many answers to one

Map/Reduce • Hadoop • Google instantiation of this architecture • (You can also build your own) • Map/Reduce can be at the level of • Sending tasks to computers in a cloud • Sending tasks to cores in a CPU • Sending tasks to cores in a GPU

Lecture 12: Cloud Computing-B