Virtualization for developers

1. Virtualization for developers The benefit of a standardized virtual stack for web development and the enterprise. LinuxFestNorthWest April 27, 2013. Presented to Vancouver Linux Users Group (VANLUG), January 21 2013.

2. Table of contents. • Intro • Virtualization • Virtual Private Servers • Virtualization for Devs • Prehistory • What it solves • An aquarium as an analogy • Stupid business practices • The future • Q&A

3. IntroTl;dr; virtualization for web development is really smart, but not always used. Here is why you use it. • Look at some of the technical aspects. • The social benefits are even greater than the technical. • Some of you may already know about virtualization • Consider the greater picture as I unpack all this for a mixed audience. • So you can sell others on it. Let’s not just preach to the converted at LFNW 2013.

4. About moi • About 12 years web dev. • The old Vancouver PHP Users Association. • Organized Several Conferences with team. • I don’t work for Oracle or other vendor. • Devand Project Management work. • In my spare time: Blogger, biker, train in Brazilian Jiu-jitsu.

5. A little credit to: David Turner http://www.turneris.com/ Who knew how to do a lot of this already… for helping me sort out some issues. My turn to buy beer next time…

6. What’s in it for me? How can I profit from this? • Standardize team skills. • IT policy with a high degree of similarity between development and production environments, which means fewer problems down the road. • Fast & flexible team creation. • Skills & tools refactored.

7. Virtualization:The absolute basics • Virtualization is the means to run an operating system within a host operating system. • A form of an emulator. • Emulators have always been around. • Very useful for web development, although its not what we usually think of when we are talking about virtualizations.

8. “Hypervisor” • In computing, a hypervisor or virtual machine manager (VMM) is a piece of computer software, firmware or hardware that creates and runs virtual machines. Source: http://en.wikipedia.org/wiki/Hypervisor • For Virtual Private Servers. • For Desktop environments.

9. Virtual Private Servers -for web hosting solutions • Something between the palace of dedicated hosting and the sewer of shared hosting. • The performance of a dedicated server on the budget of a shared host. • Good hosting companies provide VPS service. IMHO, bad ones don’t.

10. VPS vs Shared host page generation performance on an index.php page • Dreamhost.com shared host: ~2.0 seconds. • VPS: Canadian Web Hosting co-located on Peer1: 0.00002 seconds. • Or an improvement of 2*10-5 over shared hosting. • Holy crap.

11. VPS HostingLooks Like this: http://esx.starwindsoftware.com/images/content/VMware-iSCSI-SAN-diagram.gif

12. FYI: key vendors Aquemu for Linux

13. OK, so what? • This is a talk for a Linux Users Group. • Hypervisors run the LAMP stack. • The LAMP stack runs your website. • You can run a mirror of your web server locally with many advantages. • The codebase is normalized across the team with version control. • The whole stack can be normalized with virtualization. • I realize that some people may already get this, or do it. • But some don’t.

14. VMs on the Desktop (for Development)with it, I can do this:

15. Mkay… • Use virtualisation as a working tool to build websites and things like it to have performance and behavior tested to a high degree of resolution. • You are able to run the codebase on a real stack, and even possible a mirror of your production stack. • Way gooder. • Even deploy to Amazon EC2.

16. The prehistory – what sucked. • Manual install of Apache MySQL & PHP on win & mac desktop • XAMPP (http://www.apachefriends.org/) • MAMP (for Mac) • WAMP • CYGWIN

17. Dinosaurs died out for a reason

18. Devprehistory (rant) • Scratch install: was always a chore. • Still a chore for the L33t. • Skill not very transferrable. • Never done often enough to become good at it. • And if you are good at it, who cares?

19. Devprehistory (rant) • XAMPP: cannot upgrade components, must upgrade the whole thing. • MAMP: port hassles (privileged ports, as well as other issues). Under 1024 is privileged. • A challenge to set up mail sending, https. • Skills not useful for anything else. • Friends don’t let friends use CYGWIN.

20. But the real reason these dinosaurs will die out • Because they are not examples of the real LAMP stack. • No access to improvements served up by the vendors. • Technologies offer new things that aren't included. These kits are a bundle. You fall behind. It sucks to be you.

21. Because it is not the real stack.

22. Because it is not the stack your code is using in production.

23. Demo some stuff already! • Oracle VirtualBoxis FOSS. Also consider other hypervisors. • Installing Gnu-Linux ISO distros is pretty much the same as installing ‘on the metal’. • How to get the host and guest to talk to each other like a desktop machine talks to a remote server is the freakin’ trick. • See my blog, http://superwebdeveloper.com

24. The kind of things VMs for web development can solve

25. Your website is middleware. It lives in the host stack like a fish lives in this aquarium.

26. Your fish cannot live very well outside this aquarium. Neither can your website live outside of the stack.

27. BTW, what is wrong with this picture?

28. You wouldn’t know if you didn’t see the fish in the aquarium. You don’t put cichlids in with angel fish! Cichlids are aggressive fish that need brackish water. Besides, these fish cannot exist in the same Phbalance as the community environment fish (the tetras). And the plants you see would be eaten by the cichlids.

29. The same problem exists with code as with these fish out of an aquarium Even common blog middleware like Drupal and WordPress push the boundaries of the default php.ini settings. To be competitive and feature driven, new middleware products require tweaks to the vanilla directives in PHP.ini, my.ini, Apache2.conf and other software. Suhosin PHP patch comes to mind as well.

30. Sometimes, in the business world of web development, the king has no clothes.  Sorry, I could not find an appropriate slide for that.

31. PHP outside of a web server is about as useful as a fish on a sidewalk. • They want the fish without the fishbowl. • The whole product of the enterprise is never held as a discrete object. • You never own a product unless you can separate it from everything else, including hosting, and can demonstrate it at will.

32. Stupid business practices

33. Stupid business practices • No comprehensive IT policies in place is too often the reality. • No effort to standardize working stack, beyond version control. • Team leaders & management express total indifference to how team members build their environments. • A total disconnect between what a dev does and a sysadmin does.

34. More stupid business practices • Teams are made up of multiple roles, Devs, SysAdmin, designers, JavaScripters, lead, PM, sales, management, Qa, executives, deployment, consultants, third parties, clients, interns, kitties, puppies, and sociopaths. • Different roles have different skill sets, and no effort is made to standardize skills across team. • Standardize skill sets - use of this skill set is a way to determine if a person is competent to be on the team.

35. More on that… • The old way burns money on unproven ideas. • OR jobs that the client hasn’t paid for just yet. • Valuable SysAdmin skills, knowledge, authority is ‘hoarded’. • Devsaren’t given authority extending to root.

36. It got me thinking… • Standardized virtual dev environments provides a way to easily support team members, because no two people in a chair are alike. • Hypervisors can definitely help homogenize the technology the dev team runs on. • Hypervisors can definitely support the cause of spreading Gnu-Linux and the skill set for using it.

37. Use hypervisors to spread Gnu-Linux like the Norwalk virus in Vancouver in January.

38. Using Hypervisors is an easy way to get into GNU-Linux

39. Lost productivity, lost opportunity • Sure, no uniform dev team practices are not a problem most of the time. • But it can cause confusion, poor communication, persistent bugs, and additional deployment issues - FRICTION. • Opportunity to use the software in different ways. • Improvements provided by vendors & community – patches!

40. Other wisdom • “A separation of concerns” Stefan Priebsch, thePHP.cc. • Refactoring - every coders mantra, from which Design Patterns sprung forth.

41. Other wisdom • “A grouping of Concerns.” • When you standardize skill requirements, any team member can support another. • You become TRULY AGILE, can create teams in an ad-hoc fashion (even though ‘Agile’ was concocted up to sell things to people and create buy-in by giving team members the illusion of control).

42. Test web security offline functionally, with virtual cats and fishes. Create disaster scenarios.

43. Another reason why • What if you wanted to do things that were just a leeetle bit off of the beaten path? • NGINX • NoSQL • MariaDB • ApacheSOLR • LightHTTPD • You will be SOL, or something like that. Or you need a budget. Or a lot of time. Or a really good excuse. Or beg your hosting company. Or a dev server.

44. But we have a dev server.

45. But we have a dev server. • You pay for using and maintaining the service one way or another. • Code in development is noisy in the logs. Logs that fill up the server hard drive. • Security issues also apply to dev, not just production servers.

46. But we have a dev server. • Compare to $0 cost. • This toolset is useful and available to any size of project. • A common skill set for setting up new desktop machines for development.

47. But we have a dev server. • One developer can own several desktop machines, and needs to have common dev tools regardless of platform. • dd command can publish ISO. • Virtual appliances are easy to share or convert for use in other hypervisors. • Share the VM with other team members over .torrent protocol. • Sharing Gnu-Linux via .torrent was why it was made in the first place.

48. Dream Big on teh internets • Not every idea on the Net is going to turn a buck. • Business burns cash like rockets burn fuel. • BUT teh internets hazfree tools can be obtained to develop ideas for little or zero cost. • Host any number of projects on virtual machines without incurring costs while the project is in incubation, or is dormant. • Rapid deployment.

49. Or quick and dirty, if you like. • Make throwaway experiments. • Use it for a single task. • DOS utilities may be available, but working in MS DOS is hideous. Shell is universal. • Working in the real shell may suit your job. • FTP without concern for bandwidth use. • A job with ffmpeg, editing videos from the cmd line, or other resources too big to manage any other way.

50. Do more with less • Comprehensive IT policy promotes sloth!