Python and Web Development

1. Python and Web Development • Can’t possibly do justice in 60 minutes • So we’ll be flying at 30,000 feet. • Quick URLs: • http://python.org/ • http://webware.sourceforge.net/ • http://www.python.org/cgi-bin/moinmoin/WebProgramming

2. Who is your presenter? • Chuck Esterbrook • Independent Contractor/Consultant since Spring 2000 • Most work since then has been in Python • But also Java, C# and even (gasp!) VB • Going backwards: Project Manager, Senior Software Engineer, B.S. in Comp Sci • Have used a variety of tools for a variety of companies in a variety of roles • My programming language of choice is Python • Author of O.S. product: Webware for Python • Co-founder of SANDPYT

3. Why Python? • At a really high level: Productivity • In IT, Productivity = Happiness • Python was designed to be • “-able” as in readable, writeable and maintainable • Easy to use • Powerful • Balances all of these • Contrasted with lack of balance in other langs: • Perl: writeability to the detriment of others • C++: performance to the detriment of others • VB: legacy BASIC to the detriment of everything

4. Python productivity: Ubiquity • Can’t stress this enough: • Python works well in almost all areas:web, gui, sys admin, simulation, finances, etc. • So you can take your acquired skills and libraries everywhere • Hit the ground running • Easily reuse your own home-brewed libs, 3rd party libs, etc. • Contrast: • C++: Not ideal for sys admin or rapid development • Perl: Not ideal for large scale or team-based • VB: Not ideal for anything

5. Python: Selective popularity • Python has no marketing budget, but • Still gets vote of confidence by successful, well-known companies: • Google • Yahoo! • Industrial Light & Magic • These companies can afford and use any tools they want. Their choices include Python. • Dr. Dobb’s Journal lists Python across the top of their mag. cover right beside Linux, XML, Win32, etc.

6. Python programs are problem-oriented • Common sensation among new Python programmers, including myself: • “In Python, I’m dealing with my problem instead of my language.” • Again, contrast: • C++: obscure compilation problems • Perl: reading obscure code • VB(6): no inheritance, no exception handling, etc.

7. What does Python look like? • Let’s get this out of the way: # HelloWorld.py print “Hello, world”

8. #!/usr/bin/env python # What does Python really look like? # like "wc -l <filenames> | sort -n" import sys defmain(args=None): if args is None: args = sys.argv linesPerFile = [] for filename in args[1:]: # arg[0] is prog n = len(open(filename).readlines()) t = (n, filename) linesPerFile.append(t) linesPerFile.sort(myCompare) # print it for lines, filename in linesPerFile: print "%4i %s" % (lines, filename) defmyCompare(t1, t2): return cmp(t1[0], t2[0]) if __name__=='__main__': main()

9. What my Python really looks like. • Almost all my Python is Object-Oriented. • We’re talking classes, objects and methods. • Python is easiest OO language I’ve used to date. • Like rest of language: • Simple; easy to learn • Powerful • Gets to the point • More OO specifics: • Full dynamic binding • Easy introspection (aka “reflection”) • Easy hooks for operator overloading (see next slide) • Multiple inheritance (see next slide)

10. What once was evil is now good! • I left C++ thinking “multiple inheritance is bad” and “operator overloading is bad” • After experiencing them in Python, I realize both are good if done right • Multiple inheritance • Full dynamic binding • No strange compilation errors, or mysterious crashes • Used mostly as a “mix-in” stylehttp://www.linuxjournal.com/article.php?sid=4540 • Imagine if Java interfaces provided “default” or “canonical” implementations of some methods. • Bottom line: m.i. increases productivity • Operator overloading • Just normal operators

11. # What does Python OOP look like? class Node: def__init__(self, name, superNode=None): self.name = name self.superNode = superNode if superNode: superNode.subNodes.append(self) self.subNodes = [] defdump(self, out=None): if out is None: out = sys.stdout self._dump(out, 0) def_dump(self, out, indent): out.write(' '*4*indent + str(self) + '\n') indent += 1 for node in self.subNodes: node._dump(out, indent) def__repr__(self): return '<%s %r %i-subnodes 0x%x>' % ( self.__class__.__name__, self.name, len(self.subNodes), id(self))

12. Output animal = Node('animal') mammal = Node('mammal', animal) cat = Node('cat', mammal) dog = Node('dog', mammal) insect = Node('insect', animal) animal.dump() <Node 'animal' 2-subnodes 0x8fa260> <Node 'mammal' 2-subnodes 0x8fa8c8> <Node 'cat' 0-subnodes 0x8fa378> <Node 'dog' 0-subnodes 0x8fa418> <Node 'insect' 0-subnodes 0x8fa440>

13. Other Python Highlights • Built in lists and dictionaries • Exception handling • Try…except…else…finally • raise SomeError(args) • Full garbage collection • Imperative with common constructs • for, while, break, continue, func() • Var args by position or keyword • Doc strings • Modules and packages • Platform independent by default andspecific by choice

14. Learning Python • Lots of options! • I like:http://python.org/doc/current/tut/tut.html • All the links you need:http://python.org/topics/learn/ • Books abound (check BookPool.com, Amazon) • SANDPYT – San Diego Python User’s Grouphttp://sandpyt.org/

15. Web dev • One hour presentation as intro toboth Python and Python web dev! (heh) • Web dev options: • CGI: Bleck. • Might be fast enough for some sites, but certainly feels wasteful. • Encourages nekkid scripts. i.e., non-OO • FastCGI: A band-aid useful for existing CGI apps. • App Servers • Webware • Zope • Others

16. What is Webware for Python? • Server-side web development tools that leverage Python. • Most similar to Java web tools and Apple WebObjects • Some overlap with CGI, CF, Zope, PHP, ASP… • Covers common needs of web developers • Open source development and community • Python license • Cross-platform; works equally well on: • Posix in its many flavors (Linux, BSD, Solaris, UNIX…) • Windows NT/2000/XP • Modular architecture: components can easily be used together or independently • Object-oriented

17. Comparisons to other Tools • No religious fervor allowed during this slide. • I designed Webware after using various web tools. So I addressed shortcomings from the start. • Unlike PHP, Webware leverages a general purpose language (Python) and everything that comes with it • Same with ColdFusion; also not closed-source • Unlike CGI, Webware is fast and providesa good OO structure • Similar to Java web tools, but better language and less bureaucratic APIs • Zope: WW is less monolithic, direct access to Python, programmer-oriented, etc.

18. What is in Webware? • The heart of Webware is WebKit, the application server • And: • Python Server Pages (PSP) • TaskKit • MiddleKit • UserKit • All of these are Python packages • WebKit includes the App Server which you will run continuously as with other servers (web, db, etc.)

19. WebKit • A fast, easy-to-use Python application server • Multi-threading, not forking • Makes persistent data easier • Works well on Windows • Supports multiple styles of development: • Servlets • Python Server Pages (PSP) • Custom file extension handling • Extensible • Servlet factories • Plug-ins • Import any Python module. ;-)

20. Is it real? Yes! • Been around since spring 2000 including contractual work • Stable and mature • Used in several real-world, commercial projects: • http://webware.sf.net/wiki//WhoIsUsingWebware • http://StockAlerts.com/ • http://StevesStockPicks.com/ • http://www.Vorbis.com/ - open free audio • http://www.ElectronicAppraiser.com/ - real estate • http://PatientWire.com - e-commerce for optometry • And others including many private Intranets

21. Architecture Browser XML-RPC client 80 80 Server Apache mod_webkit WebKit.cgi 8086 8086 WebKit Servlets PSPs Database Filesystem

22. Starting the app server • Installation instructions areincluded with Webware • Ways to connect web server & app server: • WebKit.cgi – least common denominator • mod_webkit – fast • In your working directory, run: • Unix: • cd /usr/local/webapps/webinator • ./AppServer • Windows: • cd C:\MyWebApps\Webinator • AppServer

23. Using the Example servlets and PSP’s • To use the CGI adapter, surf to: • http://localhost/cgi-bin/WebKit.cgi • To use the mod_webkit adapter, surf to: • http://localhost/webkit • Experiment and enjoy!

24. Servlets • A Python class located in a module of the same name • Must inherit from WebKit.Servlet or one of its subclasses: • WebKit.HTTPServlet • WebKit.Page • A common technique is to make your own subclass of WebKit.Page called SitePage which will contain: • Utility methods • Overrides of default behavior in WebKit.Page • Simplest servlet: from WebKit.Page import Page class HelloWorld(Page): def writeContent(self): self.writeln(‘Hello, World!’)

25. The Request-Response Cycle • User initiates a request: • http://localhost/webkit/MyContext/MyServlet • This activates the MyContext context, and the MyServlet servlet, based on settings in Application.config • Note: no extension was specified, even though the file is called MyServlet.py • There are settings in Application.config that control the way extensions are processed • An instance of the MyServlet class is pulled out of a pool of MyServlet instances, OR if the pool is empty then a new MyServlet instance is created. • A Transaction object is created. • These methods are called on the MyServlet instance: • Servlet.awake(transaction) • Servlet.respond(transaction) • Servlet.sleep(transaction) • The MyServlet instance is returned to its pool of instances.

26. HTTPRequest • Derived from generic Request base class • Contains data sent by the browser: • GET and POST variables: • .field(name, [default]) • .hasField(name) • .fields() • Cookies: • .cookie(name, [default]) • .hasCookie(name) • .cookies() • If you don’t care whether it’s a field or cookie: • .value(name, [default]) • .hasValue(name) • .values() • CGI environment variables • Various forms of the URL • Server-side paths • etc.

27. HTTPResponse • Derived from generic Response base class • Contains data returned to the browser • .write(text) – send text response to the browser • Normally all text is accumulated in a buffer, then sent all at once at the end of servlet processing • .setHeader(name, value) – set an HTTP header • .flush() – flush all headers and accumulated text; used for: • Streaming large files • Displaying partial results for slow servlets • .sendRedirect(url) – sets HTTP headers for a redirect

28. Page: Convenience Methods • Access to the transaction and its objects: • .transaction(), .response(), .request(), .session(), .application() • Writing response data: • .write() – equivalent to .response().write() • .writeln() – adds a newline at the end • Utility methods: • .htmlEncode() • .urlEncode() • Passing control to another servlet: • .forward() • .includeURL() • .callMethodOfServlet() • Whatever else YOU decide to add to your SitePage

29. Page: Methods Called During A Request • .respond() usually calls .writeHTML() • Override .writeHTML() in your servlet if you want your servlet to provide the full output • But, by default .writeHTML() invokes a convenient sequence of method calls: • .writeDocType() – override this if you don’t want to use HTML 4.01 Transitional • .writeln(‘<html>’) • .writeHead() • .writeBody() • .writeln(‘</html>’)

30. Forwarding & Including • self.forward(‘AnotherServlet’) • Analogous to a redirect that happens entirely within WebKit • Bundles up the current Request into a new Transaction • Passes that transaction through the normal Request-Response cycle with the indicated servlet • When that servlet is done, control returns to the calling servlet, but all response text and headers from the calling servlet are discarded • Useful for implementing a “controller” servlet that examines the request and passes it on to another servlet for processing • self.includeURL(‘AnotherServlet’) • Similar to .forward(), except that the output from the called servlet is included in the response, instead of replacing the response.

31. Sessions • Store user-specific data that must persist from one request to the next • Sessions expire after some number of minutes of inactivity • Controlled using SessionTimeout config variable • The usual interface: • .value(name, [default]) • .hasValue(name) • .values() • .setValue(name, value) • And dictionary-like access for values: • sess = self.session() • sess[‘userId’] = userId

32. PSP: Python Server Pages • Mingle Python and HTML in the style of JSP or ASP • Include code using <% … %> • Include evaluated expressions using <%= … %> • Begin a block by ending code with a colon: <%for I in range(10):%> • End a block using the special tag: <%end%> • When the user requests a PSP: • It is automatically compiled into a servlet class derived from WebKit.Page • The body of your PSP is translated into a writeHTML() method

33. PSP Example <% def isprime(number): if number == 2: return 1 if number <= 1: return 0 for i in range(2, number/2): for j in range(2, i+1): if i*j == number: return 0 return 1 %> <p>Here are some numbers, and whether or not they are prime: <p> <%for i in range(1, 101):%> <%if isprime(i):%> <font color=red><%=i%> is prime!</font> <%end%><%else:%> <%=i%> is not prime. <%end%> <br> <%end%>

34. Web Services: XML-RPC • Turn your Webware site into a “web service” • Write a servlet derived from XMLRPCServlet • Define exposedMethods() method that lists the methods you want to expose through XML-RPC • Write your methods • Sorry, no time for an example. • Bottom line: • Creating XML-RPC services in Webware is easy • Using XML-RPC services in Python is easy

35. Error Reports (i.e., Tracebacks) • If an unhandled exception occurs in a servlet: • Application.config settings: • If ShowDebugInfoOnErrors = 1, an HTML version of the traceback will be shown to the user; otherwise, a short generic error message is shown. • You can configure WebKit so that it sends the traceback by email: EmailErrors, ErrorEmailServer, ErrorEmailHeaders • Include “fancy” traceback using IncludeFancyTraceback and FancyTracebackContext • Your users will NOT report tracebacks, so set up emailing of fancy tracebacks!

36. MiddleKit • Object-Relational mapper • Supports MySQL and MS SQL Server. • PostgreSQL support soon? @@ check this • Can be used anywhere, not just WebKit applications. • Write an object model in a Comma-Separated Values (CSV) file using a spreadsheet • Inheritance is supported • Numbers, strings, enums, dates/times, object references, lists of objects (actually sets of objects) • Compile the object model • This generates Python classes for each of your objects that contain accessor methods for all fields • Also, an empty derived class is provided where you can add your own methods • And, a SQL script is generated that you can run to create the tables

37. Cheetah • http://www.cheetahtemplate.org/ • A Python-powered template engine and code generator • Uses the “dollar sign-pound sign” $# syntax found in Velocity, WebMacro, et al. @@ • Integrates tightly with Webware • Can also be used as a standalone utility or combined with other tools • Compared with PSP: • Much more designer-friendly • Perhaps less programmer-friendly?

38. Zope • I used Zope before Webware even existed. • Then I wrote Webware. • Zope has strong “through the web” CMS and some nice built-in features if they matched your application. • I found it monolithic and “interfering”. • It squirreled Python away and wrapped it with DTML and UI. • But Python was designed to be user-friendly from the start, so I wanted to use it in a natural environment.

39. Webware-Zope Quotes • “In Zope, I find myself writing a lot of External Methods to do the 'heavy lifting', and call them from within the DTML. In WebKit, the 'heavy lifting' is just part of the Python, not a requisite separate entity. • Gary Perez - Jun 11, 2003 • “I guess, it's the thinness of Webware. WW is ‘pro-Python’, which means, that everything you do is more than less directly done in Python. No DHTML, ... This -at least IMO- is one of the strong points in Webware: it doesn't put anything between the application developer and Python.” • Frank Barknecht - Jun 11, 2003

40. Choosing Webware or Zope • If you’re more of a non-technical user,Zope’s point-and-click WUI interfaceand templating, might appeal to you more. • If you like programming in Python,Webware is more likely to appeal to you.

41. Other App Servers • Zope was only mature app server when I built Webware. • I haven’t tracked the others at all. • Most popular ones seem to be SkunkWeb, Quixote and possibly Twisted Matrix. • Zope 3 is in development as successor to Zope 2. • Webware is still going strong with new developers, new users and thousands of downloads.

42. That’s All! • Any questions? • URLs: • http://python.org/ • http://webware.sourceforge.net/ • http://www.python.org/cgi-bin/moinmoin/WebProgramming