1 / 19

Supporting Persistent Objects In Python Jeremy Hylton jeremy@zope

Supporting Persistent Objects In Python Jeremy Hylton jeremy@zope.com. What is persistence?. Data lives longer than programs Files, pipes, relational databases, etc. But, discontinuity of representation Orthogonal persistence Automatic management of program state

sydney
Download Presentation

Supporting Persistent Objects In Python Jeremy Hylton jeremy@zope

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Supporting Persistent ObjectsIn Python Jeremy Hyltonjeremy@zope.com

  2. What is persistence? • Data lives longer than programs • Files, pipes, relational databases, etc. • But, discontinuity of representation • Orthogonal persistence • Automatic management of program state • Independent of data type or longevity • Allow programmer to focus on application data model

  3. Python approach • Goals • Minimize changes to existing programs • Work with standard interpreter • Zope Object Database (ZODB) • Support Zope application server • Originally targeted for web development • Separate database and storage layers

  4. Not on today’s agenda • Benefits of persistence • Discussion of related work • Early work • PS-algol, Napier88 • Much recent work on Java • PJava, ObjectStore PSE

  5. ZODB key ideas • Persistence by reachability • Any reachable object is saved • Transactions to control updates • Safe sharing among clients • Implement with Persistent mixin • Provides hook for persistence • Compromise on transparency

  6. Persistent base class • C extension type that defines • Four _p_ attributes • Custom attribute accessors • Efficient C API • Marks serialization boundaries • Interacts with database • Load objects on demand • Register modifications with TM

  7. Persistence by reachability • Any object reachable from ZODB root is persistent

  8. A simple example • from Persistence import Persistentfrom Transaction import get_transactionfrom ZODB.FileStorage import DBclass Counter(Persistent): _value = 0 def inc(self): self._value += 1def main(): fs = DB(“data.fs”) conn = db.open(); root = conn.root() obj = root[“myobj”] = Counter() get_transaction().commit() obj.inc() get_transaction().commit()

  9. Object serialization • Standard pickle library • Serializes arbitrary object graph • Raises TypeError for sockets, files, &c. • Instance vars serialized via dictionary • Hooks to define custom state • __getstate__() / __setstate__() • Persistent mixin ignores _v_ attributes

  10. Pickling persistent objects • Stores objects in separate records • Persistent objs pickled as oid + class • Works with cache to maintain identity • Handling non-persistent objects • Copied into record of containing object • Sharing by persistent objs is problematic

  11. Object identity / caching • Cache maintains oid  obj mapping • Guarantees only one copy of object • Unpickler loads all referenced objects • Ghost objects • Only Persistent header initialized • No instance state loaded • State loaded on first object access • LRU cache of recent objects

  12. Attribute access handlers • Persistent implements C wrappers • Override tp_getattro, tp_setattro slots • Mediate access to instance variables • Crucial Python feature

  13. Example __getattribute__() hook class Example(object): _p_state = False def _p_activate(self): print "activate" self._p_state = True def __getattribute__(self, attr): print "intercept", attr if not attr.startswith("_p_") and not self._p_state: self._p_activate() return super(Example, self).__getattribute__(attr) >>> obj = Example(); obj.value = "test" >>> print obj.value intercept value intercept _p_state intercept _p_activate activate test

  14. Transactions • Supports multiple threads, processes • Independent database connections • Updates visible at transaction boundaries • Optimistic concurrency control • When conflict occurs, abort and retry • On error, abort to restore consistency • Reverts to last saved state

  15. Concurrency and conflicts • Invalidations sent at commit time • Clients process at transaction boundaries • Conflicting transactions aborted • Write conflict at commit time • Read conflict on object access • Application must retry on conflict • Can use generic wrapper • Can define conflict resolution method

  16. Retrying conflicts • Example wrapper for retries def transact(f, retries=3): def wrapper(*args, **kwargs): n = retries while n: try: try: return f(*args, **kwargs) finally: get_transaction().commit() except ConflictError: n -= 1 if n == 0: raise except: get_transaction().abort() raise return wrapper

  17. Other features • Undo support • Storage stores multiple revisions • Transactional undo reverts to earlier state • BTrees: efficient persistent containers • Storing code in database

  18. Limitations • Schema evolution • Must code manually in __setstate__() • Database management • Manual pack() to remove revisions, do GC • Sharing of non-persistent objects • Integration with legacy code • Multiple inheritance helps • Factory classes

  19. Getting the software • http://www.zope.org/Wikis/ZODB • info central for ZODB • ZODB 3.1 released Oct. 2002 • Uses ExtensionClass • ZODB4 alpha planned for Dec. 2002 • Based on Python 2.2 type support • Fewer onions in the varnish

More Related