Download
1 / 18
180 likes | 243 Views
Garrett "Gary" Davis Google App Engine meetup 2009-10-05. DerivedProperties. Google App Engine experts recommend: optimize queries and reads, not writes. Specifically (among other things) ... Compute and store data in advance, not when reading.
E N D
Garrett "Gary" Davis Google App Engine meetup 2009-10-05 DerivedProperties
Google App Engine experts recommend:optimize queries and reads, not writes. Specifically (among other things) ... Compute and store data in advance, not when reading. Store redundant data in one record; don't use joins The goal
Storing redundant data uses ... more storage space. Too bad. more coding time to maintain that data. That's worse. The consequences
Queries for Google App Engine have limitations. Specifically (among other things) ... Query results can return only one 'table' – no joins Queries are resolved using one, and only one, index, using elements of that 'table' Therefore ... Store everything on every record needed to support every query. A related goal
Case-insensitive search Sound-alike search Number of comments for a blog entry News article or blog "slug" per this article: "Django tips: auto-populated fields" www.b-list.org/weblog/2006/nov/02/ django-tips-auto-populated-fields/ Specific uses
How do we maintain that redundant data with minimal work? The work is ... Define an extra field or fields Derive, and store, the initial value, when creating a new object Update that value, if it ever changes The challenge
Manually calculate the value whenever needed. class NameAddress(db.Model): name_last = db.StringProperty() name_first = db.StringProperty() irrelevant_details = db.TextProperty() name_lowercase = db.StringProperty() def update_name_lowercase(self): self.name_lowercase = self.name_last.lower() A first approach
Override the db.Model.put() method in the datastore. class NameAddress(db.Model): name_last = db.StringProperty() name_first = db.StringProperty() irrelevant_details = db.TextProperty() name_lowercase = db.StringProperty() def put(self): self.name_lowercase = self.name_last.lower() return super(NameAddress, self).put() suggested by the article "Django tips: auto-populated fields" http://www.b-list.org/weblog/2006/nov/02/django-tips-auto-populated-fields/ A more general approach
Override the db.Model.put() method in the datastore. class NameAddress(db.Model): name_last = db.StringProperty() name_first = db.StringProperty() irrelevant_details = db.TextProperty() name_lowercase = db.StringProperty() def put(self): self.name_lowercase = self.name_last.lower() return super(NameAddress, self).put() Problem: This works if you always use using model_instance.put() But it doesn't work if you instead use db.put() If you do, the datastore doesn't call your put() method A problem with this approach
Build something like a Python @property attribute. to automatically maintain a 'calculated value'. class NameAddress(db.Model): name_last = db.StringProperty() name_first = db.StringProperty() irrelevant_details = db.TextProperty() @property def name_lowercase(self): return self.name_last.lower() We can't use the @property decorator directly, due to magic in the datastore's Model and Property classes. Another approach
Build something like a Python @property attribute. to automatically maintain a 'calculated value'. So we define a new datastore Property type (subclass of db.Property), and override the _get__ method, which calculates the value. class CalculatedProperty(db.Property): def __get__(self, model_instance, model_class): # calculate and return the value we want Another approach
class CalculatedProperty(db.Property): def __init__(self, calc_fn, **kwds): super(CalculatedProperty, self).__init__(**kwds) self.calc_fn = calc_fn def __get__(self, model_instance, model_class): if model_instance is None: return self return self.calc_fn(model_instance) we override the __get__ method, so the system calculates the value whenever needed ... including when the datastore stores the record: see get_value_for_datastore(), which calls property.__get__() CalculatedProperty, defined
class NameAddress(db.Model): name_last = db.StringProperty() name_first = db.StringProperty() irrelevant_details = db.TextProperty() name_lowercase = CalculatedProperty( lambda rec: rec.name_last.lower() ) CalculatedProperty, applied
class NameAddress(db.Model): name_last = db.StringProperty() name_first = db.StringProperty() irrelevant_details = db.TextProperty() @CalculatedProperty def name_lowercase(self): return name_last.lower() CalculatedProperty, the decorator
Number of comments for a blog entry class BlogItem(db.Model): slug = CalculatedProperty(slugify_fn) title = db.StringProperty(required=True) body = db.TextProperty(required=True) author = db.UserProperty() created = db.DateTimeProperty(auto_now_add=True) updated = db.DateTimeProperty(auto_now=True) comment_count = CalculatedProperty(comment_count_fn)) def slugify_fn(self): # trivial / incomplete return re.sub('\s+', '-', self.title.lower()) def comment_count_fn(self): what_do_we_do_here? Try this one yourself ...
A similar concept: DateProperty(auto_now=True) DateProperty(auto_now_add=True) For further information on extending properties, read this article: "Extending Model Properties" by Rafe Kaplan of Google code.google.com/appengine/articles/extending_models.html Too much information
Nick Johnson's blog entry blog.notdot.net/2009/9/Custom-Datastore-Properties-1-DerivedProperty My article code.google.com/p/gawsh/wiki/CalculatedProperty Sources and references
Google App Engine in Action Garrett Davis, Nick Johnson, et. al. from Manning Publishing Coming to a bookstore near you, sometime next year A plug
