Generic Change Management: The SERF Project and Beyond

1. WPI Generic Change Management:The SERF Project and Beyond In collaboration with students at WPI including: Kajal Claypool, Hong Su, Amy Lee, and others. Elke A. Rundensteiner Dept. of Computer Science Worcester Polytechnic Institute

2. WPI “Nothing is as constant as change itself” Christina Baldwin

3. WPI Motivation: Change is Everywhere • Persistent data everywhere! • And it changes ... • Why? • Mistakes in database design or modeling • Changes in real world (understanding of domain) • Inherent in design and exploratory processes • Addition of new applications to system • Support for comprehensive and generic change management needed !!!

4. WPI Views Structure change DBMS Applications Data Change DBMS Web pages DBMS Structure change Constraint change Data Change Issues in Change Management • How to specify change? • How to execute change? • How to manage effect on dependent information? DBAs

5. WPI Specification of Change

6. WPI address name name Person Person Student Employee Student Employee Schema Evolution add_attribute(address, Person) • Changes to structure of class and class hierarchy add_attribute(address, Person) add_attribute(address, Employee) add_attribute(address, Student) • Object transformations to conform to schema changes

7. WPI Issues – Specification of Change • Traditional change • Structural change and data change • Schemaevolution • Pre-defined fixed operations [Objectstore,Versant,Objectivity] • E.G.: Add or delete of an attribute • Pre-defined complex operations [Breche96, Lerner96] • E.G.: Merging two classes, split a class • Not enough – hard to meet all users’ needs... • Goal 1: need flexible mechanism

8. WPI Issues – Specification of Change • Many types of change: • Structural, data, constraint, rules, time, model • Related work: • Schema evolution [Objectstore94, Versant92, Objectivity93, Breche96, Lerner96, O2-94] • Constraint evolution [Mayol99] • Time evolution [Nascimento95] • Problem: • Individual similar solutions for each change type [ER -Roddick99] • Goal 2: Need to different kinds of changes

9. WPI Limitations of Current Evolution Technology • lack of flexibility • a fixed set of primitives • lack of extensibility • pre-determined semantics • lack of re-usability • ad-hoc programs for complex transformations

10. WPI SERF: Schema Evolution through an Extensible, Re-usable and Flexible Framework

11. WPI Goal – Generic and Flexible Change Specification Framework • Flexible: • user or domain-specific transformation • Extensible: • user can specify new semantics • Reusable: • generic and not application-specific library • Portable: • not tied to a particular system

12. WPI SERF Transformation SERF Transformation : combination of evolution primitives + specification of object transformations + OQL as the “glue logic” • SERF schema evolutionframework sits on top of existing OODB systems.

13. WPI name name street Person Person city address street state Address city state Example of Inlining

14. WPI Step A Step B Step C Step B Inline Transformation // Add required attributes to Person class add_attribute (Person, Street, String, ""); add_attribute (Person, City, String, ""); add_attribute (Person, State, String, ""); // Get all objects for Person class define extents() as select c from Person c; // Update all objects for all obj in extents(): obj.set (obj.Street, valueOf(obj.address.Street)); obj.set (obj.City, valueOf(obj.address.City)); obj.set (obj.State,valueOf(obj.address.State)); // Delete the address attribute delete_attribute (Person, address);

15. WPI SERF Template SERF Transformation + Re-callable (Named) + Generally Applicable (MetaData) + Re-usable (Parameters)

16. Re-callable named begin template inline (…..) end template Parameterized input and output parameters (className, refAttr) Example of Instantiation inline (Person, address) WPI Naming and Parameterizing a Transformation

17. Remove dependency to specific classes by using metadata (ODMG System Repository) WPI Generalizing a Transformation define localAttrs($cName) as select c.localAttrList from MetaClass c where c.metaClassName = $cName; for all attrs in localAttrs(refClass) add_attribute ($className, attrs.attrName, attrs.attrType, attrs.attrValue);

18. Inline Template WPI begin template inline (className, refAttrName) { refClass = element ( select a.attrType from MetaAttribute a where a.attrName = $refAttrName anda.classDefinedIn = $className; ) define localAttrs(cName) as select c.localAttrList from MetaClass c where c.metaClassName = cName; for all attrs in localAttrs(refClass) add_attribute ($className, attrs.attrName, attrs.attrType, attrs.attrValue); define extents(cName) as select c from cName c; for all obj in extents($className): for all Attr in localAttrs(refClass) obj.set (obj.Attr, valueOf(obj.refAttrName.Attr)) delete_attribute ($className, $refAttrName); } end template

19. WPI Error to User Bind-check Instantiate Execute syntax-check Type-check pass pass Flow of a Template inline(className, refAttrName) Template className = ClassPerson, refAttrName = Attributeaddress error error inline(Person, address) Pure oql statements error pass

20. WPI Schema Viewer Template Processor Schema Repository Schema Evolution Manager SERF Architecture Template Manager User Interface Template Editor Instantiates and executes uses Template Library SERF Framework uses uses uses OODB System Query Engine uses operates on queries queries Object Repository

21. WPI SERF Architecture

22. Flexible define semantics of choice Extensible define new transformations Re-usable define embedded transformations Portable applicable to different ODMG systems WPI Advantages of SERF

23. Identify limitation of fixed SE primitives Developed SERF solution Transformations primitives + OQL + system update methods Templates generalized, named and parameterized transformations Library of Templates (from Case Study) ODMG based SERF prototype WPI Conclusions

24. WPI Does it give us Consistency? name name inline street Person Person city address street state Address city Department Department state Office Office

25. WPI Consistency Responsibilities • Who should be responsible for guaranteeing consistency? • SERF Templates or • Core Schema Evolution Primitives or • End User or Application Developer

26. WPI Concept of “Contracts” into Templates • Mechanism for declaratively expressing constraints of “schema” in template • System-enforced consistency guarantee • Use for consistency of relationship evolution --> ER’2000 • Use for software upgrade itself for better maintainability --> TOMORROW

27. WPI Performance of such Complex Templates • Schema evolution are expensive methods • Evolution is embedded within multiple query statements • CHOP: ECOOP OODB workshop’2000 on an optimizer for evolution sequences • CHOP-OQL: optimizer for embedded oql queries [ MS thesis, Ming Li, 2000 ]

28. WPI Supporting Other Applications

29. WPI Issues: Supporting Other Applications Views • Current approaches • flexibility in structure of queries [Lieberherr94] • Adapting queries using available redundant information [EVE97] • Hiding change from other applications [TOSE98] • Versioning [Zdonik, Lauteman, Sommerville] Applications ? View Evolution change ? ? DBMS Structure change DBAs

30. Schema Evolution • Examples: • Add-attribute: capacity-augmenting (CA) • Delete-attribute: capacity-reducing (CR) • Change-class-name: capacity-equivalent (CE)

31. App1 App2 App1 Data Deleted Application Interoperability ProblemI: Deleting Data old new Schema Data

32. App1 App2 App1 View Data Removed Transparent Object Schema Evolution: View-based Solution old View View new Schema Data

33. App1 App2 App2 new Application Interoperability Problem:II: Adding Data old Data Added Schema Data

34. App1 App2 App2 new TOSE for Capacity-Augmenting Changes old View View View Data Added Schema ? Data ?

35. Previous Solution:Transparent Schema Evolution • For Capacity-Reducing/-Equivalent: Derive target view schema from input view schema by e.g. hiding • For Capacity-Augmenting: Require capacity-augmenting views to store additional capacity Capacity-Augmenting Views not (yet) Commercially Available!

36. VS VS’ CA Schema Change User-defined SC 2 In-Place SC 1 BS BS’ View Derivation 3 TOSE: Three-Pronged Process for Capacity-Augmenting Schema Changes 1. Base augmentation step (schema change) 2. Target generation step (view) 3. Original base restoration step (view) Requires only conventional view- and schema evolution tools!

37. 2 Degenerate Process for Capacity-Reducing/-Equivalent SCs 1. Base Augmentation Step (skip) 2. Target Generation Step (from VS) 3. Original Base Restoration Step (skip) CR or CE SC VS VS’ User-defined SC BS=BS’ View Derivation

38. Update Semantics Preservation(USP) • Given: Users work on view schemas instead of on shared database • Goal: Provide base-like working environment • Requires: • Updates on views translated into updates on base schema • Back-propagation from base to views have same effects as if updates directly performed on views

39. WPI USP Virtual Class • Definition: Update on virtual class (VC) has same effect as if VC were base class • Mechanism: Create, delete, and set update applied to VC is translated to its source class, and then propagates back to VC. • Example: Person’ is usp. hide(income,Person) name Person’ Person name income (hide virtual class) (base class)

40. WPI USP Case Study • Use MultiView’s object algebra as basis • Identify set of algebra operators generating virtual classes sufficient for TOSE, namely: ident, hide, union, difference, intersect. • Prove this set of view derivation operators to be USP.

41. WPI USP Definitions • Definition: (VCi,VCj) is a usp pair, if (VCj,VCi) possesses typical update behavior as if they were base classes. • Definition: view schema VS is usp if every pair of classes in VS is a usp pair. • Definition: VD(Si) = Si, is usp derivation if Si is guaranteed to be usp whenever Si is usp.

42. name income name Person income Capacity-Augmenting Schema Change Example VS’ VS add-attr(birthday, Person’) Root Root name Person’ income Person’’ birthday Student’ Student’’ Root Dept Student BS

43. name income name income Step 1: Base Augmentation VS’ VS add-attr(birthday, Person’) Root Root name Person’ income Person’’ birthday Student’ Student’’ 1 Root Root name income Person* Person Dept Dept* birthday Student Student* BS BS’

44. name income name income Step 2: Target Generation VS’ VS add-attr(birthday, Person’) Root Root name Person’ income Person’’ birthday Student’ 2 Student’’ 1 Root Root name income Person* Person Dept Dept* birthday Student Student* BS BS’

45. name income name Person income Step 3: Original Base Restoration VS’ VS add-attr(birthday, Person’) Root Root name Person’ income Person’’ birthday Student’ 2 Student’’ 1 Root Root name income Person* Dept Dept* birthday 3 Student Student* BS BS’

46. name income Capacity-Reducing Schema Change Example del-attr(income, Person’) Root Root Person’ Person’’’ name Student’ Student’’’ VS’’ VS VS’’ = VDdel-attr(income,Person’)(VS(BS))

47. name income Capacity-Reducing Schema Change Example del-attr(income, Person’) Root Root Person’ Person’’’ name hide(income, Person’) Student’ Student’’’ VS’’ hide(income, Student’) VS VS’’ = VDdel-attr(income,Person’)(VS(BS))

48. WPI TOSE Transparency Theorem • Study complete taxonomy of schema evolution primitives • If capacity-reducing or -equivalent: give corresponding usp view derivation • If capacity-augmenting transformation a. give three steps of process b. prove their correctness

49. WPI Advantages of TOSE • Applications can interoperate on same set of objects flexibly • Shared schema can evolve without affecting existing applications (migration) • Data consistency is guaranteed, since there is only one copy of base objects • TOSE relies on standard tools only

50. WPI TOSE Implementation • SPARCstation 10 • GemStone 4.0 • SunOS 4.1.3 TOSE MultiView View System MultiView Object Model GemStone