Data Exchange Innovations: Coming [Someday, We Hope] to a Computer Near You

1. Data Exchange Innovations:Coming [Someday, We Hope] to a Computer Near You West Region Natural Heritage Conference April 2008

2. Objectives • Present a “new” paradigm for data exchange that decouples the taxonomic update function from the MJD refresh function • Provide an update on a tool to help implement the new paradigm: TAXI (Taxonomic Analysis and eXchange Interface) • Discuss your concerns about the current data exchange process and tools

3. What are the Goals of Data Exchange? • To get global data from Central Databases to Network Databases • To get local data from Network Databases to Central Databases The Requirement AND the Result: DATA SYNCHRONIZATION!!

4. The Chicken and Egg Problem

5. The Chicken and Egg Problem

6. The Chicken and Egg Problem

7. The Chicken and Egg Problem

8. Our goal is to stay in synch all the time.... BUT HOW CAN WE GET THERE? Separate Data Synchronization from Taxonomic Updates (make them independent)

9. TAXI: The Original Plan Automate the delivery of taxonomic updates outside of traditional data exchanges

10. Spiranthes brevilabris var. floridana should be called Spiranthes floridana One of the authorities for our Element list decides to update a taxon. Centrally, we accept this change (in this case, a name change). Central Scientist initiates the change process ….

11. …by creating a Taxonomic Capsule describing the change and the authority. Central Scientist loads the capsule into the Taxonomic Analysis and eXchange Interface (TAXI).

12. The Taxonomic Analysis and eXchange Interface (TAXI) sends the appropriate taxonomic change capsule to each affected System, including the Central Databases.

13. Central Datamanager receives the capsule (notification by email) and applies it to Central Databases, accepting all changes. Local Datamanager receives capsule (notification by email) and applies it to the Local database, deciding either to accept or reject some or all of the changes.

14. Each Biotics System where the taxonomic change is pertinent must process the capsule. THEN: TAXI receives notification from each Biotics system of the change being made, and what decisions were made.

15. Agreeing to disagree If the Local Program rejects a taxonomic change, the staff person indicates the reason and submits it. TAXI generates another capsule to apply the necessary update to the now “nonstandard” Element records both in the Central and Local databases The local perspective is now correctly represented in the Central databases and matching Element records exist in both databases

16. Annual Data Exchange • Even with TAXI, we would still have Data Reconciliation on an annual basis to catch anything that fell through the cracks • It would be easier because most data would already be synchronized and most taxonomic updates already done

17. Local Programs Also Update Taxonomy What happens when the local program makes a taxonomic change? How is it communicated to the Central databases? We need a TAXI that provides service in both directions!

18. TAXI Today • Three components • Capsule Editor • Registry • Processor • Prototypes of the Capsule Editor and Registry have been created • Developed by Frank McLean with input from Margaret Ormes and Gwen Davis (and years of documentation developed by Maggie Woo and members from the Botany and Zoology Depts.)

19. TAXI: Basic Description • Revisions to plant and animal taxonomy can be described in a standard way via the Taxi Editor • These descriptions (“capsules”) will be stored in a registry accessible via the Internet to Network Members so that they will have the information to make comparable changes in their local databases • The capsule is generalized (not specific to Biotics) so that changes can be communicated to any party • There is potential to develop a processor for Biotics so that changes can be more automated

20. The Editor • For each taxonomic change, user records: • Metadata (describing the change in standardized terms and free text) • Existing Classification • New Classification • The Editor then reports a standardized description of what changes have been made

21. Step 1 • The Metadata: Indicate the type of taxonomic change, corresponding to a type on the Taxonomic Change Checklist (see KnowledgeBase Item 558!)

22. Steps Two and Three Example:Simple lump: Species lumped; no infraspecific taxa involved • Enter Existing Classification • Sorex dispar, B93WIL01NAUS • Sorex gaspensis, B93WIL01NAUS • Enter New Classification • Sorex dispar, A04RHY01NAUS

24. Step Four • “Capsules” are posted to a Taxonomic Changes Registry website • Users can select and preview information about changes • Selected documentation can be downloaded • Changes are applied manually in local databases

25. The Future • Lengthy To-Do list • Additional functionality for capsule editor • Additional functionality for registry (i.e., delete, archive) • Bug fixes • Processor - Needs to be developed; the ultimate objective is to have a processor that takes the info from the capsule and automates (or semi-automates the change in Biotics • Resources are constrained!