Download
1 / 1
10 likes | 163 Views
This system is designed to efficiently manage and update movie records by fetching existing movie IDs from various data sources. It utilizes advanced algorithms to merge, update, and create new records within the database. The program interacts with query sources and Freebase, ensuring that data integrity is maintained throughout the process. Each operation allows for the identification of matching records and the creation of references to maintain consistency. Whether updating a single entry or handling large datasets, this system streamlines the process of movie record management.
E N D
Legend: S Datasource Fetcher Datasource Fetcher datasource fetcher <program module> start / end point S E E E E E E get all existing movie ids <query source> push <db access> datasource id list datasource id list <merging> datasource id list pop <work lists> Freebase Updater Freebase Updater independent parts S datasource updater fetch movie from source record in raw DB? Raw DB update record create record no yes update Prov-O create Prov-O yes no no yes record changed? enqueue URI for merger push merger URI list pop merger S new raw record? Main DB update matching record search for matching record Update Prov-O found matching record? yes no create new main record merge into main record update Prov-O Create Prov-O create same-as refs
