EMu and Fotoware:

1. EMu and Fotoware: Integrating the EMu Collections Management Program with Image Management Software - Dr. Lance Wilkie, EMu Unit, Australian Museum

2. Introduction • In 2006 there was a policy decision to centralise all digital images stored in the museum onto a single image server. • Images were, with appropriate securities, to be generally accessible to the entire AM staff. • Access was to be via an IMS (Image Management System) database run via the museum Intranet. The software ultimately chosen for this task was Fotoware (Pivotal Technologies Inc.). • Other software resources that utilise images have to adjust to accommodate the new system.

3. What difficulties are presented to EMu Users?

4. Difficulty #1. Version Control: • Neither EMu nor Fotoware actually utilise the original version of an image. • Rather, they each create a copy of the original in a known location on a server.

5. Workstation with EMu client installed Default situation EMu: EMu database (eg. “Amlive”) Open EMu Multimedia, create data record ie. This is the data record Select “attach image” Reference link to main image copy inserted in data record Thumbnails inserted in data record EMu Server Exact copy of original made on EMu server OUTSIDE of Emu database Main EMu copy of original image Original image (eg. On C: drive)

6. Default situation Fotoware: Get original image Main copy of original image added to Fotoware Original image (eg. On C: drive) Workstation with Fotoware installed Record metadata Image metadata inserted into main image ie. This is the “data record” Image Server Thumbnails created in Fotoware from main image

7. ...leading to: EMu Server IMS Server Original image (now a completely unmanaged resource) THESE ARE NOW TWO TOTALLY UNRELATED IMAGES. ie. EDITS TO ONE WILL NOT BE SEEN IN THE OTHER!! EMu copy of original image (attached to data record with ~ 2000 available data fields) Fotoware copy of original image (data inserted into image, hence image itself is the data record)

8. Workstation with EMu client installed The simple solution: Thumbnails inserted in data record EMu database (eg. “Amlive”) Reference link to main image copy inserted in data record Open EMu Multimedia, create data record EMu Server Navigate to location of original image Select “attach image” Exact copy of original made on Image server instead of EMu server Original image (eg. On C: drive) IMS Server

9. ...leading to: EMu Server IMS Server EMu client on pc. EMu database (eg. “Amlive”) EMu data record containing thumbnails and reference link to main image on IMS server Fotoware & EMu copy of original image both housed on IMS server

10. However: • This does not accommodate the directive that all images are to be made available to Fotoware. • It merely transfers the location EMu saves its copy of the image from the EMu server to the IMS server. • None of the metadata essential to Fotoware is embedded in the image.

11. Difficulty #2. Data Redundancy: • Fotoware images must have embedded metadata – it requires additional fields to those automatically populated by eg. a digital camera for the purpose of image search and retrieval, hence manual data entry is required. • For EMu users who have already attached images in the EMu multimedia module and added metadata to data fields in the associated Multimedia record this represents double data entry for a very large backlog of records. • Furthermore any subsequent data edits done to Multimedia records would have to be also done in Fotoware as embedded metadata to ensure version control is maintained.

12. Difficulty #3. Resource Types: • Non-image digital formats such as video files, sound files and pdf’s cannot be stored on the IMS server. • EMu has to recognise what sort of resource has been attached to a Multimedia record, then determine how to process it ie. Is it to be stored on the IMS server or the EMu server?

13. The Technical Solution Proposed By The Software Companies: Changes to EMu: Changes to Fotoware: • Change location for storing image copy from EMu server to IMS server • Do a one-off move of all images currently on the EMu server to the IMS server and update links in EMu • Detect multimedia field type on attachment and determine if it is going to make it’s copy on the IMS server (static image files), or the Emu server (pdf’s etc). • For images, on attachment a text file is automatically generated drawing information from pre-specified fields and saved to a specified location on the IMS server (for subsequent metadata population in Fotoware) • The original EMu copy is not saved to exactly the same location on the IMS server as the image used by Fotoware, but in a separate subdirectory. • After Fotoware has embedded metadata from the text file, EMu deletes it’s copy of the image and the text file • EMu then re-imports the image from the Fotoware location. It’s auto-extract function should be set to ignore the specific metadata that was just embedded from the text file, as that came originally from EMu anyway. • Fotoware detects a new image and text file in the EMu subdirectory on the IMS server, and automatically embeds the text file contents into pre-specified metadata fields in the Fotoware version of the image. • Fotoware then places the newly edited image in a second subdirectory for retrieval by EMu. • Fotoware does the same thing with text updates to key fields in the Emu Multimedia module.

14. Workstation with EMu client installed Visually: Copy of original Image inserted into subdirectory on IMS server Find and attach original image Emu version of original image deleted Create EMu data record Record metadata EMu Server Text file deleted Saved into subdirectory on IMS server Data in txt file imported into new Fotoware version of image and moved to new subdirectory Data extracted from EMu in txt file Image Server Link to IMS version of image reimports updated image to EMu

15. NOTE: • The above is only for static digital images • Video, Audio and other digital formats such as pdf all have to be recognised automatically by the new EMu Multimedia module at the point of image attachment and treated differently • These files are simply treated in the current fashion, ie. copy of original file saved on EMu server • The text file of metadata is not created and saved to the IMS server for ANY of the above formats

16. Benefits to EMu Users • Version control will be established and maintained. There will be only one resource being utilised by both pieces of software, and key metadata will be dynamically shared between the two. • EMu users will not be required to do double data entry, even when subsequently updating certain key fields in the Multimedia module. • EMu users will not be required to do a vast backlog of metadata population in Fotoware for images already existing in EMu with associated metadata in fields in the Multimedia module

17. How Did We Do? • Received final version of modified EMu Multimedia handler in June 2009. • From June to July did an extensive period of testing the integration with Fotoware. All bugs detected in testing were successfully resolved. • In July 2009 attempted a bulk migration of all static images currently on EMu server to the IMS server. • Of 28427 images transferred, 27800 were sucessfully incorporated into Fotoware, updated and made available back to EMu. • 627 images however were never resupplied back to EMu. Further investigation also revealed other glitches not detected in testing for some records including lack of data in key fields from source material, truncation of metadata by Fotoware on import, and conversion of file extensions by Fotoware making them unrecognisable to EMu.

18. However....

19. Tips from our experience: • Avoid non-standard file extensions for images (eg. JPG, jpeg). UNIX-based systems like Emu are very literal in their capacity to recognise file extensions. Windows-based multimedia handlers like Fotoware are not. They have a tendency to re-name file extensions (in this example to jpg). • Ensure all key metadata fields are fully populated in EMu prior to migration. • Avoid special characters in metadata fields in EMu that are to be shared with Fotowareeg. #, /. • Avoid lengthy file names.