1 / 23

EMu 24/7 @ Museum Victoria

EMu 24/7 @ Museum Victoria. ‘The bird that never sleeps’ Donna Fothergill David Zhang. Overview. Why? The decision to make EMu available 24/7 at Museum Victoria How? The Infrastructure making this possible (hardware/scripts/resources) What’s Next? Other Tools. The MV EMu Guarantee.

merrill
Download Presentation

EMu 24/7 @ Museum Victoria

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EMu 24/7 @ Museum Victoria ‘The bird that never sleeps’ Donna Fothergill David Zhang

  2. Overview • Why? • The decision to make EMu available 24/7 at Museum Victoria • How? • The Infrastructure making this possible (hardware/scripts/resources) • What’s Next? • Other Tools

  3. The MV EMu Guarantee EMu available 24/7 • In the event of: • Primary Server Crash • System/Database Maintenance • EMu upgrades

  4. Importance Core business system • System Integration • - MV Images • - MVWISE • Live Feeds • - e.g. The Learning Federation • Online collections • e.g. Natural Sciences

  5. MVWISE ‘EMu in the palm of your hand’ + = EMu MVWISE Mobile Technology

  6. MV Images

  7. Systems Integration • EMu Full synchronisation – desktop and handheld devices Data sharing using iMu web services Core database for all Collection items • MV Images • MVWISE Application for location and inventory control Digital asset management system for all digital images and audio in the State Collection and other resources

  8. Live Data Feeds ‘The Learning Federation’

  9. Online Collections ‘Search Natural Sciences Collections’

  10. Remote Access ‘regardless of connection, location or device’

  11. Extending Access ‘Where ever Internet Access is available, so too is EMu! ’ • Fieldwork Projects/Special Projects • Out of Hours Availability • Testing environments

  12. Primary Server Failure Synchronisation Maintenance/Upgrades Snapshot

  13. Backend processes • EMu is synchronised from live server to backup server every hour • Full EMu access is available using the EMu client (except during maintenance) • Snapshot taken of backup service prior to maintenance • During maintenance, the access becomes read-only.

  14. Infrastructure ‘EMu Synchronisation process’ Primary Server Data processing is carried out on the backup server for ‘fast’ synchronisation Changes to live EMu are transported to EMu on the backup server Live EMu Server Backup EMu Server Dedicated 10 GB link and RAM for synchronisation temp space Email notification sent to support staff

  15. Synchronisation Performance Live EMu service = 75GB Tue Aug 30 12:02:05 EST 2011...start emu sync to backup server Tue Aug 30 12:02:05 EST 2011...stop emu load Tue Aug 30 12:02:11 EST 2011...get all the deletions search for new/changed multimedia objects Tue Aug 30 12:02:18 EST 2011... start searching new img Tue Aug 30 12:02:19 EST 2011...start exporting modified/new records Tue Aug 30 12:02:22 EST 2011...move data to backup server Tue Aug 30 12:02:25 EST 2011...moving loads directory to backup server Tue Aug 30 12:02:25 EST 2011...restart backgroud load Tue Aug 30 12:02:26 EST 2011...finish sync on bronte

  16. Synchronisation Summary • A dedicated private link and RAM for temp space • Data processing is deferred to the backup server • It takes less than 30 seconds for synchronisation to complete • ‘Checks and whistles’ in place to avoid data corruption

  17. Infrastructure Maintenance is carried out on Live EMu ‘EMu Snapshot prior to maintenance ’ Backup Server Primary Server Live EMu is synched to the Backup server Live EMu Service Backup of live EMu Service Snapshot created from backup of live EMu Service Snapshot EMu is read only Service changes from emu to snapshot EMu And back again when maintenance completes

  18. Snapshot Summary • Live EMu is synched to backup server • The snapshot is taken from the backup live service and placed on the primary server. • It is a read only copy of the live service. • New connections and/or requests after 5pm are directed to the snapshot • When maintenance is complete, the service switches back to the live service. • Synchronisation process starts again

  19. What’s Next? Live EMu 24/7 ?

  20. Other Tools Managing our large data files • fupdateCompactSort • Update load/data file compaction tool • fselect • Recover Deleted Records • getDbRefDetails • Delete batches of records in the backend

  21. fupdateCompactSort Example: update 80 fields in 284,297 catalogue records + compaction texload texcompact Index Rebuild Data file 1 hour 37 minutes Update + Compaction Index Rebuild Data file < 1 minute

  22. getDbRefDetails delete batches of records /usr/local/emu/bin/getDbRefDetails etaxonomy Primary DB = "etaxonomy"; Linked Modules ecatalogue => TaxTaxonomyRef_tab 10 BirCuckooRef 1 IdeFiledAsRef 1 enarratives => TaxTaxaRef_tab 45 SpeLookalikesRef_tab 20 etaxonomy => HisAccNameRef_tab 20 TaxAvailabilityCorrectNameRef_tab 20 HomNameRef_tab 20 TaxAvailabilityCompetingHomRef_tab 20 ClaCurrentNameRef 1 PriTypeAboveSpeciesNameRef 1 ClaHybridParent2NameRef 1 SynNameRef_tab 100

  23. fselect Recover deleted records usage: fselect [options] dbname Options are: -d alternative data file -D print out ins forms -Sstatus record status -c set the status of selected records to current -a analyse -kirn get the records with this irn -un get nth record with this irn -fFile save processed records into File

More Related