System Architecture versus System Infrastructure

1. Professional Content Management Systems8th Lecture: CM System Infrastructure Dr. Andreas MautheSCC – Lancaster University

2. System Architecture versus System Infrastructure • Software Architecture • Specifies the characteristics and functionality of a CMS by defining • A generic framework • The functionality of individual components • The interfaces between components • Deals with software modules that run on top of the System Infrastructure • System Infrastructure • Physical structure of the system • Components • Hardware, i.e. IT and broadcast equipment • Server • Encoders • Storage systems • System software for the different components • Communication sub-system • Using IT based technology • More flexible • Better cost/ performance ratio

3. Infrastructure Components & Sub-Components • Servers • Computation platforms that run parts of the system software • Standard CMS servers • PC based, running MS Windows or Linux • Unix servers • Special broadcast servers • CMS servers host the different Core and Services software modules • Storage • To host essence and metadata • Disk based systems • Mass storage (tape based, tape libraries, DVD & CD juke boxes) • Network • Communication links between • Servers-to-servers • Servers-to-storage • Clients-to-servers • Network types • LAN, WAN • Dedicated networks • Broadcast networks • Serial Digital Interface (SDI) • Serial Data Transport Interface (SDTI)

4. Key Resilience Principle: Redundancy • Basic Idea • Increasing System Reliability • Requirements • 24 hours a day, 7 days a week operation • Minimise down times • Approach • Prevent and compensate from (complete) system failures • Types of Redundancy • Component redundancy • Adding vital hardware components • Redundant power supplies • Redundant network interfaces • Mirrored system disks • Redundant Array of Independent Disks (RAID) • Clustering • To support software that requires very high level of availability • E.g. databases • Service groups • Stand by server • Hot stand by • Cold stand by

5. Hardware Design & Configuration Principles • Deciding on Server Configurations • Classification of servers • Application Server • SAN Server • Database Server • Rules Try to deploy a minimum number of different server configurations. In order to do so, try to classify the servers according to common requirements, assign applications to the categories and derive the required server configuration from this • Deciding on Redundancy • Component Redundancy Add redundancy to the sub-components of the various system components, especially in the area of power supplies, cooling fans, system disks, network and host bus adapters. • Cluster Use server clusters only where necessary to support the availability of a very sensitive application. Ensure that the application has to be cluster enabled • Stand by servers Add a limited number of standby servers into the overall architecture. Choose the number and configuration of these standby servers by considering the requirements of the applications that shall be installed on these servers

6. Hardware Design & Configuration Principles (cont.) • Deciding on Storage • Online storage • High availability Use FibreChannel storage devices wherever possible. Integrate these devices into a Storage Area Network deployed as switched FibreChannel fabric • Near-online storage Use hard disk based mass storage as archive systems wherever feasible. Use data tape libraries only for content qualities that prohibit archiving on disk due to economic reasons, and for backup • Isolating Interfaces • Issue • Integration of broadcast specific devices into a generic IT infrastructure • Specific broadcast features are example video server: • SDI, AES/EBU input/ output • Linear Time Code (LTC), Virtual Interval Time Code (VITC) • RS-422 control connection • Approach • Many broadcast servers have standard IT connections  Interface Server between the two worlds Isolate domain specific interfaces by employing Interface Servers as gateways between the CMS and other systems wherever possible

7. Hardware Design & Configuration Principles (cont.) • Designing the Networks • Network types • FibreChannel networks (SAN) • A SAN communication network • A CMS communication network • Depending on the size of the operation: • Fast Ethernet • GB Ethernet • For some time to come also broadcast networks • I.e. SDI, SDTI • SAN private networks Interconnect all servers that share a SAN file system with a non-routed private network. The SAN file system server should be connected to this network via 1000baseT • Server/ SAN networks Servers that do not move mass data do not necessarily need to be connected to the SAN. It is typically also sufficient to connect them to the CMS communications network via Fast Ethernet. Servers that move mass data shall be connected to the SAN. They shall be connected to the EMMS communications network via Gigabit Ethernet • QoS Guarantees When a component must be connected to the CMS that requires a minimum guaranteed bandwidth, consider using private connections when feasible

8. Storage Systems: Server Attached Storage (SAS) • SAS Characteristics • Features • Storage directly attached to server • Server controls storage access • Issues • Operating system (i.e. file system) dependency • Performance issues • Servers tasks, e.g. serving applications, data base read-write operations, providing file and print services • Interconnection links limited to shared network bandwidth • Low-cost alternative in small scale solutions

9. Storage Systems: Network Attached Storage (NAS) • NAS Characteristics • File centric IT paradigm • Features • LAN attached disk arrays • File level I/O commands over LAN • NFS, CIFS • Support multiple OS in parallel • Issues • LAN bandwidth can become bottleneck • Lack of QoS provisions • TCP/ IP features

10. Storage Systems: Storage Area Networks (SAN) • SAN Characteristics • Dedicated High-speed network for data transmission • Features • Based on FibreChannel Multi-Layer Network Architecture • Optimised for different distances • Bandwidth of about 800 Mb/s • Hardware structure • First level: Server and client systems • Second level: A SAN fabric composed of gateways, hubs, etc. • Third level: Storage devices (online and near-online) • Advantages • Sharing of resources

11. SAN Management • SAN Management • Task • Providing a central control to all geographically distributed components • GUI access for administrator • Technologies • Control • Using Simple Network Management Protocol (SNMP) • Inband • Using SAN interconnection • Outband • Existing LAN interconnections

12. SAN-based Backup • Backup Steps • Exchange of metadata between target server and source server • Reading of user data from disk and subsequent transfer to source server • Data transfer from the source server to the target server via LAN • Securing of user data within library system connected to the server

13. Mass Storage Solutions • Requirements • Example: Video archive with 100,000 hours • 4 Mb/s = 180 TB + 25 Mb/s = 1.125 PB + 1,5Mb/s = 68 TB • Tape base mass storage considered most efficient • Videotape Based Systems • Video Library Management Systems (LMS) • Host video tapes only • Advantages • Streams can be used directly in broadcast environment • Timcode-based partial file retrieval • Disadvantages • Tight to video formats and recorders (separate solution for audio and data) • No file transfer support • Costs • IT Based Systems • IT based Mass Storage Systems (MDS) • Host data tapes (can be mixed media, i.e. video- and data tapes) • Advantages • Format agnostic • File transfer and streaming (no SDI streaming) • Disadvantages • No timecode partial file retrieval

14. Storage Devices for Long-Term Storage • Videotape Recorders • Recording • Drive records incoming video signal to a videotape • Compression can be used • Playback • From timecode to timecode • Streamed via SDI/ SDTI • Bit error rate 10 -11 • Optical Disk Drives • Examples • CD Formats, DVD, magneto-optical formats • Capacity limitations makes it only suitable for • Browse video and 44.1 kHz PCM encoded audio • Hard Disk based Online Storage • Online storage devices in SAS, NAS & SAN • Role in mass storage • Part of large storage infrastructure • Storage option for browse audio and video • RAID systems for data security

15. Data Tape Drives • Features • Recording • Data is written as files onto data tapes • Format agnostic storage • Data retrieval • Locate cartridge • Load into drive • Retrieve file • Bit error rate 10 –17 • Concealment mechanisms are available • Specific CM Requirements • Data tapes require sustained data rates • I/O buffers are required to deal with communications bandwidth variations • Combining Technologies • Different technologies to create the most cost effective solution • To consider • Bandwidth and other bottlenecks • Integration requirements • Important  Avoid proprietary solutions

16. LAN V-LAN Highspeed LAN SDI/SDTI FibreChannel/SCSI Terminals On Air Contribution Logging Journalists Editing Cataloguing ProductionServer BroadcastServer Administration CrossbarController Automation Codecs HiResCache Server LowRes Cache Server Keyframe-Server ContentMgmt. Server WebServer LegacySystems Internet MassStorage Analysis Trans-coding ImageSimilaritySearch CMS Baseline System Infrastructure

17. Disk Recorder/Stream Converter File Converter Terminals On Air Stream Server Contribution Logging Journalists Editing Cataloguing FC Switch ProductionServer BroadcastServer Administration CrossbarController Automation High Resolution File Server Codecs HiResCache Server LowRes Cache Server Keyframe-Server ContentMgmt. Server WebServer LegacySystems Internet MassStorage Analysis Trans-coding ImageSimilaritySearch Detail: High Res

18. MPEG–1 Encoders Browse Servers Terminals On Air Contribution Logging Journalists Editing Cataloguing Fast/Gigabit Ethernet Switch Conversion Servers ProductionServer BroadcastServer Administration CrossbarController Automation Low Resolution File Server Codecs HiResCache Server LowRes Cache Server Keyframe-Server ContentMgmt. Server WebServer LegacySystems Internet MassStorage Analysis Trans-coding ImageSimilaritySearch Detail: Low-Res

19. Keyframe Servers Terminals On Air Contribution Logging Journalists Editing Cataloguing Fast/Gigabit Ethernet Switch ProductionServer BroadcastServer Administration CrossbarController Automation Codecs Low Resolution File Server HiResCache Server LowRes Cache Server Keyframe-Server ContentMgmt. Server WebServer LegacySystems Internet MassStorage Analysis Trans-coding ImageSimilaritySearch Detail: Keyframes

20. Switch Switch Terminals On Air Contribution Logging Journalists Editing Cataloguing ProductionServer BroadcastServer Administration ATS Automated Cartridge System ATS CrossbarController Automation Codecs HiResCache Server LowRes Cache Server Keyframe-Server ContentMgmt. Server WebServer LegacySystems Internet MassStorage Analysis Trans-coding ImageSimilaritySearch Detail: Mass Storage

21. Switch Terminals On Air Contribution Logging Journalists Editing Cataloguing Clustered ContentManagement Server Full Text RetrievalServers ProductionServer BroadcastServer Administration CrossbarController Automation Codecs HiResCache Server LowRes Cache Server Keyframe-Server ContentMgmt. Server WebServer LegacySystems Internet MassStorage Analysis Trans-coding ImageSimilaritySearch Detail: DBMS

22. Archive Infrastructure • Core Elements • Online Storage • High-Res, Low-Res, Keyframes • Near-Online • Tape library • Tabe drives with data rates of more than 30 MB/s (i.e. 240 Mb/s)

23. Query & Retrieval • Core Elements • Web Retrieval Services run on an Application Server • High-Res, Low-Res, Keyframes • DBMS Clusters • Data Manager Elements • Essence Management DB

24. Interfaces to Production and Playout