Understanding Java Media Framework (JMF) for Multimedia Applications

1. Java Media Framework Matthew Mastracci

2. Overview • What is JMF? • Players • Capturing • Plugins • Codecs/Effects • Mux/Demux

3. What Is JMF? • A method for working with time-based data in Java • Acquisition, processing and delivery of media • Handles real-time or stored audio and video

4. Raw Video Capture

5. Video Stream Playback

6. Audio Stream Playback

7. JMF Data Model Controls Duration Manages Datasource SourceStream Control

8. Player • Player object allows easy embedding of content • Create widget and catch realization events • Heavy-weight widget

9. Player [cont’d] public void init() { setLayout( new BorderLayout() ); String mediaFile = getParameter( "FILE" ); try { URL mediaURL = new URL( getDocumentBase(), mediaFile ); player = Manager.createPlayer( mediaURL ); player.addControllerListener( this ); } catch (Exception e) { System.err.println( "Got exception " + e ); } }

10. Player [cont’d] public synchronized void controllerUpdate( ControllerEvent event ) { if ( event instanceof RealizeCompleteEvent ) { Component comp; if ( (comp = player.getVisualComponent()) != null ) add ( "Center", comp ); if ( (comp = player.getControlPanelComponent()) != null ) add ( "South", comp ); validate(); } } Add the control panel to the bottom of the dialog Add the content to the center of the dialog

11. Player [cont’d] <APPLET CODE=PlayerApplet WIDTH=320 HEIGHT=300> <PARAM NAME=FILE VALUE="sprk.mpeg"> </APPLET>

12. Player [cont’d] <APPLET CODE=PlayerApplet WIDTH=320 HEIGHT=20> <PARAM NAME=FILE VALUE="etr.mp3"> </APPLET>

13. Capturing 1. Query the CaptureDeviceManager 2. Get a CaptureDeviceInfo object for the device 3. Get a MediaLocator to create a DataSource 4. Create a Player or Processor using the DataSource 5. Start the Player or Processor

14. Capturing for Display • One step: use a player and a content specifier URL • Example: vfw://0, rtp://ip-address/type/[ttl]

15. Plugins • Plugins can be dynamically loaded in system • Add functionality to the system • Codecs, Effects, Mux, Demux, Renderers • Media handlers

16. Plugins Interface Hierarchy Plugin Codec Demux Mux VideoRenderer Renderer Effect

17. Plugin Manager • Use addPlugin() to add a new plug-in to the list • commit() updates changes to the plug-in list on disk • Register with the manager at install-time

18. Codecs • Codec interface extends Plugin • Codecs have input and output, modifying the content Codec A Codec B Data 1 Format A Data 2 Format B Data 3 Format C

19. Effects • Effect interface extends Codec • Input format of Effect is same as output Effect A Effect B Data 1 Format A Data 2 Format A Data 3 Format A

20. Multiplexer • Merges multiple streams to a single stream Data 1A Audio Data 1 Compound Mux Data 1B Video

21. Demultiplexer • Takes a single buffer and outputs to many buffers Data 1A Audio Demux Data 1 Compound Data 1B Video

22. MediaHandler Demux • Outputs multiple stream formats Audio stream Video stream

23. Applications of JMF • Teleconferencing • Multimedia editing/sequencing • Streaming multimedia web content

24. Problems with JMF • JMF 2.0 is still beta • Crashes occasionally • Can lock up video capture devices • API well documented, but usage and examples not

25. Conclusions • JMF is highly flexible • Shows a lot of promise • Major stability and documentation issues • Video For Windows (VFW) support adds great value (no need for native drivers) • Currently difficult to use to create a complex program

26. Issues for Discussion • Can JMF help propel Java into the field of multimedia display and editing? • Will Sun develop a Linux-native "performance pack" for JMF, even though Linux competes with Solaris? • What changes need to be made to the current beta version of JMF to make it more usable/stable when it becomes a release?