1 / 59

Java Media Framework

Java Media Framework. The Java Media Framework (JMF) is an application programming interface (API) for incorporating time-based media into Java applications and applets The JMF 1.0 API (the Java Media Player API) enabled programmers to develop Java programs that presented time-based media

azra
Download Presentation

Java Media Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Java Media Framework • The Java Media Framework (JMF) is an application programming interface (API) for incorporating time-based media into Java applications and applets • The JMF 1.0 API (the Java Media Player API) enabled programmers to develop Java programs that presented time-based media • JMF 2.0 API extends the framework to provide support for • capturing and storing media data, • controlling the type of processing that is performed during playback, • performing custom processing on media data streams • JMF 2.0 defines a plug-in API to allow developers to customize and extend JMF functionality

  2. JMF Media Processing Model

  3. Media Streams • Often contain multiple channels • Tracks • Example • MPEG-1 file usually 2 tracks • Audio track • Video track • Demultiplexing • Multiplexing

  4. Example • Process an mpeg-1 a/v media stream • Transcode video track to H.263 • Transcode audio track to GSM • Steps • Demultiplex to obtain tracks • Decompress video track • Compress using H.263 • Decompress audio track • Compress using GSM • Multiplex two tracks • Save to file

  5. JMF Design Goals • Enable input, processing and output of time-based media • Provides common cross platform API for accessing underlying media frameworks • Extensible • support additional content types and formats, optimize handling of supported format, create new presentation mechanisms

  6. Supported Content Types • Supported types • Not always both decode and encode • Differences between platform independent and dependent versions • Audio • WAV, GSM, MIDI, etc • Image • JPEG, etc • Video • H.261, H.263, MPEG-1, Quicktime, AVI, etc

  7. Recording, processing, and presenting time-based media

  8. High-level Architecture

  9. Some JMF Base Interfaces • Clock • Controller • Control

  10. Time Model • The Clock interface • Defines basic timing and synchronization operations • Contains a Timebase • Based on the system clock • time-base time • Simply provides current time • Clock marks time for a particular media stream • media time • Current position within a media stream

  11. Time Model

  12. Clock • Playback rate • How fast the Clock is running in relation to its TimeBase • Examples: • rate of 1.0 represents normal running time • rate of 2.0 means presentation will run at twice the normal rate • Clock Transform • Media-time = media start-time+ Rate(time-base time – time-base start-time)

  13. Example • Example: Have a 20 sec MPEG video stream • MediaStartTime= 10 secs, • TimeBaseTime= 3 secs • TimeBaseStartTime= 0 secs, TimeBaseTime–TimeBaseStartTime= 3 secs • Media-time= media start-time+ Rate(time-base time – time-base start-time) • So if Rate = 1.0, MediaTime= ?? • Alternatively, if rate = -2.0, MediaTime= ??

  14. Achieving Synchronization • Example • Want to force a video renderer to sync to the timebase of an audio renderer • X = audio_renderer.getTimeBase() • Video_renderer.setTimeBase(X) • Both objects would now use the same source of time.

  15. Controller Interface • Controller Interface • Defines basic state and control mechanism for an object that controls, presents or captures time-based media • Two types of Controller: Players and Processors (considered later…)

  16. Controllers

  17. Controller lifecycle

  18. Controller Events • JMF objects can post a MediaEvent • Events posted by a Controller: • TransitionEvents • Posted when a controller changes state • Change notification events • e.g. RateChangeEvent • ControllerClosedEvents • Posted when Controller shuts down • Corresponding listener interface for each type of JMF object that can post MediaEvents

  19. JMF Event Model

  20. Controls • Mechanism for setting and querying attributes of an object • Certain objects expose Controls • e.g. often used by PlugIns to provide access to their Control objects • Examples • FrameRateControl • GainControl • Can associate listener for when volume changes

  21. Key objects in JMF • Managers • DataSources • Players • Processors • DataSinks

  22. General Managers • Intermediary objects • Enables new implementations of key interfaces • 4 main types • Manager • PlugInManager • PackageManager • CaptureDeviceManager

  23. The Manager object • Manager object used for instantiating: • DataSources, • used to deliver time-based multimedia data • Players, • used to control and render multimedia data • Processors, • used to process data and output the processed data • DataSinks, • takes a DataSource as input and renders the output to a specified destination

  24. The Manager Object

  25. Data Model in JMF • Data Sources • Media players use DataSources to manage the transfer of media-content • DataSource encapsulates location of media and the protocol used to deliver the media • Typically Identified by a: • URL, MediaLocator

  26. Capture • Capture devices represented as DataSources • e.g. microphone, video capture board, etc… • Devices can deliver multiple data streams • e.g. audio and video from a camera • e.g. multiple audio tracks from a recording session • You may then wish a single DataSource to contain multiple SourceStreams • Manager.createMergingDataSource(SourceStreams)

  27. Push and Pull Data Sources • Data sources can be categorized according to how data transfer is initiated • Pull Data-Source Client initiates the data transfer • e.g. HTTP and FILE • Push Data-Source Server initiates the data transfer • e.g. broadcast and multicast media

  28. Players • Processes an input stream and renders it at a precise time • Does not provide any control over the processing that it performs or how it renders the media data

  29. Players • Player extends the Controller interface. • Has a lifecycle • Sends media events • Player as a MediaHandler • player = Manager.createPlayer(myDataSource); player = Manager.createPlayer(myMediaLocator); player = Manager.createPlayer(myUrl);

  30. Players

  31. UI Components • Players provides access to UI Components • Player (or Processor) can provide two UI components • Visual component • Control-panel component • Can retrieve these components using methods: • getVisualComponent() • getControlPanelComponent()

  32. Player States Continued • Players post transitional events as they move between states • ControllerListener • Is the Player in an appropriate state? • Only certain methods make sense in certain states • e.g. calling getTimeBase method on an unrealized player gives an error

  33. Processors • Can also be used to present media data • Specialized type of Player that provides control over processing performed on the input media stream

  34. Processing

  35. Processor Stages

  36. Additional Processor States • Two additional stand-by states: • Configuring • Configured – can use TrackControls

  37. Processing Controls • For a given track, can control processing operations performed by the Processor by using the TrackControl for that track. TrackControl[] = processor.getTrackControls() • Can explicitly select: • Effect, Codec and Renderer plug-ins to use TrackControl[1].setCodecChain( array_of_codecs )

  38. Configuring the Processor • Consider using a processor to transcode an (audio+video) QuickTime movie – changing mpeg video track to h.263… p = Manager.createProcessor(dataSource) p.configure() p.setContentDescriptor.QUICKTIME tcs[] = p.getTrackControls() • Returns an array, e.g. 2 track controls…

  39. Configuring the Processor Format f0 = new VideoFormat(VideoFormat.h263, new Dimension(width, height), Format.NOT_SPECIFIED, Format.byteArray, (float)frameRate); l Format f1 = new AudioFormat(AudioFormat.mpeg, 8000 8, 2); tcs[0].setFormat(f0) tcs[1].setFormat(f1) p.realize() p.start()

  40. Processor Summary • A Processor does not have to output data as a DataSource, such a processor (i.e. one that renders the data) is effectively a configurable player.

  41. Data Storage and Transmission • DataSink • Used to read data from a DataSource and render the media to an output destination • Typical actions… • Write data to a file, across a network etc

  42. Using the DataSink MediaLocator dest = new MediaLocator(file://newfile.wav); dsink = Manager.createDataSink(ds, dest); dsink.addDataSinkListener(this); dsink.open(); p.start(); dsink.start(); • Wait for EndOfStream event • Close DataSink and remove Listener • dsink.close()

  43. Example • Applet Movie Player

  44. Applet Movie Player • Simple Java Applet that demonstrates how to create a simple media player with a media event listener. It will play the media clip right away and continuously loop. <!-- Sample HTML <applet code=TVApplet width=587 height=510> <param name=file value=“playme.mov"> </applet> -->

  45. Basic Steps • Initialisation… • Retrieve applet’s FILE parameter • Use this to locate media file and build URL • Create Player using the Manager object • Register applet as a ControllerListener

  46. Steps 1 & 2: Resolving a URLfor the media stream // The applet tag should contain the path to the // source media file, relative to the html page. if ((mediaFile = getParameter("FILE")) == null) Fatal("Invalid media file parameter"); try { // Create a url from the file name and the url // to the document containing this applet. if ((url = new URL(codeBase, mediaFile)) == null) Fatal("Can't build URL for " + mediaFile);

  47. Step 3: Using Manager toCreate a Player // Create an instance of a player for this media try { player = Manager.createPlayer(url); } catch (NoPlayerException e) { System.out.println(e); Fatal("Could not create player for " + url); }

  48. Step 4: Register applet as aControllerListener // Add ourselves as listener for player's events player.addControllerListener(this); } catch (MalformedURLException e) { Fatal("Invalid media file URL!"); } catch (IOException e) { Fatal("IO exception creating player for " + url);

  49. Controlling the Player… • Starting the Player public void start() { if (player != null) player.realize(); • Stopping the Player public void stop() { if (player != null) { player.stop(); player.deallocate(); } }

  50. Responding to media events • Need to Implement ControllerListener • When the Player is realized • Posts a RealizeCompleteEvent • Get the Visual component if (( visualComponent = player.getVisualComponent())!= null) { cPanel.add(visualComponent); • Get the Control Panel component • When the media has reached the end… • Posts an EndOfMediaEvent • Rewind and start over player.setMediaTime(new Time(0));

More Related