1 / 24

NEESGrid Data Turbine Video as Data

NEESGrid Data Turbine Video as Data. These notes are based on a lot of work from people from the NEES sites and the NEESGrid SI Data/Deployment team. Comments to Charles Severance (csev@umich.edu). Overview.

denton
Download Presentation

NEESGrid Data Turbine Video as Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NEESGrid Data Turbine Video as Data These notes are based on a lot of work from people from the NEES sites and the NEESGrid SI Data/Deployment team. Comments to Charles Severance (csev@umich.edu)

  2. Overview • This is a design document to build common understanding and consensus - it will change over time • These new requirements and designs come out of the Experiment Based Deployment efforts at Minnesota and UC Davis • They represent significant new work in the area of data, video, images, and telepresense • The work is being undertaken jointly by the SI team and the sites involved as part of the EBD • Much of this is triggered by the availability of Data Turbine and making use of its features and rethinking how we approach solving problems with Data Turbine as a framework

  3. Capturing Still Capture PTZ/ USB DT Client rbnbjcap BT848 DT Client Audio Encoder Audio DT Client Data Capture DAQ DT Client Each still capture produces two channels - Small 1-5fps stream + large single images when picture is taken Camera Control Control Plugin DT Main System NEES NSDS Driver Still Capture - Minnesota / Paul Hubbard Video capture - From Creare Audio capture - From Creare (TBD) Data Capture - From sites (upwards compatible) NEES NSDS Driver - Paul Hubbard Camera Control Plugin - UM / Minnesota

  4. User Views / Still Camera Still Image / Camera Control ^ < > ^ ~ < > Thumbnail + Audio + Data Data Viewer < > + Control Plugin Thumbnail - UBuffalo / Umichigan Thumbnail viewer - Creare v1 / Michigan v2 Camera Control Applet - Minnesota / Michigan Quicktime Slicing tool - Michigan (lower priority) Stored Data Viewer (existing) - Michigan JPEG Viewer - Creare Quickime Viewer - Apple DT Main System Thumbnail Process Quicktime Storage System Quicktime Slicing Tool

  5. Robotic Control Robotic Control Applet ^ < > ^ DT Client Video Frames BT848 DT System Control Plugin Robotic Control Robotic Control: UCDavis RBNB Control Plugin: Michigan

  6. RBNB to Streaming Quicktime Macintosh(es) DT Main System QT -> MP4 Encoding DT Client QT Encoding - Michigan QT Viewer - Apple :) Quicktime Storage System Quicktime Streaming System Quicktime Streaming Server This effort is lower priority than the other efforts Live Viewers

  7. RBNB and the Axis Format DT Main System Inbound Axis DT Client Outbound Axis DT Client Inbound Axis - Written by ORST Outbound Axis - Written by ORST and uBuffalo (separate versions) Texas Telepresense - Chris Stanton NEES Telepresense - ANL This should interoperate with the standard Creare viewer NEES / ANL Telepresense This effort is lower priority than the basic video as data and still image capture, but is not expected to be too difficult. Texas Telepresense Web Browser

  8. RBNB and RTP Video rbnbjcap BT848 DT Client Audio Encoder Audio DT Client DT Main System Outbound RTP DT Client Access Grid JMStudio RTP Viewer This effort has very low priority and the design is included for architectural completeness and to illustrate possibilities.

  9. Software: JPEG Frame Capture Retrieves and compresses frames into JPEG and places them in a designated DT channel This is part of Creare distribution

  10. Software: Audio Encoder Creare had this in the previous release based on a modified version of JMStudio but in the next release they rewrote the code as the JPEG encoder and did not port the audio code We are working with Creare to get this feature added to the JPEG Capture Written in JAVA on XP

  11. Software: Still Image Capture This will send out a 320x240 thumbnail channel of JPEGS at a rate of 1-5 per second (or as quickly as we can send the thumbnails) When high resolution pictures are taken, they are also placed in a Data Turbine channel as JPEGs The control of the camera will be done using NTCP with a control gateway which is accessed through a Data Turbine plug-in. We are researching which camera to use and whether or not to use the PTZ (standard) interface to the camera or to use a proprietary interface - as of now, it looks as though we will use the a proprietary interface as it is the only way to get both streaming low-res for control feedback and still take high-res at the same time. Written in: ??? Using the SDK on XP ??? This will be developed by Minnesota and ANL The camera control will be jointly developed by Michigan and Minnesota

  12. Software: Data Capture This will develop a Data Turbine data injector which will comply to the DAQ -> NSDS Driver Protocol so existing DAQ code can communicate (and have the data injected into DT) with no changes to existing DAQ code. Responsibility: ANL

  13. Software: Robotic Control This is a very specialized robotic control system which is being co-developed with the custom robotic system It will use an NTCP control protocol for control Responsibility: UC Davis (Stephanie)

  14. Software: QT - MPEG 4 Encoding This software will subscribe to the JPEG video channels and the audio channel from Data Turbine and compress these into MPEG-4 The MP4 data will be both sent to the QuickTime Streaming Server and into permanent storage at 640x480 Users may see 8-10 second lag from real-time when watching using Quicktime Streaming Written in: Java on either Mac OS/X or XP This is lower priority than stills and JPEG streaming Responsibility: Michigan (Chuck)

  15. Software: Control Gateway This software will act as a gateway between the applets which are providing the control interface and the internal control protocol (NTCP) This software will also be responsible for enforcing security so that only the properly authorized users will be able to perform the control operations. Written in: JAVA on Linux Responsibility: Michigan (Terry)

  16. Software: Still Image/Camera Control This software will provide the user interface for the control of the camera including pan, tilt, zoom, and “take picture” This software will subscribe to the thumbnail channel for the camera and display the images to allow the operator to get some feedback as they move the camera. Applet written in JAVA with some support in CHEF Responsibility: Michigan (Lars) and Minnesota (Chen)

  17. Software: Thumbnail + Audio + Data Viewer This software allows the simultaneous display of an audio channel and some number of data and thumbnail channels. Thumbnail channels can either be still camera or video or video camera video. The user can launch the still image viewer from one of the thumbnails. The user can also launch the stored data viewer for a portion of the time scale with one or more of the video cameras and data channels. This software only displays sensor data in a numeric format. This software can view live data or view previous data.

  18. Software: Thumbnail + Audio + Data (cont) The timeline can be scrubbed forwards and backwards. This will depend heavily on Data Turbine’s capability to move forward and backwards through data while maintaining data synchronization - while scrubbing, the images may not flip instantly - but within a second after letting go of the scrubber, the data and images should appear. The user can request a subset of the timeline to be extracted and viewed using the stored viewer - this may take a minute or so. Applet written in JAVA with some support in CHEF Responsibility: Michigan (Lars, Chuck)

  19. Software: Data Viewer This is an updated version of the current Stored data viewer Applet written in JAVA with some support in CHEF Responsibility: Michigan (Lars)

  20. Software: Robotic Control Applet This software will control the UCDavis Robot It will make use of the functionality used in the Camera Control applet Applet written in JAVA with some support in CHEF Responsibility: UC Davis (Stephanie) Michigan (Terry)

  21. Software: Launch Desktop Application This is a relatively simple new feature where we use the repository capability of NEESgrid to map data out of the repository and launch it with a particular (configurable) MIME type. This will allow direct launching of the UCDavis 3D viewer from information stored in the repository. Responsibility: Michigan (Chuck) UC Davis

  22. Other Important Details Streaming DT channels (audio, video, data) into the repository Metadata issues / project browser issues for automatic configuration and access control

  23. Miscellaneous Michigan is writing a “fake video injector” so we can develop applications without needing a real live video source

  24. Conclusion • There is a lot of work here • The designs have and will continue to change as we gain experience with Data Turbine

More Related