1 / 38

US DTO VACE Phase III Meeting Washington DC October 2006

US DTO VACE Phase III Meeting Washington DC October 2006. IIT-NRC Video Recognition Systems program http://iit-iti.nrc-cnrc.gc.ca http://iit-iti.nrc-cnrc.gc.ca/about-sujet/cv-vi_e.html http://synapse.vit.iit.nrc.ca (www.perceptual-vision.com) Leader: Dr. Dmitry Gorodnichy.

sukey
Download Presentation

US DTO VACE Phase III Meeting Washington DC October 2006

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. US DTO VACE Phase III Meeting Washington DC October 2006 IIT-NRC Video Recognition Systemsprogramhttp://iit-iti.nrc-cnrc.gc.cahttp://iit-iti.nrc-cnrc.gc.ca/about-sujet/cv-vi_e.htmlhttp://synapse.vit.iit.nrc.ca (www.perceptual-vision.com)Leader: Dr. Dmitry Gorodnichy

  2. NRC-IIT Video Recognition Systemsprogram +Canadian national interests in deploying video technologiesLeader: Dr. Dmitry Gorodnichy US DTO VACE Phase III Meeting Washington DC 1/XI/2006

  3. Outline Part 1: - Who we are: NRC  IIT  CVG  VRS - VRS projects and results Part 2:- Canadian National/Regional interests in Video Technology • Government (OGD)  contacts and programs • Universities, Industry  references (in particular wrt national security) 3. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  4. National Research Council of Canada (NRC-CNRC, or NRC) • Over 30 institutes • Across Canada, Divided in clusters • Logo: “From Discovery to Innovation…” • Mandate: “Make science work for Canada”(i.e. provide solutions for OGD and industry) • Funding: government & revenue 4. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  5. NRC’s role & place HIA IBS INMS IRC BRI ICPET SIMS NINT GHI PBI INH IOT IMS IIT IIT IMB IBD IMI IFCI IMTI IAR LS PS ENG Social values driver impact Knowledge/ Discovery Technology/ Services IIT NRC Universities OGDs Industry Industry 5. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  6. wrt. Video Technologies NRC’s role & place Social values Other clients:HealthMediaEducation CBSA, RCMP DRDC, DND/CF CSIS, PPTC, CIC, PCOPSERC, TC, CATSA driver papers,conferencesCVPR, CRV impact Knowledge/ Discovery Technology/ Services CPRC - VRS program Flight Facility IIT NRC Universities OGDs Industry … Industry 6. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  7. A closer look 7. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  8. A closer look 8. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  9. A closer look IIT FF 9. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  10. Institute for Information Technology (IIT-ITI, or IIT) Make smth that works (and sales)! • Funding: Govern + Revenues • Location: Ottawa + Atlantic (e-IT) Total: $18,976M NRC-IIT generates most of its revenues through licensing its enabling technologies 10. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  11. IIT: Areas of Specialty 3D Imaging, Modeling and Visualization Data and Text Mining Computational Video Natural Language Processing Information Security, Privacy and Trust Human Computer Interaction Intelligent Internet Applications Software Engineering 11. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  12. IIT: Some technologies • From Space Shuttle (Canadarm & Canadarm2), to Hollywood, to Fine Arts, to Security applications 12. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  13. Video Recognition Systems(VRS) • Part of Computational Video Group (formed in 2001) • Our Mandate:Develop Video Recognition technologies for Canadian companies and OGD’s • Formerly called Perceptual Vision Technology • Higher mission: Computer Vision allows computers to see. Perceptual Vision allows computers to understand what they see.™ Test-bed and Criteria:If you are able to recognize him / it, why computer can’t ?.. 13. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  14. What we do • Research, coding… • Working with companies: consulting, making prototypes • Working with OGD: consulting, joint proposals • “Outreach & influence”: raising awareness about Video Recognition Technology – as opposed to VideoTechnology • public (as you see surveill. cam in the hall or watch TV), • scientists (from diff. dept: CS,Phys,NeuroBio,Psych) • government (as they allocate money) • Organizing conferences – CRV VideoRec’07 What we don’t do (vs. Universities): teaching (except specialized lectures) 14. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  15. A bit of history • First projects: Canadarm2 • Then: Nouse™, Surveilance 15. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  16. VRS now • 4 permanent Dr.’s, • 3-10 contract / guest / coop / student workers • 2 Tech.Support • Collaborate with Ottawa U. (VIVA lab) • Works closely with OGD that require Video Tech. • Lead: D. Gorodnichy • Collaborators: M. Fiala:- Markers Tracking algorithms - Panoramic Surveillance systemsG.Roth:- Feature detection, - Projective Vision algorithms 16. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  17. Application directions (And partners) Domain 1: Security, Surveillance and Monitoring.  Domain 2: Visually-enabled computer-human interaction (inc. hand-free user interfaces)  Domain 3: Visually-enhanced communication and Intelligent video processing(inc. annotation of video)  • Canada Borders Services Agency (CBSA) • Canadian Police Research Center (CPRC, RCMP) • Ottawa Health Center • U of Ottawa Music Dept. • Several Canadian/Ontario private companies 17. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  18. Research & Results Technologies developed • Neuro-associative memorization/ recognition  Face Recognition from (low-res) Video (Better than correlogram) Object learning • (Multiple) object detection and tracking  Automated tele-operating,  … • Nose (convex-shape) tracking  Hands-free interfaces • Critical Evidence Snapshot extraction  Intelligent Surveillance • Fingers, hands detection tracking Piano playing annotation 18. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  19. Research & Results Technologies developed (X,Y) name • Neuro-associative memorization/ recognition  Face Recognition from (low-res) Video (Better than correlogram) Object learning • (Multiple) object detection and tracking  Automated tele-operating,  … • Nose (convex-shape) tracking  Hands-free interfaces • Critical Evidence Snapshot extraction  Intelligent Surveillance • Fingers, hands detection tracking Piano playing annotation 19. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  20. 1. Face Recognition in Video Goal:Recognize faces from regular video quality: Bad news: low-resolution (TV:320x260) data  i.o.d = 12 pixels ! Good news: face detectors (Viola,CMU,MIT,Pitts) can detect such faces ! • So, real problem is: Different recognition modality! • So, for solution: make use of temporal domain! (“Super-resolution” comment) • Applications: (not for identification (yet), but …) for user recognition, TV annotation, soft biometrics, multiple-people tracking, multiple-camera tracking, back-tracking 20. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  21. Keys principles we adhere to (from human vision recognition system) (X,Y) “Paul” 12 pixels between the eyes should be sufficient Nominal face resolution for video-based recognition 1)Efficient visual attention mechanisms (motion etc) 2) Decision based on accumulating results over t. 3) Three main principles of neuro-processing • non-linear processing, • massively distributed collective decision making • synaptic plasticity. Allows: a) to accumulate learning data in time by adjusting synapses, b) to associate a visual stimulus to a semantic meaning based on the computed synaptic values 21. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  22. Face Recognition in Video Solution: • 12 pixel i.o.d. face model + accumulation over time while tracking (both in training and recognition) • combination of neuro-biological and computer vision approaches Tested: on TV shows, on computer user recognition 22. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  23. See also: Image and Vision Computing Journal, Special Issue on “Face Processing in Video Sequences”. (Vol. 24, No. 6). June 1, 2006. – Six-page Editorial… • On what makes video processing special • On nominal (optimal for video) face resolution • On promising directions and future trends • On video test-sets: Why not to use “Gone with the wind”? Also • CVPR’04 Workshop on Face Processing in Video (FPiV’04), Washington(www.vision-interface.net/fpiv04 ) • CRV’05 Workshop on Face Processing in Video (FPiV’05), Victoria (www.vision-interface.net/fpiv05 ) 23. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  24. 2. A.C.E. Surveillance Goals: • make surveillance data manageable, • be suitable for long-range monitoring, • be suitable for low quality video (which they often are), • be affordable (time-wise, space-wise) Challenge: foreground is not easy to detect/track in low-quality data: thru glass, blurred, against sun (24h), etc.. Solution: • Automated extraction of A.C.E. (Annotated Critical Evidence) • Integrates recent advances in video detection and tracking 24. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  25. A.C.E. Surveillance • Is currently used to monitor several premises 24/7 (long-term) • Fast/Succinctsummarizationobtained (in terms of ACE snapshots) • Little space • Browsable by timeby objectby similarity* 25. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  26. From 22:00 to 7:00with web-cam See also: CRV’06 Workshop on Video Processing for Security (VP4S-06), Quebec. (www.computer-vision.org/4security ) 26. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  27. 3. Automated Tele-operator • Replaces manual video operator: automatically switches cameras • Code taken from surveillance monitoring application(another test-bed comment on Test Data) 27. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  28. 4. Nouse™ (Nose as Mouse) PUI binary event ON x y , z a , b g OFF recognition /memorization Unknown User! monitor • Goal: To enable hands-free control • Problem: sub-pixel precision, robustness and convenience is needed. + For truly hand-free user face recognition is required. 28. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  29. 5. Pianist playing annotationWhich hand/finger played a note ? • Problem: Detect/Track hands and fingers. Multi-object, deformable object tracking is hard. • Unbiased test-bed for hands tracking – very different from sign language + pianist hands (flexible and fast) 29. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  30. Image Search for Security Apps.(Gerhard Roth) • UAV takes large number of images • Returns to same environment at a later time • Find images matching ones previously taken • City has a host of cameras at intersections • Given a single view of a vehicle • Want to find this vehicle at other intersections • Take many views of an object or vehicle • Many images of vehicle at different viewpoints • Want to find this vehicle in surveillance video 30. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  31. ARTag Fiducial Marker System (Mark Fiala) bi-tonal(only black and white) 4 corners:for 6-DOF camera/pattern pose determination Digital Methods: Error Correction, CRC-16 Checksum 31. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  32. Marker Detection Applications(Fiala, Roth) Augmented Reality • General Purpose Tracker Robot Navigation and control 32. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  33. Part 2: Interests in Video Technology from OGD Receptor capacity for Video Technology: users and developers National/Regional security programs and intentions 33. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  34. Ottawa sector and national(by interest as customers vs. R&D) Capacity/Readiness for R&D Capacity/Readiness for testing/using as customer * - expressed interest in participation in external evaluation for DTO VACE technologies 34. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  35. Companies to know • Point Grey Research (Vancouver, UBC) – firewire cams & video processing • i3DVR – “Intelligent” DVR • Deep Development Corp / Gatekeeper Systems “High Resolution for Tough Environments” – DVR (won US Force contract for Boeing Military planes) • March Networks – DVRs 35. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  36. Universities to know • UBC (Vancouver) • Ottawa U, VIVA Lab  works closely with NRC • York U (Toronto) • … (see PC for CRV VideoRec’07 workshop) (also www.vision-interface.net – for complete list) 36. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  37. 17 slides omitted for online version 37. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

  38. Summary • How can we contribute to VACE: • Science-wise: Reviewing, Testing • Client-wise: Testing • Liaisoning with OGD and national/regional/municipal surveillance programs • Consulting, Tips, Feedback, Cheering-up :) • Inviting (Making) everyone here to participate in: CRV Workshop on Video Processing and Recognition(VideoRec’07)Montreal, Canada, May 28-30 38. Video Recognition projects and interests (Dr. Dmitry Gorodnichy) DTO VACE, Phase III meeting - October 2006, Reston, USA

More Related