1 / 126

Knowledge and Information Retrieval

Learn about multimedia retrieval, web scraping, content representation, metadata, and more. Discover the importance of optical flow, speech recognition, and MPEG standards in multimedia.

billyrodriguez
Download Presentation

Knowledge and Information Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Knowledge and Information Retrieval Session 7 Multimedia and Web Retrieval

  2. Agenda Multimedia Retrieval Application of Multimedia Retrieval Web Crawler and Retrieval

  3. Introduction Multimedia is essentially any digital data, including plain text, mostly unstructured, that we use to communicate or capture information Most general form of the multimedia retrieval problem: The retrieval of text, image, video and sound data related to the interest of the user and their ranking according to a similarity degree For searching, user could describe a scene in video by typing “Keanu Reeves avoiding bullets in a helicopter crash in the movie The Matrix”

  4. Multimedia IR Multimedia Information Retrieval (MMIR) encompasses: Content representation and multimedia object representation such as extraction of low lever features (color, shape, texture) in images and videos. Integrated searching and browsing Techniques for searching multimedia based on their contents

  5. Metadata is not enough

  6. Web Scraping Web scraping is a technique to automatically access and extract large amounts of information from a website, which can save a huge amount of time and effort.  Important notes about web scraping: Read through the website’s Terms and Conditions to understand how you can legally use the data. Most sites prohibit you from using the data for commercial purposes. Make sure you are not downloading data at too rapid a rate because this may break the website. You may potentially be blocked from the site as well.

  7. Download the Dataset from Web import requests import urllib.request import time from bs4 import BeautifulSoup url = 'http://web.mta.info/developers/turnstile.html' # Connect to the URL response = requests.get(url) # Parse HTML and save to BeautifulSoup object soup = BeautifulSoup(response.text, "html.parser") # To download the whole data set, let's do a for loop through all a tags for i in range(36,len(soup.findAll('a'))+1): #'a' tags are for links one_a_tag = soup.findAll('a')[i] link = one_a_tag['href'] download_url = 'http://web.mta.info/developers/'+ link urllib.request.urlretrieve(download_url,'./'+link[link.find('/turnstile_')+1:]) time.sleep(1) #pause the code for a sec

  8. Histogram Histogram is a representation of the distribution of data. In the X-direction of the graph is the intensity values of the image and in the Y-direction of the image is the number of repetition of the values. We can see most of the pixel values are concentrated in brighter region for that the darker region is not clear.

  9. Optical Flow Consider a pixel I(x,y,t) in first frame. It moves by distance (dx,dy) in next frame taken after dt time. So since those pixels are the same and intensity does not change, we can say, I(x,y,t)=I(x+dx,y+dy,t+dt)

  10. Speech Recognition using Deep Learning To turn this sound wave into numbers, we just record of the height of the wave at equally-spaced point

  11. Speech Recognition using Deep Learning The input to the neural network will be 20 millisecond audio chunks. For each little audio slice, it will try to figure out the letter that corresponds the sound currently being spoken.

  12. Speech Recognition using Google Technology import speech_recognition as sr with sr.Microphone(..) as source: print "Say Something" #listens for the user's input audio = r.listen(source) try: text = r.recognize_google(audio) print "you said: " + text except sr.UnknownValueError: print("Google could not understand audio") Google Text to Speech import pyttsxengine = pyttsx.init()engine.say('Good morning.')engine.runAndWait()

  13. MPEG Standards 1. MPEG-1, standar pengompresan suara dan gambar pada Video CD termasuk MP3 format kompresi untuk suara (audio). 2. MPEG-2, standar untuk penyiaran suara dan gambar televisi digital ATSC, DVB dan ISDB, satelit televisi digital, sinyal digital cable television dan juga DVD 3. MPEG-3, standar untuk High-definition television HDTV 4. MPEG-4, pengembangan dari MPEG-1 untuk mendukung objek suara/gambar televisi tiga dimensi (3D) 5. MPEG-7, standar menggambarkan isi dari suatu multimedia (semantics of media) 6. MPEG-21, standar MPEG untuk generasi masa depan

More Related