Content Adaptation Framework: Bringing the Internet to Information Appliances

1. Content Adaptation Framework:Bringing the Internet to InformationAppliances Content-Based Transcoding of Images in the Internet IBM T.J Watson Research Center J.R Smith,R Mohan and Chung-Sheng Li Lin-Kai Chen VClab

2. Introduction • This paper present Content Adaptation Framework for customizing internet content for diverse client devices. • Converting content items into alternate modalities and resolutions so as to meet the characteristics of the client devices. • Different modalities: video,image,text,audio. • Different fidelities: reduced resolution and summaries. Lin-Kai Chen VClab

3. Internet Universal Access Via Content Adaptation On the road Information Content In the office Lin-Kai Chen VClab

4. Introduction(Cont.) • The content can be adapted on the fly or transcoded in advance. • This content adaptation can be performed at server client or proxy. Lin-Kai Chen VClab

5. Content adaptation Framework Lin-Kai Chen VClab

6. Content adaptation Framework(Cont.) • The components are: • Analysis：To extract meta-data, such as resource requirements and the type and purpose of the content, used in guiding subsequent transcoding and selection process. • Transcoding：Based on the capabilities of client devices,different transcoding modules are employed to generate versions of the content in different resolutions and modalities. Lin-Kai Chen VClab

7. Content adaptation Framework(Cont.) • Selection:Selects the resolutions or modalities that best meet the client capabilities. • Rendering:The selected content is then rendered in a suitable delivery format(eg.HTML). • Description:The InfoPyramid is used to represent the multiple resolutions and modalities of the transcoded content. Lin-Kai Chen VClab

8. Lin-Kai Chen VClab

9. Content Analysis • Resource requirement • Content size in bits. • Display size:height,width,and area. • Streaming bit-rate. • Color requirements. • Software requirements:support for specific codecs. • Hardware requirements. Lin-Kai Chen VClab

10. Content Analysis • Content type and purpose • The type of content decides which specific transcoding algorithm are applied. • One focus of the content analysis is images as they currently account for the largest amount of web traffic in terms of bits. • Analyzing images to determine their type and purpose. Lin-Kai Chen VClab

11. Image content analysis Related text Purpose class Image type X[m.n] Decision tree Xr, Xg, Xb Features Real time trans. Parameters Pv, Vv μh, μv μs Training set Color B/W Gray B/W graphicphoto Lin-Kai Chen VClab

12. Image Content Analysis • Image type classes: • BWG : b/w graphic. • BWP : b/w photo. • GRG : gray graphic. • GRP : gray photo • SCG : simple color graphic • CCG : complex color graphic • CP : color photo. Lin-Kai Chen VClab

13. Image Content Analysis • Image purpose classes • ADV: advertisement, ie. Banner ads. • DEC: decoration, ie. Background textures. • BUL: bullets, points,dots. • RUL: rules,lines,separators. • MAP: maps, ie,images with click focus. • INF: information:icons,logos. • NAV: navigation, ie,arrows • CON: content related: news photos. Lin-Kai Chen VClab

14. Image Content Analysis • Image type classification • Decision tree as a classifier. • It consists of 5 decision points, each of which utilizes a set of features extracted from the images. • Classification parameters from the training set of 1282 images. Lin-Kai Chen VClab

15. Lin-Kai Chen VClab

16. Color vs. non-color • Each image X[m,n] has xr, xg, xb. • Using the measurement of the mean saturation per pixel μs. The saturation channel ys . ys = max(xr, xg, xb) – min(xr, xg, xb) • μs discriminates well between color and non-color images. Lin-Kai Chen VClab

17. B/W vs. Gray • Intensity channel • Intensity entropy • ,where Lin-Kai Chen VClab

18. B/W vs. Gray • Intensity variance Lin-Kai Chen VClab

19. BWG vs. BWP • It uses the minimum of the mean number of intensity switches in horizontal and vertical scans of images. Lin-Kai Chen VClab

20. GRG vs. GRP • Using the intensity switch measure Wv and intensity entropy Pv . • As to switch measure, it has a much lower value for b/w graphics. Lin-Kai Chen VClab

21. SCG vs. CCG vs. CP • The images are transformed to HSV and vector quantized. • The process generates a 166-HSV color representation of the image y166 ,where each pixel refers to an index in the HSV color look-up table. • The mean color switch per pixel Wv • Color entropy P166 Lin-Kai Chen VClab

22. Lin-Kai Chen VClab

23. Lin-Kai Chen VClab

24. Image purpose classification • Image purpose is detected by applying a rule induction engine on the image type,document context and related text. • Context for images. • BAK: background, ie <body background=…..> • INL: inline,ie <img src=…> • ISM: is map • REF: referenced , ie <a href=…> • LIN: linked, ie <a href=….><img src=…></a> Lin-Kai Chen VClab

25. Image purpose classification • Text terms:extracted from the ‘alt’ tag, the image URL address strings,and the text nearby the images in the web documents. • D={‘ad’,’texture’,’bullet’,’map’,’logo’,’icon’} Lin-Kai Chen VClab

26. Client device characteristics Lin-Kai Chen VClab

27. Transcoding policy • Transcoding functions • Size:minify,crop,and subsample. • Fidelity:JPEG,GIF compress, quantize, reduce resolution, smooth,sharpen…etc. • Color content: reduce color, convert to gray,b/w. • Substitution: attributes, text,type,purpose, and remove image. Lin-Kai Chen VClab

28. Transcoding policy Lin-Kai Chen VClab

29. Transcoding policy • For compression, JPEG works well for gray photo graphics but not for graphics.For GIF, the reverse is true. • The transcoding policies also make use of the image purpose analysis. Lin-Kai Chen VClab

30. Conclusion • The content adaptation can be performed at various nodes in the internet:the server,proxy and client. • They proposed the content discription scheme based on the InfoPyramid, to the MPEG-7 and W3C standards bodies. • The system utilizes transcoding policies based on the content classes to transcode and adapt images. Lin-Kai Chen VClab