An automated image prescreening tool for a printer qualification process

1. An automated image prescreening tool for a printer qualification process • by • †Du-Yong Ng and ‡Jan P. Allebach • † Lexmark International Inc. • ‡ School of Electrical and Computer Engineering, Purdue University

2. Synopsis • Anatomy of a formatter-based EP laser printer • Overview of a printer qualification process • Motivation • Examples of artifacts • Image fidelity metrics – prior work • Prescreening tool • Experiment • Results • Conclusion

3. Anatomy of a formatter-based EP laser printer A standalone network device Print Engine Formatter Control Panel

4. Overview of a printer qualification process • Phase I • The formatter (hardware + firmware) and the print engine are developed in parallel. • The hardware portion of the formatter is not ready. • The firmware of formatter is tested using a simulator . Test image (PDL) Firmware of an earlier product + simulator Newly developed firmware + simulator Digital outputs match qualitatively? Master Current

5. Overview of a printer qualification process (cont.) • Phase II • Preproduction print engines are either scarce or not available yet. • Preproduction formatter hardware is available. • The formatter (firmware + hardware) is tested with a print engine emulator. Test image (PDL) Formatter of an earlier product + print engine emulator Newly developedformatter + print engine emulator Digital outputs match qualitatively? Master Current

6. Overview of a printer qualification process (cont.) • Phase III • Preproduction printers (formatter + print engine) are available. Test image (PDL) Earlier printer model Newly developedprinter Hardcopies match qualitatively? Master Current

7. Overview of a printer qualification process - summary Simulator without actual formatter / emulator with actual formatter Softcopy current image Debug Softcopy Formatter/ firmware development teams Test image (PDL) Error flagged masters Comparison Hardcopy Debug Actual formatter Print engine Hardcopy current image

8. Motivations • Image screening (comparison of master-current image pairs) • is mostly performed by trained observers. • is needed for thousands of softcopy and hardcopy image pairs throughout the printer development process. • is very labor intensive. • is often performed only on a fraction of test suites before the product is rolled out due to • a relatively short development time. • a large number of test images in the test suites. • Our goal • is to develop a automated tool to reduce the workload of trained observers and increase the volume of tests the by screening out softcopy image pairs with • highly objectionable errors (failed). • visually insignificant errors (passed). • Trained observers only need to focus on image pairs which could not be screened out by the tool (further evaluation)

9. Examples of master-current image pair Current • Different halftone algorithms Master • Missing pixels Master Current

10. Examples of master-current image pair (cont.) Master • Different in character size Current • Sporadic difference Master Current

11. Image fidelity metrics – prior work • First category • Examples • Peak-signal-to-noise ratio (PSNR), root mean square error, and ∆Ea*b* • Pros • They are fast and easy to compute, and produce a single number. • Cons • Averaging effect destroys local information and spatial interaction of pixels is ignored. • Second category • Examples • Wu’s color image fidelity assessor (HVS model) and structural similarity image metric (first and second order statistics the luminance channel of a local window) • Pros • Spatial processing model is included. • Cons • The algorithm is computational expensive and does not produce a single number (Wu’s color fidelity assessor) • SSIM produces a single number but suffers from the averaging effect.

12. Prescreening tool - requirements • needs to work reasonably fast. • classifies the master-current pairs into one of the categories: ‘passed’, ‘failed’, and ‘further evaluation required’. • must adapt automatically as the spatial resolution of the images changes in dpi. • has to be capable of processing any image content. • needs to handle both halftone and continuous-tone images. • will process full color, indexed color, grayscale, and bilevel images consistently. • will detect and ignore small spatial shifts in content and differences in orientation

13. 13 The prescreening tool Master image Current image Preprocessing Compute 2D error map Cluster error map Compute error metric Thresholding FAILED PASSED FURTHER EVALUATION

14. 14 Prescreening tool - preprocessing Check orientation Check dimension Rotated CCW Rotated CW Content size Image size Master Current Current Determine shift in content Other processing • Determine image type • Perform quick diff • Determine resolution • Correct orientation • Correct spatial shift Master Current

15. Prescreening tool – compute and cluster 2D error map (illustration) Master M i i Binary Error Map j j Clustering Clustered Error Map i Current C i kth cluster j (k+2)th cluster j (k+1)th cluster

16. Prescreening tool – compute error metric (contrast sensitivity function CSF component) Master i j correspond to the lth pixel of the kth cluster Current i j

17. Prescreening tool – compute error metric (contrast sensitivity function CSF component) (cont.) • Average filtered pixel value for the kth cluster • Master : • Current: • Error for the kth cluster: • CSF error component for the image pair: To CIE L*a*b*

18. Prescreening tool – compute error metric (visual acuity filter VAF component) Master i j correspond to the lth pixel of the kth cluster Current i j

19. Prescreening tool – compute error metric (visual acuity filter VAF component)(cont.) • Average filtered pixel value for the kth cluster • Master : • Current: • Error for the kth cluster: • VAF error component for the image pair: To CIE L*a*b*

20. Prescreening tool – compute error metric (overall error) • Combined error • Error metric value for the image pair

21. 21 Experiment • There are 147 image pairs. • Test image pairs are of • 300, 600 and 1200 dpi. • binary, grayscale, color bitmap and full color. • Six expert observers classify the image pairs into 3 categories • Acceptable (passed). • Need further evaluation. • Objectionable (failed). • These image pairs are processed with our prescreening tool. • The peak signal to noise ratio (PSNR) metric and structural similarity image metric (SSIM) are also computed for each preprocessed (to ensure fair comparison) image pair.

22. 22 Results – error metric • A zero error metric value indicates a perfect match. • Decision thresholds exist to pass and fail image pairs. • The prescreening tool is able to screen out as many as 35% of the image pairs tested.

23. Results - PSNR • A larger PSNR value (PSNR = ∞ for a perfect match) indicates a closer match. • Only decision thresholds to pass image pairs exist. • The PSNR metric is able to screen out only as many as 4% of the image pairs.

24. 24 Results - SSIM • A larger SSIM value (SSIM = 1for a perfect match) indicates a closer match. • Only decision thresholds to fail image pairs exist. • The SSIM metric is able to screen out only as many as 3.4% of the image pairs.

25. 25 Conclusion • We have successfully developed an automated prescreening tool along with an image fidelity metric for a printer qualification process. • This tool works for a wide range of image types, content, and image resolutions. • It requires no training and it is able to reduce the workload of expert observers by a substantial amount.