Industrial Vision: Image acquisition

1. Industrial Vision: Image acquisition CCD camera Digitalisation Data acquisition cards Vision software Cameras and sensors; lenses

2. Image acquisition: CCD camera

3. Image acquisition: CCD camera CCD camera CCD: Licht wordt in lading omgezet. CCD chip bevat aan elkaar geschakelde capaciteiten. Ladingen worden doorgeklokt naar buiten. ADC kan niet op de CCD chip, dus steeds een analoge uitgang. CMOS: elk beeldelement bevat 1 fotodiode en 3 transistoren: een voor lading-spanningsconversie, twee voor adressering.

4. Photon Gates Silicon Charges Potential Well Image acquisition: CCD sensor

5. 0...255 A Volts D Volts Greyvalue 0,7 8 Bit 0,348 brighter Greyvalue 127 255 Image acquisition: digitalisation

6. camera image Pixel mask 8 bit grayscale digital image Digitalisation Pixel = Picture Element

7. Digitalisation Grayscale image & numeric representation

8. Video sources The video source can be: • Video camera • Camcorder • Video recorder (VCR) • Television broadcasts • X-ray equipment • Scanning Electron Microscope (SEM) • CT scanner

9. Signal types for Image acquisition boards Composite video = signal containing bothvideo data (luminance + colour) and the timing (synchronisation) information. It is the standard which interconnects almost all video equipment (TVs, laserdisc, videorecorders, camcorders) at home. It can be transported over RCA connector or SCART (=Peritel=Euro) - connector. Examples of composite video standards: • RS-170: • used in North America and Japan • Monochrome signal • Spatial resolution: 640 pixels x 480 lines • Frequency: 60 fields/second (equivalent to 30 frames/second) • NTSC/RS-330 • used in North America and Japan • Equivalent to RS-170 but colour information is superimposed on the monochrome signal. • NTSC = National Television System Committee

10. Signal types for Image acquisition boards More composite video standards: • CCIR • used in Northern Europe • Monochrome signal • Spatial resolution: 768 pixels x 576 lines • Frequency: 50 fields/second (equivalent to 25 frames/second) • CCIR = Comité Consultatif International Radio • PAL • used in Northern Europe • Equivalent to CCIR but colour information is superimposed on the monochrome signal. • PAL = Phase Alteration Line • SECAM • used in France, Russia and the Sovjet Republic States • Equivalent to CCIR but colour information is superimposed on the monochrome signal. • SECAM = Séquencial Couleur Avec Memoire

11. Signal types for Image acquisition boards S-Video (also called Y/C video): luminance (Y) and chrominance (C) are separate signals. The Y signal contains timing (synchronisation) information. S-video can be transported over 4 pin mini DIN connector, or over SCART connector. Some image sources produce “nonstandard” video signals: • Video and timing information can vary in format as well as in single or multiple signals. They do not adhere to particular spatial resolutions, signal timing schemes, signal characteristics … Consult the documentation provided with your video source. Progressive scan (25-30 frames/sec) cameras produce non interlaced signals. All previous camera signals are analogue. DIGITAL CAMERAS: No frame grabber required! • Cameras with FireWire (IEEE 1394) interface. Supported by Apple, Windows XP • Cameras with USB interface

12. Fuga Allegro

13. Image acquisition boards • The video capture device is often called frame grabber card. • Frame grabber puts a pixel mask over the image: the card converts the analogue image (or images) supplied by a video source into a digital array (or arrays) of data points. • It is a plug in card (PCI) with AD convertor. The ADC must have video speed: 20 MHz or higher (30 or 25 video frames per second, 300 kB [640 x 480 x 8 bit] per frame. • Other features: • input multiplexer (to select one of the 4 inputs) • Colour notch filter = chrominance filter (to acquire monochrome signals from colour sources) • Programmable gain stage (to match the signal into the ADC input range) • Timing and acquisition control (to synchronise grabbing with sync pulses of incoming signal: PLL or Digital Clock Synchronisation) • Camera control stage (to send to the camera or to receive from the camera setup and control signals, e.g. horizontal and vertical sync signals, pixel clock and reset signals) • Most cards provide digital I/O for input or output operations, to communicate with external digital devices (e.g. industrial process). This saves a separate I/O board.

14. Block diagram of analog frame grabber © Data Translation

15. Image acquisition boards (continued) • Plug-in cards (image grabber, frame grabber card) for analogue cameras • Are plugged in at a VME or PCI bus • Are delivered with Windows 98 or NT drivers • Accept cameras according to the EIA (30 frames/sec) or CCIR (25) standards • Good cards have their own processor (DMA data transfer to PC) and large RAM • Others (cheaper ones) use the PC processor • They accept the signals: S video, composite video TV or VCR signals (NTSC/PAL/Secam) • Some cards have camera control output

16. Sensor types: • Line • Array Image acquisition - Cameras • Interface standards: • CCIR / RS-170 (B&W, 50-60 fields/sec.) • PAL / SECAM / NTSC (Colour) • Progressive scan (25-30 frames/sec.) • FireWire (IEEE 1394) • USB • Sensor technology: • CCD (Charge Coupled Device) • CMOS (Complementary Metal Oxide Semiconductor). A CMOS camera produces a 1000*1000 pixel image

17. Spatial resolution • The number of rows (N) from a video source generally corresponds one-to-one with lines in the video image. The number of columns, however, depends on the nature of the electronics that is used to digitize the image. Different frame grabbers for the same video camera might produce M = 384, 512, or 768 columns (pixels) per line. • a CCIR / PAL image source can result in max 768 x 576 pixel image • a RS-170 / NTSC source can result in max 640 x 480 pixel image • Depending on video source or camera used, the spatial resolution can range from 256 x 256 up to 4096 x 4096. • Most applications use only the spatial resolution required. For fast image transfer and manipulation, often 512 x 512 is used. For more accurate image processing, 1024 x 1024 is common. • The pixel aspect ratio (pixel width : pixel height) can be different from 1:1, typical 4:3. Some frame grabbers don’t convert video data into square pixels but into rectangle ones. This creates the effect of a circle appearing ovular, and squares appearing as rectangles.

18. 520 rows max CCIR Spatial resolution • Example 768 x 512 (aspect ratio 3 : 2) Brightness resolution • Brightness resolution = bit depth resolution: number of gray levels (monochrome) or number of colours • RS-170 / NTSC image: 8 bits = 256 gray levels • A standard RS-170 image is 307 kB large: 640 x 480 x 8bit. 768 pixels 512 rows 2 3

19. Interlaced / non interlaced formats • A video signal consists of a series of lines. Horizontal sync pulses separe the lines from each other. • All composite video sources (RS-170/NTSC, CCIR/PAL) and some nonstandard video sources transmit the lines in interlaced format: first the odd (first field), afterwards the even lines (second field). • Vertical sync pulses separate the fields from each other. • Some nonstandard video sources transmit the lines in non-interlaced format = progressive scan. Only one field, containing all the lines, is transmitted. • Progressive scan is recommended for fast moving images. • If one is planning to use images that have been scanned from an interlaced video source, it is important to know if the two half-images have been appropriately "shuffled" by the digitization hardware or if that should be implemented in software. Further, the analysis of moving objects requires special care with interlaced video to avoid "zigzag" edges.

20. Field 1 Field 2 Image sensors - Interlacing Interlaced image (TV)

21. Industrial Vision software • Labview • …

22. 1 1 1 -1 -1 1 0 1 -1 1 1 -1 1 0 -1 1 0 0 9 1 -1 1 0 -1 1 -1 1 -1 1 -1 0 -1 -1 -1 -1 1 Image enhancement • Neighbourhood operations • Sharpening / blurring • Edge detection Result Image

23. Image enhancement • Contract enhancement • Thresholding • Histogram equalisation • See course Digital Image Processing, R. Catthoor

24. Image acquisition: moving objects • Resolution calculation • Cd resolution = cross detection • Md resolution = moving detection • Example cd resolution: • Image width = 1900 mm • Required resolution = 0.25 mm • Number of pixels: 1900/0.25 = about 8000 • This requires minimum 2 line scan CCD cameras • Example md resolution: • Required resolution = 0.25 mm (same as cd) • Object velocity = 1600 mm/sec • This requires 1600/0.25 = 6400 lines per second, or a camera with 6.4 kHz line scan frequency.

25. Image acquisition: cameras and sensors Camera • Contains CCD or CMOS chip. It produces an analogue signal • CCD chip contains opt. sensors with bucket memory and shift register • The shift register is controlled by a pixel clock • Synchronous reset camera: the pixel clock runs independently • Asynchronous reset camera: the shift register is set externally: a picture is captures immediately after an external trigger. • A Matrix camera captures interlaced or progressive scan (=non-interlaced) images • Interlaced camera: sensor can be slower; reduced flickering • Standards: an image has a 2x3 ratio • A Linescan camera captures one single line only • Sensors producing an interlaced image are not good for moving images (comb effect) • Now there are digital cameras: The CCD is coupled to a built in ADC • Digital camera’s allow digital remote camera settings; no frame grabber card is required.

26. Image acquisition: cameras and sensors CCD or CMOS • CCD chip bestaat uit lichtsensoren in de vorm van een emmertjes geheugen, en een shift register. Dit laatste wordt gecontroleerd door een pixel clock • Het licht wordt in lading omgezet. CCD chip bevat aan elkaar geschakelde capaciteiten. Ladingen worden door deze capaciteiten naar buiten geklokt. Is er 1 pixel stuk, dan is er geen informatie over de volledige lijn die erop volgt. • Een ADC kan niet op de CCD chip, dus steeds een analoge uitgang. • CMOS: elk beeldelement bevat 1 fotodiode en 3 transistoren: één voor lading-spanningsconversie, twee voor adressering. Bij CCD is de ganse cel beschikbaar voor belichting, dus 10x meer gevoelig dan CMOS. De donkerstroom is 10x hoger bij CMOS. • Elke lichtgevoelige cel is individueel uitleesbaar. Is er 1 pixel stuk, dan bijft de rest van de lijn intact. • Dynamisch bereik is hoger. Bij CCD is er bij overbelichting een “overlopen” van ladingen. • CMOS is sneller uit te lezen.

27. CCD CMOS Image sensors - CCD versus CMOS

28. Optics • Choices: • Magnification • Depth of field • Distance to object • Distance to image sensor • Light intensity • Optical resolution • Telecentric • Artefacts: • Added size • Added weight • Image distortion

29. Image acquisition: cameras and sensors Lens • Choose a lens with right focus distance • A “telecentric objective” sees only the surface plane of a prismatic object Price • B/W Camera: 500 EUR • Frame Grabber card with software: 300 EUR