1 / 20

Contactpersonen: luc.mertens@uantwerpen.be gunther.steenackers@uantwerpen.be rudi.penne@uantwerpen.be Website: https://w

TETRA-project: SMART DATA CLOUDS (2014 – 2016) . Flemish Agency for Innovation by Science and Technology. Contact persons: luc.mertens@uantwerpen.be gunther.steenackers@uantwerpen.be rudi.penne@uantwerpen.be Website : https://www.uantwerpen.be/op3mech/.

bart
Download Presentation

Contactpersonen: luc.mertens@uantwerpen.be gunther.steenackers@uantwerpen.be rudi.penne@uantwerpen.be Website: https://w

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TETRA-project:SMART DATA CLOUDS (2014 – 2016) .Flemish Agency for Innovation by Science and Technology Contact persons:luc.mertens@uantwerpen.be gunther.steenackers@uantwerpen.be rudi.penne@uantwerpen.be Website: https://www.uantwerpen.be/op3mech/ Contactpersonen:luc.mertens@uantwerpen.be gunther.steenackers@uantwerpen.be rudi.penne@uantwerpen.be Website:https://www.uantwerpen.be/op3mech/

  2. TETRA: Smart Data Clouds (2014-2016): industrialcases a. Security, track and trace [Haven van Antwerpen]. b. Traffic control and classifications [Macq, LMS, Melexis] c. (Smart) Navigation [Wheelshairs, PIAM, mo-Vis, Automotive LMS] d. Dynamic 3D body scan [RSscan, ICRealisations, ODOS] Data Fusion: ind. Vision linkedwith CAE. Data from RGB, IR, ToF and HS cameras.

  3. TETRA: Smart Data Clouds (2014-2016): healthcareapplicationsa. Wheelchair control - navigation: [mo-Vis, PIAM, Absolid] b. Gesture & Body-analysis: [RSScan, ICrealisation, Phaer, ODOS] ‘Data Fusion’ usableforfuture Healthcare .

  4. Time of Flight: cameratypes: 03D2xx-camerasPMD[Vision]CamCube 3.0 FotonicRGB_C-SeriesIFM-Electronics352 x 288 pix! 160 x 120 pixel 64x50 pix. Recent: pmdPhotonICs® 19k-S3 Previous models: P – E series. Melexis: EVK75301 80x60 pix. NearFuture: MLX75023 automotive QVGA ToF sensor . Swiss Ranger4500 MESA176 x 144 pix. DepthSense325( 320 x 240 pix ) BV-ToF 128x120pix. ODOS: 1024x1248pix. Real.iZ-1K vision system Swiss Ranger SR4000 (MESA) Optrima OptriCam ( 176 x 144 pix. ) ( 160 x 120 pix. )

  5. ToF VISION: world to image, image to world - conversion P = [ tg(φ) tg(ψ) 1 d/D ] , P’ = [ tg(φ) tg(ψ) 1 ] . ( f can be chosen to be the unit.) 1 j 1 J/2 i j0 J φ i0 R ψ horizon I/2 (0,0,f) A u.ku f = 1 D kr ,kc r v.kv N d z I D/d = x/uk = y/vk = z/f u = j–j0; uk = ku*u v = i – i0; vk = kv*v tg φ = uk/f tg ψ = vk/f r = √(uk²+f²) d = √(uk²+vk²+f²) x y Everyworld point isuniquewith respect toa lot important coordinates: x, y, z, Nx, Ny, Nz, kr, kc, R, G, B, NR , NG , NB , t° , t ... The basis of our TETRA-project: ‘Smart Data Clouds’

  6. D/d-related calculations. (1) For navigation purposes,the free floor area can easily be found from: di/Di= e/E = [ f.sin(a0) + vi.cos(a0) ] / E = [ tg(a0) + tg(ψi) ].f.cos(a0)/E .Since (d/D)i = f /zithis is equivalent with: zi . [ tg(a0) + tg(ψi) ] = E/cos(a0) . Camera sensor Camera inclination = a0 . Camera bounded parallel to the floor. f a0 e di ψi zi E vi Di Floor.

  7. D/d-related calculations. (2) Fast calculations !! 1 D The worldnormalvector n , at a random image position (v,u) . 4 3 d n v d² = u² + v² + f². 2 u f O nx ~ f.(D4/d4 - D3/d3)/(D4/d4 + D3/d3) = f.(z4 – z3)/(z4 + z3) ny ~ f.(D2/d2 - D1/d1)/(D2/d2 + D1/d1)= f.(z2 – z1)/(z2 + z1) nz~ - (u.nx + v.ny + 1 ) ▪

  8. Coordinate transformations yPc • Camera x // World x // Robot x • World (yw, zw) = Robot (yr, zr) + ty • yPw = - (z0 – zPc).sin(a) + yPc.cos(a) = yPr+ ty • zPw = (z0 – zPc).cos(a) + yPc.sin(a) = zPr f vt a zw P zPc A camera re-calibration for ToF cameras is easy andstraightforward !! zr a yw Work plane = reference. yr ty

  9. Pepper handling First image = empthy world plane Next images = random pepper collections. Connected peppers can bedistinguished by means of local gradients. Gradients can easilybe derived from D/d-ratios. Thickness in millimeter Calculations are ‘distance’ driven, x, y and z aren’t necessary. Fast calculations !

  10. YouTube: KdGiVL Bin picking & 3D-OCR Analyse ‘blobs’ one by one. Find the centre of gravity XYZ, the normal direction components Nx, Ny, Nz ,the so called ‘Tool Centre Point’ and the WPS coordinates.

  11. x y Beer barrel inspection. IDS uEye UI-1240SE-C O3D2xx MESA SR4000 x y z ToF - RGB correspondency vc,P/F – kv.vt/f = tx.√(z²P+y²P) uc/F = ku.ut/F .

  12. RGBd Packed… …with a plastic warp • DepthSense 311 • Find the z –discontinuity • Look for vertical and forward oriented regions • Check the collineraity • Use geometrical laws in order to find x, y, z and b.

  13. ToF Packed …with a foil IFM O3D2xx. 1. Remove weak defined pixels. 2. Find the z –discontinuity 3. Look for vertical and forward oriented regions 4. Check the collineraity 5. Use geometrical laws in order to find x, y, z and b. CamBoard Nano

  14. Basic tasks of ToF cameras in order to support Healthcare Applications: • Guide an autonomous wheelchair along the wall of a corridor. • Avoid collisions between an AGV and unexpected objects. • Give warnings about obstacles (‘mind the step’…, ‘kids’, stairs…) • Take position in front of a table, a desk or a TV screen. • Drive over a ramp at a front door or the door leading to the garden. • Drive backwards based on a ToF camera. • Automatic parking of a wheelchair (e.g. battery load). • Command a wheelchair to reach a given position. • Guide a patient in a semi automatic bad room. • Support the supervision over elderly people. • Fall detection of (elderly) people.

  15. Imagine a wheelchair user likes to reach 8 typical locations at home: 1. Sitting on the table. 2. Watching TV. 3. Looking through the window. 4. Working on a PC. 5. Reaching the corridor 6. Command to park the wheel- chair. 7. Finding some books on a shelf. 8. Reaching the kitchen and the garden.

  16. ToF guided navigation forAGV’s. D2 = DP ? Instant Eigen Motion: translation. Random points P! (In contrast: Stereo Vision must find edges, so texture is pre-assumed )

  17. ToF guided Navigation of AGV’s. D2 = DP ? Instant Eigen Motion: planar rotation. Image data: tg(β1) = u1/f ; tg(β2) = u2/f . Task: find the correspondence β1 β2 ; Procedure: With 0 < |α| < α0 With |x| < x0 Projection rules for a random point P : Next sensor position P z2 Here: x < 0 R > 0 . D2 D1 DP z1 β2 β1 Previous sensor position α R+ δ1 α D2² = xP2² + zP2² x Parallel processing possible!

  18. ToF guidednavigation of AGV’s. Instant Eigen Motion: planarrotation. e.g. Make use of 100 Random points. Result = Radius & Angle D2,i = DP,i ?

  19. Research FTIMaster Electromechanics Info: luc.mertens@uantwerpen.be ; gunther.steenackers@uantwerpen.be Research: ToF drivenQuadrocopters Combinations with IR/RGB Security flights over industrialareas. TETRA-project 2014-2016 ‘Smart Data Clouds’

  20. Quadrocopter navigation based on ToF cameras The target is to ‘hover’ above the end point of a black line. If ‘yaw’ is present it should be compensated by an overall torque moment.Meanwhile the translation t can be evaluated. The global effect of the roll and pitch angles repre-sent themselves by means of the points P and Q. The actual copter speed v is in the direction PQ. At the end P and Q need to come together without oscillations, while |t| becomes oriented in the y-direction. An efficient path can be followed up by means of Fuzzy Logic principals. t BV-ToF 128x120pix.

More Related