Snakes : Active Contours

1. Snakes: ActiveContours

2. Introduction

3. Parametricactivecontours • A traditional snakeis a curve:that moves through the spatial domain of an image to minimize the energy functional: • The functional can be viewed as a representation of the energy of the contour and the final shape of the contour corresponds to the minimum of this energy.

4. Internalenergy (1) • The first term of the functional: is the potential energy due to stretching of the flexible contour, and is generalization of Hooke’s law, in which potential energy is proportional to square tension. The second term represents the potential energy due to bending. • The weighting parameters and control the snake’s tension and rigidity. They determine the extent to which the snake can stretch or bend at any point on the snake.

5. Internalenergy (2) • As is decreased, the strength of the elasticity of the contour at a node is decreased, giving nodes greater freedom in movement thus possibly producing more irregular contours. • As is decreased, the stiffness of the contour at a node is decreased, allowing sharper corners to develop. • Usually any change in the elasticity coefficient should typically be accompanied by a similar change in the rigidity coefficient.

6. Externalenergy (1) • The secondterm in (*), isthe externaldeformationenergy and representsthe effectofforcesthat are notintrinsicpropertiesof the contour, butwhichare imagedependent:wheredenotes a scalar potentialfunctiondefined on the imageplane. • The contourcomesto a restwhen the net effectof the damping, external and internalforcesreaches zero. The externalforceisdesignedto pull anactivecontourtowardsobjectboundaries or otherfeaturesof interest.

7. Externalenergy (2) • Given a gray-levelimage , viewedas a functionofcontinousvariables , typicalexternalenergiesdesignedtoleadcontourtowardstepedges are [Kasset al.]:whereis the gradientoperator, is the convolutionoperator, is a two-dimensionalGaussianfunctionwith a standard deviation , and is a positive weightingparameter. • The first oftheseexternalenergyfieldshas minima at placesof high imagegradients (e.g. edges). The secondhas minima at locationswhere a Gaussianfilteredimagehasmaxima, and is more robust in the presenceofnoise.

8. Externalenergy (3) • If the imageis a linedrawing (black and white), then appropriate externalenergies include: • It can beseenfromthesedefintionsthatlarger ‘s will cause the boundariestobecomeblurry. Suchlarge ‘s are oftennecessary, however, in ordertoincrease the capturerangeofactivecontours.

9. Minimizing the energyfunctional (1) • Withenergydefinedas:and an appropriate choiceof the energy , itremainstofind the functionthatminimized the functional. • In accordancewith the calculusofvariation, the snakethatminimizes the energyfunctionalmustsatisfy the Euler-Lagrangeequation: which can beviewedas a forcebalanceequation

10. Minimizing the energyfunctional (2) • Whileitisnaturaltoviewenergyminimizationas a staticproblem, a potentapproachtofind a solutionto (**) istoconstruct a dynamical system thatisgovernedby the functional and allows the system to evolve toequilibrium: • When the solutionstabilizes, the termvanishes and weachieve a solutionof (**). A numericalsolution can befoundbydiscretizing the equation and solving the discrete system iteratively.

11. Behaviouroftraditionalsnake (1) • A difficultyof the formulationgivenby (***) isthat in generalthereexistmanyconfigurationsfor the snakethat are localenergy minima, butthat do notprovide the boundaryof interest tous. • As a generalrule, the algorithmwillprovide a convergentsolutionthatiscloseto the initialsnakeconfiguration, hence, the accuracyof the methodishighly sensitive toinitialization. Thisis due to the factthat a gradient-basedexternalenergyfieldisquitelocallydefined.

12. Behaviouroftraditionalsnake (2) • In the Figure above, although the externalforcescorrectlypointtowards the objectboundary, within the boundary the forcespointhorizontally in oppositedirections. Therefore, the activecontourispulledaparttowardeachof the “fingers” of the U-shape, butdoesnot progress downwardinto the concavity. Thereis no choiceof and thatwillcorrectthisproblem. • Also, weseethat the magnitudeof the externalforcesdies out rapidlyawayfrom the objectboundary. Increasingwillincreasethisrange, but the boundarylocalizationwillbecomeless accurate and distinct, ultimatelyobliterating the concavityitselfwhenbecomestoolarge.

13. Gradientvector flow (1) • In responsetotheseproblems, the gradientvector flow (GVF) fieldwasoriginallyintroducedtoimprove some poorpropertiesof the externalforcefieldgeneratedby the gradientoperator. • A newstaticexternalforcefieldisdefinedas , whichis just a gradientvector flow fieldprovidingbetter long rangeattraction, whilepreservingsharplocal minima in the energyfield. • Toobtain the correspondingdynamicequation, the potentialforce in (***) isreplacedwith , yielding:

14. Gradientvector flow (2) • The idea isbased on the Helmholtztheoremwhichstatesthat the mostgeneralstaticfield can bedecomposedintotwocomponents: anirrotational (curl-free) and a solenoidal (divergence-free) component. • The GVF fieldisdefined so that , minimizes the energyfunctional: whereis the edgemapobtainedfrom the image(forexample: ).

15. Anisotropicdiffusion • An improvementof the classicalimagedenoisingtechniquesbased on the Gaussianfiltering can beobtainedbymeansof a nonlinearfilterknownas the anisotropicdiffusionfilterbased on a multiscaleedge detection scheme. • The methodconsiders the imageintensityas a fluidconcentration: • The smootingkerneliscontrolledby a diffusioncoefficient, . Differentfunctions can beusedfor the choiceof , forexample:whereis a positive constant; ifis low, smallintensitygradients are ableto block conduction and hencediffusionacrosssteepedges.

16. Exampleofanisotropicdiffusion (1)

17. Exampleofanisotropicdiffusion (2)

18. Minimizing the GVF functional • It can beshownthat the GVF field can befoundbysolving the followingEulerequations:whereis the Laplace operator. • Theseequations can besolvedbytreating and asfunctionsoftime, and solving: • Theseequations are knownasgeneralizeddiffusionequations.

19. Behaviourof the GVF snake • Comparing the GVF fieldto the traditionalpotentialforcefieldit can beobservedthat the GVF fieldhas a muchlargercapturerangethantraditionalpotentialforces. • A secondobservationisthat the GVF vectorswithin the boundaryconcavity at the top of the U-shapehave a downwardcomponent. • Finally, it can beseenthat the GVF fieldbehaves in analogous fashion whenviewedfrom inside the object. In particular, the GVF vectors are pointingupwardinto the “fingers” of the U-shape, whichrepresentconcavitiesfromthisperspective.

20. Pressureforce • Itispossibletoimprove the performance and convergenceof the GVF snakebydeforming the snake in the givenexternalforcefieldwithpressureforceadded. The pressureforce can eitherinflate or deflate the snake. • The forceisdefinedas:whereis the inwardunitnormalof the model at the pointand is a constantweightingparameter. The signof thedetermineswhethertoinflate or deflate the model and itsvaluedetermines the strengthof the pressureforce. Inflation ( ) Deflation ( )

21. ExampleofGVF-basedsnake performance (1)

22. ExampleofGVF-basedsnake performance (2)

23. GVF withpriordirectional information (1) • Wedefine the G&DVFtobe the vectorfieldthatminimizes the energyfunctional: where and denote the GVF functional and the direction vector flow (DVF) functionalthatisgeneratedby the directionallines. is a positive weightingparameterthatcontrols the influenceof the directionallines on the proposedexternalfield .

24. GVF withpriordirectional information (2) • Given a vectorfieldpre-setafter the normalcontourinitialization, the functionalisdefinedas: • The actionof the DVF functionalistokeepbeingequaltowhere the normofisrelativelylarge. • We can build a vectorfieldsimplybytwo mouse clicks in the image domain (or naturallyby a mouse click plus a drag). Then, the vectorfieldisautomaticallycomputedby: where and denote the starting and the end pointof the line , respectively.

25. Minimizing the G&DVFfunctional • Using the calculusofvariationsfor the energyfunctionalwecabwrite the followingEuler-Lagrangeequaiton: • Parametrizing the descent direction byanartificialtime ,the gradientdescentof (*) isgivenby: • The numericalimplementationof (**) isverysimilartothatof the GVF field, and the convergenceof the iterative processisguaranteedif the artificialtimestepfor (**) satisfies the constraint:

26. Directionalline • Since the image domain is discrete, the approachthat can approximatelyfindallpoints on a directionalline , where and . Without loss ofgenerality, assumingthat and , the point set formedby the allpoints on can beapproximatedby the point set: wheredenotes the floorfunctionthatrounds the variableto the nearestintegerthatis no more than .

27. Generalizedgradientvector flow (1) • The externalforcefieldsderivedfrom the generalized GVF (GGVF) – includingtwospatiallyweightingfunctions– improveactivecontourconvergenceinto long, thinboundaryidentifications, whilemaintainingotherdesirableproperties fo GVF, suchas the extendedcapturerange. The orginal GVF can beseenas a special case of GGVF. • Startingfrom the GVF variationalformulation, the followingnew GGVF variationalproblem can beformulated: • The weightingfunctions and applyto the smoothing and data terms, respectively. Sincewewant the vectorfieldtobeslowelyvarying (or smooth) at locations far fromedgnes, buttoconformtonear the egdes, and shouldbe...

28. Generalizedgradientvector flow (2) …monotonicallynon-increasing and non-decreasingfunctionsof , respectively. Equation (*) reducesto GVF when: • There are many way tospecify and , forexample: • The specificationofdeterminatesto some extent the degreeoftradeoffbetweenfieldsmoothness and gradientuniformity.

29. Minimizing the GGVF functional (1) • Using the calculusofvariationsweobtainfollowingEulerequation: • The solutionofthisvectorequation can beobtainedbycomputing the steady state of the followingequation:or written more explicitly (it can bethenassumedthat ):

30. Minimizing the GGVF functional (2) • As in GVF the PDE specifying GGVF can beimplementedusinganexplicit finite differencescheme, whichisstableif: • Itturns out that a goodresultsimilartothatof GGVF can beachievedusing GVF withverysmallvalueof the regularizationparameter (forexamplecomparedto ). However, becasueissmall in homogeneousregionsaswellasnear the edges, the convergenceof GVF isvery slow and it can take and orderofmagnitudelongerthan GGVF or GVF with .

31. Directionalgradientvector flow (1) • With the GVF snake, only the gradient magnitude is used to compute the gradient vector flow. Thus, the traditional GVF snake may be attracted to strong edges that have the opposite gradient direction with respect to the intended boundary. • In order to utilize the edge direction information in tracking the boundaries, we can use an edge map function that incorporates gradient direction information, which is computed by:where is the original image, is the gradient and is a two-dimensional vector which represents the – known a priori – direction of the edge, which can be specified by the user

32. Directionalgradientvector flow (2) • Let: • Replacingby in the GGVF functionalweobtain the directionalgradientvector flow (DGVF)of the imagebyminimizing the followingequation: • The DGVF can bethenobtainedbysolving the followingequation:

33. ExampleofDGVF-basedsnake performance

34. Multidirectionalgradientvector flow (1) • The fixed DGVF can beextendedto a multi-direction GVF. The basic idea istocompute the directionalgradientforeach pixel with a varying direction. Suppose the center of the objectiswhichisdefinedas: where are the coordinatesof the pixels in the object and is the numberofpixels in the object. Letbe the pixel value at location . The direction vectorfrompointing at the center of the object can beobtainedusing:

35. Multidirectionalgradientvector flow (2) • Afterweobtain the direction vectorfieldofanimagewe can useittocompute the gradientalong the directionfrom the vector set where , , byminimizing the followingdistancemeasure: • The minimizationofequation (*) istoselect a vectorfromninevectorssuchthat the selectedvectorisclosestto the direction vector . The secondterm in (*) is cosine similaritymeasurementwhichisusedtomeasure the similarityoftwodirections. If cosine similarity (rangingfrom 0 to 1) is high then the similarityoftwodirectionsis high.

36. Multidirectionalgradientvector flow (3) • Denoting the approximatevalueof the direction vector at pixel by , the directionalgradientat pixel can becomputedas: • We can avoidsnakeconvergenceto the wrong objectbyusing the negative valueof the directionalgradientobtained in (**): • Byreplacingwithwe can obtain the multi-DGVFbysolving:

37. Applicationto diesel engineimages

38. Case I: snake’s inflation Originalimagewithinitalsnake Edgemap Originalimagewithfinalsnake GVF field

39. Case I: snake’s deflation Originalimagewithinitalsnake Edgemap Originalimagewithfinalsnake

40. Case I: rawimage vs. Gaussianfilter

41. Case I: rawimage vs. anisotropicdiffusion

42. Case II: attemptof jet identification

43. Case III: jet identificationusing DGVF (1)

44. Case III: jet identificationusing DGVF (2) directionalline initial“snake” final“snake” iterations