Refactoring Functional Programs

1. Refactoring Functional Programs Huiqing Li Claus Reinke Simon Thompson Computing Lab, University of Kent

2. Refactoring • Refactoring means changing the design of program … • … without changing its behaviour. • Refactoring comes in many forms • micro refactoring as a part of program development • major refactoring as a preliminary to revision • as a part of debugging, … • As programmers, we do all the time.

3. Refactoring functional programs • What is possible? • What is different about functional programs? • Building a usable tool vs. … • … building a tool that will be used. • Reflection on language design. • Experience, demonstration, next steps.

4. Refactoring • Paper or presentation • moving sections about; amalgamate sections; move inline code to a figure; animation; … • Proof • introduce lemma; remove, amalgamate hypotheses, … • Program • the topic of the lecture

5. Overview • Example refactorings • Refactoring functional programs • Generalities • Tooling: demo, rationale, design. • Catalogue of refactorings • Larger-scale examples … and a case study • Conclusions

6. f x y = …  Name may be too specific, if the function is a candidate for reuse. findMaxVolume x y = …  Make the specific purpose of the function clearer. Rename Needsscopeinformation: just change thisfand not allfs(e.g. local definitions or variables).

7. f x y = … h … where h = …  Hide a function which is clearly subsidiary to f; clear up the namespace. f x y = … (h y) … h y = …  Makes h accessible to the other functions in the module (and beyond?). Lift / demote Needsfree variableinformation: which of the parameters of f is used in the definition of h? Need h not to be defined at the top level, … , DMR.

8. f :: Int -> Char g :: Int -> Int …  Reuse supported (a synonym is transparent, but can be misleading). type Length = Int f :: Length -> Char g :: Int -> Length  Clearer specification of the purpose of f,g. (Morally) can only apply to lengths. Introduce and use a type defn Avoid name clashes Problem with instance declarations (Haskell specific).

9. f :: Int -> Char g :: Int -> Int …  Reuse supported, but lose the clarity of specification. data Length = Length {length::Int} f :: Length -> Char g :: Int -> Length  Can only apply to lengths. Introduce and use branded type • Needsfunction callinformation: where are (these definitions of) f and g called? • Change the calls of f … and the call sites of g. • Choice of data and newtype (Haskell specific).

10. Lessons from the first examples • Changes are not limited to a single point or even a single module: diffuse and bureaucratic … • … unlike traditional program transformation. • Many refactorings bidirectional … • … there is no single correct design.

11. Refactoring functional programs • Semantics: can articulate preconditions and … • … verify transformations. • Absence of side effects makes big changes predictable and verifiable … … unlike OO. • XP is second nature to a functional programmer. • Language support: expressive type system, abstraction mechanisms, HOFs, …

12. Composing refactorings • Interesting refactorings can be built from simple components … • … each of which looks trivial in its own right. • A set of examples … • … which we have implemented.

13. Example program • showAll :: Show a => [a] -> String • showAll = table . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: [String] -> String • table = concat . format

14. Examples • Lift definitions from local to global • Demote a definition before lifting its container • Lift a definition with dependencies

15. Example 1 • showAll :: Show a => [a] -> String • showAll = table . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: [String] -> String • table = concat . format

16. Example 1 lift • showAll :: Show a => [a] -> String • showAll = table . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: [String] -> String • table = concat . format

17. Example 1 • showAll :: Show a => [a] -> String • showAll = table . map show • where • table :: [String] -> String • table = concat . format • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs

18. Example 1 lift • showAll :: Show a => [a] -> String • showAll = table . map show • where • table :: [String] -> String • table = concat . format • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs

19. Example 1 • showAll :: Show a => [a] -> String • showAll = table . map show • table :: [String] -> String • table = concat . format • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs

20. Example 2 • showAll :: Show a => [a] -> String • showAll = table . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: [String] -> String • table = concat . format

21. Example 2 demote • showAll :: Show a => [a] -> String • showAll = table . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: [String] -> String • table = concat . format

22. Example 2 • showAll :: Show a => [a] -> String • showAll = table . map show • where • table :: [String] -> String • table = concat . format • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs

23. Example 2 lift • showAll :: Show a => [a] -> String • showAll = table . map show • where • table :: [String] -> String • table = concat . format • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs

24. Example 2 • showAll :: Show a => [a] -> String • showAll = table . map show • table :: [String] -> String • table = concat . format • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs

25. Example 2 lift • showAll :: Show a => [a] -> String • showAll = table . map show • table :: [String] -> String • table = concat . format • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs

26. Example 2 • showAll :: Show a => [a] -> String • showAll = table . map show • table :: [String] -> String • table = concat . format • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs

27. Example 3 • showAll :: Show a => [a] -> String • showAll = table . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: [String] -> String • table = concat . format

28. Example 3 lift with dependencies • showAll :: Show a => [a] -> String • showAll = table . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: [String] -> String • table = concat . format

29. Example 3 • showAll :: Show a => [a] -> String • showAll = table format . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: ([String] -> [String]) -> [String] -> String • table format = concat . format

30. Example 3 rename • showAll :: Show a => [a] -> String • showAll = table format . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: ([String] -> [String]) -> [String] -> String • table format = concat . format

31. Example 3 • showAll :: Show a => [a] -> String • showAll = table format . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: ([String] -> [String]) -> [String] -> String • table fmt = concat . fmt

32. Example 3 lift • showAll :: Show a => [a] -> String • showAll = table format . map show • where • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: ([String] -> [String]) -> [String] -> String • table fmt = concat . fmt

33. Example 3 • showAll :: Show a => [a] -> String • showAll = table format . map show • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: ([String] -> [String]) -> [String] -> String • table fmt = concat . fmt

34. Example 3 unfold/inline • showAll :: Show a => [a] -> String • showAll = table format . map show • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: ([String] -> [String]) -> [String] -> String • table fmt = concat . fmt

35. Example 3 • showAll :: Show a => [a] -> String • showAll = (concat . format) . map show • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: ([String] -> [String]) -> [String] -> String • table fmt = concat . fmt

36. Example 3 delete • showAll :: Show a => [a] -> String • showAll = (concat . format) . map show • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: ([String] -> [String]) -> [String] -> String • table fmt = concat . fmt

37. Example 3 • showAll :: Show a => [a] -> String • showAll = (concat . format) . map show • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs

38. Name? Example 3 new definition • showAll :: Show a => [a] -> String • showAll = (concat . format) . map show • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs table

39. Example 3 • showAll :: Show a => [a] -> String • showAll = table . map show • format :: [String] -> [String] • format [] = [] • format [x] = [x] • format (x:xs) = (x ++ "\n") : format xs • table :: [String] -> String • table = concat . format

40. Beyond the text editor • All the refactorings can – in principle – be implemented using a text editor, but this is • tedious, • error-prone, • difficult to reverse, … • With machine support refactoring becomes • low-cost: easy to do and to undo, • reliable, • a full part of the programmer's repertoire.

41. Information needed • Syntax: replace the function called sq, not the variable sq …… parse tree. • Static semantics: replace this function sq, not all the sq functions …… scope information. • Module information: what is the traffic between this module and its clients …… call graph. • Type information: replace this identifier when it is used at this type …… type annotations.

42. Machine support invaluable • Current practice: editor + type checker (+ tests). • Our project: automated support for a repertoire of refactorings … • … integrated into the existing development process: tools such as vim and emacs. Demonstration of the tool, hosted in vim.

43. Proof of concept … • To show proof of concept it is enough to: • build a stand-alone tool, • work with a subset of the language, • ‘pretty print’ the refactored source code in a standard format.

44. … or a useful tool? • To make a tool that will be used we must: • integrate with existing program development tools: the program editors emacs and vim. • work with the complete Haskell 98 language, • preserve the formatting and comments in the refactored source code.

45. Consequences • To achieve this we chose to: • build a tool that can interoperate with emacs, vim, … yet act separately. • leverage existing libraries for processing Haskell 98, for tree transformation, yet … • … modify them as little as possible. • be as portable as possible, in the Haskell space.

46. The Haskell background • Libraries • parser: many • type checker: few • tree transformations: few • Difficulties • Haskell98 vs. Haskell extensions. • Libraries: proof of concept vs. distributable. • Source code regeneration. • Real project

47. First steps … lifting and friends • Use the Haddock parser … full Haskell given in 500 lines of data type definitions. • Work by hand over the Haskell syntax: 27 cases for expressions … • Code for finding free variables, for instance …

48. Finding free variables … 100 lines • instance FreeVbls HsExp where • freeVbls (HsVar v) = [v] • freeVbls (HsApp f e) • = freeVbls f ++ freeVbls e • freeVbls (HsLambda ps e) • = freeVbls e \\ concatMap paramNames ps • freeVbls (HsCase exp cases) • = freeVbls exp ++ concatMap freeVbls cases • freeVbls (HsTuple _ es) • = concatMap freeVbls es • … etc.

49. This approach • Boiler plate code … • … 1000 lines for 100 lines of significant code. • Error prone: significant code lost in the noise. • Want to generate the boiler plate and the tree traversals … • … DriFT: Winstanley, Wallace • … Strafunski: Lämmel and Visser

50. Strafunski • Strafunski allows a user to write general (read generic) tree traversing programs … • … with ad hoc behaviour at particular points. • Traverse through the tree accumulating free variables from component parts, except in the case of lambda abstraction, local scopes, … • Strafunski allows us to work within Haskell … other options are under development.