Framework Design Using Function Generalization

1. Framework Design Using Function Generalization H. Conrad Cunningham 1 Pallavi Tadepalli 1 Yi Liu 2 1 Computer & Information Science University of Mississippi 2 Electrical Engineering & Computer Science South Dakota State University

2. Outline • Software framework • Motivation • Function generalization • Cosequential processing • Binary tree traversal • Conclusion • Future work

3. Software Framework • Generic application allowing creation of members of family of related programs • Reusable design expressed as set of abstract classes and way they collaborate • Common and variable aspects known as frozen spots and hot spots Frozen spots Framework library Framework Hot spots User-supplied code(application specific)

4. Motivation • Nontrivial to identify needed hot spot abstractions • Difficult to specify hot spot behaviors • Need systematic generalization methodology • Explore function generalization • incrementally generalize functional structure of specification to produce general application • executable specification expressed as set of functions in Haskell

5. Haskell • Purely functional language • forces explicit consideration of computational effects (no implicit state) • Polymorphic, higher-order, first-class functions & user-defined algebraic data types • enables generic programming • Concise, equational notation • allows convenient mathematical manipulation

6. Function Generalization • Create executable specification for a concrete application as Haskell program • Define scope of family • Identify frozen spots and hot spots • Analyze and design each hot spot system • generalize Haskell program for hot spot • transform simple function to generalized function (e.g., with higher-order parameters) • Transform generalized Haskell program to Java framework

7. Examples • Cosequential processing • Binary tree traversal

8. A1…An CosequentialProcessing C1…Cn B1…Bn Cosequential Processing • Coordinates processing of two input sequences ordered by same total ordering • Create third sequence in incremental steps by merging and matching input elements • Includes set operations and sequential file update applications Input sequences Output sequence

9. 21 18 50 Preorder 21 18 3 1 20 50 30 3 1 20 30 Binary Tree Traversal procedure preorder(t) { if t null, then return; perform visit action for root of tree t; preorder(left subtree of t); preorder(right subtree of t); }

10. Cosequential Processing • H. C. Cunningham and P. Tadepalli. “Using Function Generalization to Design a Cosequential Processing Framework,” In Proceedings of the 39th Hawaii International Conference on System Sciences (HICSS), 10 pages, IEEE, January 2006.

11. A1…An CosequentialProcessing C1…Cn B1…Bn Cosequential Processing • Coordinates processing of two input sequences ordered by same total ordering • Create third sequence in incremental steps by merging and matching input elements • Includes set operations and sequential file update applications Input sequences Write result C1 to output Process A1 and B1 Process Ai and Bj Write C2 to output Output sequence

12. Executable Specification(Merging two ascending integer sequences) merge0 :: [Int]->[Int]->[Int] merge0 [] ys = ys merge0 xs [] = xs merge0 xs@(x:xs’) ys@(y:ys’) | x < y = x : merge0 xs’ ys | x == y = x : merge0 xs’ ys’ | x > y = y : merge0 xs ys’

13. Framework Scope • Process two ordered sequences of values to produce third ordered sequence • Use ordering to restrict to a few current values from sequences • Include classic examples: • sequential file update programs • set and bag operations

14. Frozen Spots • Input sequences have same total ordering • Input sequences are immutable • Output sequence has same ordering as input sequences • Incremental processing • current element from each sequence examined • at least one input sequence advanced by one element • Appropriate action taken after examining current elements from each sequence • Represented by merge function in Haskell

15. Hot Spots • Variability in total ordering • Variability in record format • Variability of input and output sequences • Variability of transformations • Variability of source/destination • Represented as additional functions, types, and class definitions to merge function

16. Hot Spot #1(Variability in total ordering) • Generalizes element type of sequences and associated comparison operators • Restricts polymorphic type of elements to class Ord (having usual relational operations) • Results in generalized comparisons merge1 :: Ord a => [a] -> [a] -> [a] merge1 [] ys = ys merge1 xs [] = xs merge1 xs@(x:xs’) ys@(y:ys’) | x < y = x:merge1 xs’ ys | x == y = x:merge1 xs’ ys’ | x > y = y:merge1 xs ys’

17. Hot Spot #1(Initial prototype as special case) If merge1 type variable a restricted to Int, then merge1 xs ys == merge0 xs ys

18. Hot Spot #2(Variability in record format) • Allows elements of sequences to be records with keys • Adds key extraction function as higher order parameter • Results in generalized record format merge2 :: Ord b =>(a -> b) -> [a] -> [a] -> [a] merge2 key [] ys = ys merge2 key xs [] = xs merge2 key xs@(x:xs’) ys@(y:ys’) | key x < key y = x:merge2 key xs’ ys | key x == key y = x:merge2 key xs’ ys’ | key x > key y = y:merge2 key xs ys’

19. Hot Spot #2(Keyless version is special case) If id is the identity function, then merge2 id xs ys == merge1 xs ys

20. Hot Spot #3(Variability of input and output sequences) • Allows different element format in each sequence • Requires separate key extraction functions for each • Introduces transformation functions • Results in independent sequence formats merge3 kx ky tx ty xs ys = mg xs ys where mg [] ys = map ty ys mg xs [] = map tx xs mg xs@(x:xs’) ys@(y:ys’) | kx x < ky y = tx x : mg xs’ ys | kx x == ky y=tx x: mg xs’ ys’ |kx x>ky y = ty y : mg xs ys’

21. Hot Spot #3(Multikey version is special case of single key version) If xs and ys have the same type, then merge3 key key id id xs ys == merge2 key xs ys

22. Hot Spot #4(Variability of transformations) • Enables use of more general transformations on input • Introduces explicit state to record ongoing computation • Adds accumulating parameter to maintain local state throughout processing • Transforms state to output at end of input sequence processing

23. merge4b kx ky tl te tg nex ney ttx tty res s xs ys = mg s xs ys where mg s [] ys = res (foldl tty s ys) mg s xs [] = res (foldl ttx s xs) mg s xs@(x:xs’) ys@(y:ys’) | kx x < ky y = mg (tl s x y) xs’ ys | kx x == ky y = mg (te s x y) (nex s xs) (ney s ys) | kx x > ky y = mg (tg s x y) xs ys Variable Sequence Transformations

24. Hot Spot #4(Progress requirement) For each call of mg if (kx x == ky y) then (length (nex s xs) < length xs) || (length (ney s ys) < length ys) else True

25. Hot Spot #4(Backward recursive version special case of forward recursive version) merge4b kx ky (\ss x y -> ss ++ [tx x]) -- tl (\ss x y -> ss ++ [tx x]) -- te (\ss x y -> ss ++ [ty y]) -- tl (\ss xs -> tail xs) -- nex (\ss ys -> tail ys) -- ney (\ss x -> ss ++ [x]) -- ttx (\ss y -> ss ++ [y]) -- tty ss xs ys == ss ++ merge3 kx ky tx ty xs ys

26. Hot Spot #5(Variability of source/destination) • Allows diverse sources for inputs and destination for output • No changes to merge4b except its use • sequences already represented as pervasive polymorphic list data type • supply different input sequence arguments • use result by different function

27. Transformation to Java Framework • Drive using shape of Haskell program • Use design patterns (Template, Strategy, etc.) • Construct cosequential framework • recursive legs become main while loop • nonrecursive legs become post-loop code • interfaces and classes represent various hot spot generalizations • java.lang.Comparable for Ord • Keyed for key extraction functions

28. Cosequential Processing Framework in Java public final void merge() // template method { advXs(); advYs(); // uses Interators and Keyed while(xsNotEmpty && ysNotEmpty) { int cmpxy = xKey.compareTo(yKey); // Ord as Comparable if (cmpxy < 0) { transLt(); advXs(); } // tl else if (cmpxy == 0) { transEq(); advEqXs(); advEqYs(); } // te, nex, ney else { transGt(); advYs(); } // tg } while (xsNotEmpty) { transYsEmpty(); advXs(); } // ttx while (ysNotEmpty) { transXsEmpty(); advYs(); } // tty finish(); // res }

29. Applications of Cosequential Framework • Implemented master-transaction file update program (bank account)

30. Binary Tree Traversal • H. C. Cunningham, Y. Liu, and P. Tadepalli. “Framework Design Using Function Generalization: A Binary Tree Traversal Case Study,” In Proceedings of the ACM SouthEast Conference, pp. 312-318, March 2006.

31. 21 18 50 Preorder 21 18 3 1 20 50 30 3 1 20 30 Binary Tree Traversal procedure preorder(t) { if t null, then return; perform visit action for root of tree t; preorder(left subtree of t); preorder(right subtree of t); }

32. Executable Specification data BinTree a = Nil | Node(BinTree a) a BinTree a) preorder :: BinTree a -> [a] preorder Nil = [] preorder (Node l v r) = v : preorder l ++ preorder r

33. Framework Scope • Standard kinds of depth-first traversals • Flexible visit actions that are functions of accumulated state along traversal • Other traversals orders (e.g. level by level) • Binary search trees, but not multiway trees or graphs

34. Frozen Spots • Structure of tree (BinTree) not redefined by clients • Traversal accesses every node once (unless stopped early) • Traversal performs one or more visit actions on access to node of tree • Represented by a traversal function in Haskell

35. Hot Spots • Variability in the visit operation’s action • Variability in ordering of visit action with respect to subtree visits • Variability in tree navigation technique (not just left-to-right, depth first) • Represented as additional functions, types, and class definitions to traversal function

36. Hot Spot #1(Generalizing the visit action) • Represent visit action by update-state function (us) passed into traversal function • Accumulate state along traversal path gaPre :: (a -> b -> b) -> (b -> b) -> b -> BinTree a -> b gaPre us un is t = po t is where po Nil s = un s po (Node l v r) s = po r (po l (us v s))

37. Hot Spot #1(Initial prototype as special case) The following identity holds: gaPre (\x y -> y ++ [x]) id [] t == preorder t

38. Hot Spot #2(Generalizing the visit order) • Allow visit actions at three points (Euler tour traversal) • first arrival (left) • between subtree traversals (bottom) • before final departure (right) gvTraverse :: (a -> b -> b) -> (a -> b -> b) -> (a -> b -> b) -> (b -> b) -> b -> BinTree a -> b gvTraverse ul ub ur un is t = tr t is where tr Nil s = un s tr (Node l v r) s = ur v (tr r (ub v (tr l (ul v s))))

39. Hot Spot #2(Single visit action as special case) The following identity holds: gvTraverse us id id un is t == gaPre us un is t

40. Hot Spot #3(Generalizing the tree navigation) • Pass in tree navigation function (nav) • Navigation function generates a list of update functions • Folding (composition) of list from initial state generates traversal traverse nav ua ub ud un is t = compose (nav ua ub ud un t) is where compose fs s = foldl (flip (.)) id fs s

41. Hot Spot #3(Euler tour traversal as special case) euler ua ub ud un t = doEuler t where doEuler Nil = [un] doEuler (Node l v r)] = [(ua v)] ++ doEuler l ++ [(ub v)] ++ doEuler r ++ [(ud v)] The following identity holds: traverse euler ua ub ud un is t == gvTraverse ua ub ud un is t

42. Transformation to Java Framework • Drive using shape of Haskell program • Use design patterns (Composite, Strategy, Template, Visitor, etc.) • Construct binary tree traversal framework • use Composite pattern to represent tree • use Strategy pattern to encapsulate higher order parameters of functions (i.e. state update functions) • use Visitor pattern to separate tree navigation from the application of state updates

43. Transformation to Java Framework public class Node extends BinTree { public Node(Object v, BinTree l, BinTree r) { value = v; left = l; right = r; } … // accept a Visitor object public void accept(BinTreeVisitor v) { v.visit(this); } … private Object value; // instance data private BinTree left, right; } public interface BinTreeVisitor { abstract void visit(Node t); abstract void visit(Nil t); }

44. Transformation to Java Framework public class EulerTourVisitor implements BinTreeVisitor { public EulerTourVisitor(EulerStrategy es, Object ts) { this.es = es; this.ts = ts; } public void setVisitStrategy(EulerStrategy es) { this.es = es; } public void visit(Node t) // Visitor hook implementations { ts = es.visitLeft(ts,t); // upon first arrival t.getLeft().accept(this); ts = es.visitBottom(ts,t); // upon return from left t.getRight().accept(this); ts = es.visitRight(ts,t); // upon completion of node } public void visit(Nil t) { ts = es.visitNil(ts,t); } public Object getResult() { return ts; } private EulerStrategy es; // encapsulate state change ops private Object ts; // traversal state }

45. Conclusion • Framework construction followed function generalization • Each transformation produced an executable specification • Appropriate hooks (hot spot abstractions) defined • Constructed cosequential processing framework with better understanding of hot spot behaviors • Constructed general binary tree traversal framework with better understanding of hot spot behaviors

46. Future Work • Develop better guidelines for generalizing Haskell programs • Investigate usage of Haskell features like modules, classes, and monads • Develop better guidelines for creating Java frameworks from Haskell programs • Conduct case studies of larger programs • Investigate usage of Ruby instead of Haskell and Java ☺

47. Acknowledgments • Cuihua Zhang (Northeast Lakeview College) • Acxiom Corporation • University of Mississippi • South Dakota State University

48. Discussion Any questions or comments?