LINQ in Breadth

1. LINQ in Breadth Querying everything everywhere • Bart J.F. De Smet • Microsoft Corporation • bartde@microsoft.com • blogs.bartdesmet.net/bart

2. LINQ in your DNA DNA is declarative

3. Infinite possibilities Image: http://www.hawkdigital.com/images/InfiniteCre8tivity.jpg

4. LINQ to Anything? to Alcohol Image: http://www.hiphongkong.com/bars/linq.jpg

5. LINQ from Anywhere? Image: http://www.linq.com.hk/findus.html

6. C# • Visual Basic • LINQ ~ ∞² • F# • PowerShell • Anything Fan in LINQ Fan out

7. A different mindset? Image: http://www.endlessgames.com/ns/Linq/Linq.gif

8. C# 3.0 – LINQ and Lambda Image: http://vijay.screamingpens.com/archive/2008/05/04/linq-amp-lambda.aspx

9. Demo To refresh, hit F5

10. Syntactic sugar from x inxswhere x % 2 == 0 select x xs.Where(x => x % 2 == 0).Select(x => x) Image: http://hummer.syntacticsugar.com:8080/kb_upload/Image/company_images/logo.png

11. Query operators IEnumerable<R> SelectMany<T,R>(thisIEnumerable<T>,Func<T, IEnumerable<R>>) IQueryable<R> SelectMany<T,R>(thisIQueryable<T>,Expr<Func<T, IQueryable<R>>>) Monads Bind<T> :: M<T>  (T M<R>)  M<R> Return<T> :: T  M<T> Join<T> :: M<M<T>>  M<T>

12. Nullables through LINQ • Don't fear the monads from x in 123.AsMaybe()from y in Maybe<int>.Nothingselect x + y; Source: http://channel9.msdn.com/shows/Going+Deep/Brian-Beckman-Dont-fear-the-Monads

13. Extension methods Source: http://en.wikipedia.org/wiki/Extension_method IEnumerable<R> SelectMany<T,R>(thisIEnumerable<T>,Func<T, IEnumerable<R>>) IQueryable<R> SelectMany<T,R>(thisIQueryable<T>,Expression<Func<T, IQueryable<R>>>)

14. Extension by design Declare Implement interface IAmTooBig{LoremIpsum(Dolor sit);AmetConsectetur();AdipiscingElit(Uteu);Felis At(Enimcommodo);ConsecteturDonec();TinciduntLaoreet();TurpisVestibulum(); …} class MegaThingy: IAmTooBig{ …publicAmetConsectetur( {throw newNotImplementedException ("Sounds like Latin"); } …}

15. You gotta love lambdas… Image: http://minimonimania.files.wordpress.com/2008/11/maimicode.png

16. …but don't overdo it Image: http://stackoverflow.com/questions/515413/what-are-some-resources-for-learning-lambda-calculus

17. Expressiveness for free Parallel.For(0, height, j => { PixelData* outPixel = fastOut.GetInitialPixelForRow(j); PixelData* startPixel = fastStart.GetInitialPixelForRow(j); PixelData* endPixel = fastEnd.GetInitialPixelForRow(j); for (int i = 0; i < width; i++) { outPixel->R = (byte)((startPixel->R * blend) + .5 + (endPixel->R * (1 - blend))); outPixel->G = (byte)((startPixel->G * blend) + .5 + (endPixel->G * (1 - blend))); outPixel->B = (byte)((startPixel->B * blend) + .5 + (endPixel->B * (1 - blend))); outPixel++; startPixel++; endPixel++; } }); Operator Operand

18. Closures Captured outer int max = 100; xs.Where(x => x < max) <>__Closure c = new<>__Closure(); c.max = 100;xs.Where(c.<>__Where); class<>__Closure {publicint max;publicbool <>__Where(int x) {return x < max; }}

19. Know your side-effects! Func<string> read;using (varfs = File.OpenRead("bar.txt")) read = () => fs.ReadToEnd();string result = read(); IEnumerable<int> invert = null;try { invert = from x inxsselect 1 / x;}catch (DivideByZeroException) {Console.WriteLine("Oops!");}foreach (var x in invert)Console.WriteLine(x); Image: http://www.undoge.org/wp-content/uploads/2008/04/cartoon-sideeffect.gif

20. Lazy = good • C# 2.0 iterators • yield return • On-demand fetch • E.g. LINQ to Objects • C# 3.0 lambdas • () => compute • On-demand evaluation • E.g. TPL futures • “Yielding control” to library/runtime Image: http://onemansblog.com/2007/04/03/man-too-lazy-to-walk-dog/

21. foreach • foreach • foreach Query results • Where • Where • Where • Select • Select • Select IEnumerable<T> Data source IEnumerable<R>

22. Demo Lambda the ultimate

23. Query expressions cheat sheet http://community.bartdesmet.net/blogs/bart/archive/2008/08/30/c-3-0-query-expression-translation-cheat-sheet.aspx

24. Freedom of choice classHomer {publicMarge Where(Func<Bart, bool> p) { … } …}classMarge {publicHomer Select(Func<Maggie, Lisa> p) { … }} var res = (from x innewHomer()where x.HasSkateboardselect x.Sister).Father;

25. Are predicates Boolean-valued? Dim res = From product Inctx.Products _Where (<And><Gt><FieldRefName="UnitPrice"/><ValueType="Integer">123</Value></Gt><BeginsWith><FieldRefName="ProductName"/><Value Type="String">C</Value></BeginsWith></And>) _SelectNewWith {.Name = product.ProductName, _ .Price = product.UnitPrice } Func<T, XElement>

26. Demo LINQ to Regular Expressions

27. The runtime in control from x inxswhere x > 0 where x < 100 select x from x inxswhere x > 0 && x < 100 select x from x inxsorderby x where x < 100 select x from x inxswhere x < 100 orderby x select x from x inxswhere x < 0 && x > 0 select x Always false

28. Where do expression trees come from? • Visual Basic and C# lambdas • Expression<Func<…>> • Expression<Action<…>> • Dynamic LINQ • Included in VS 2008 samples • ”Lambdas as strings” But no statement trees yet!

29. Expression trees Dynamic Language Runtime You are here

30. Meta-programming Func<int, int, int> add = (a, b) => a + b;var res = add(1, 2); Expression<Func<int, int, int>> add = (a, b) => a + b;var res = ((Func<int, int, int>)add.Compile())(1, 2); Code as data λ ldarg.0ldarg.1addret + { a, b } + a b a b

31. Visitors < < stringVisitAdd(BinaryExpression ex) {returnVisitPrefix(ex, "+");} stringVisitLess(BinaryExpression ex) { returnVisitPrefix(ex, "<"); } stringVisitConst(ConstantExpression ex) {return"" + ex.Value;} stringVisitParam(ParameterExpression ex) {returnex.Name;} stringVisitPrefix(BinaryExpression ex,string op) {var left = Visit(ex.Left);var right = Visit(ex.Right);return"(" + op + " " + left + " " + right + ")";} + + 0 0 a a b b (< (+ a b) 0) (+ a b) a b a (+ a b) 0 VisitParam VisitPrefix VisitAdd VisitConst VisitPrefix VisitLess

32. Demo Fun with Expression Trees LINQ through PowerShell

33. Life without expression trees?

34. Domain-specific trees products.Where(p => p.Price > 100 && p.Name.StartsWith("C")) NumberColumn TextColumn Operator overloading Image: http://www.railwaysleeper.com/KFdeliveryservice.htm

35. Intelligent operator overloading Func<Product, Condition> p => p.Price > 100 && p.Name.StartsWith("C") Condition & Condition Condition > StartsWith NumberColumn TextColumn int string Price 100 Name “C” classNumberColumn { staticConditionoperator >(NumberColumncol,int value) { … }} classCondition { staticConditionoperator &(Condition left,Condition right) { … }} classTextColumn { ConditionStartsWith(string value) { … }}

36. Fit-n-finish user experience classTable<T> {publicFiltered<T> Where(Func<T, Condition> filter) { … }} Determines next operators in the query from p in products wherep.Price > 100 select p.Name from p in products wherep.Price > 100 orderbyp.Price from p in products orderbyp.Pricewherep.Price > 100 from p in products orderbyp.Price, p.Name from p in products orderbyp.Price from p in products wherep.Price > 100 from p in products orderbyp.Priceselect p.Name from p in products select p.Name ThenBy* Ordered<T> OrderBy* Select Table<T> Where Projected<T> OrderBy* Filtered<T> Where Select Select

37. The result • Expression trees • Everything works by default • Runtime exceptions to restrict • Possible clash of semantics • Hand-crafted query patterns • Nothing works by default • Static compile-time checking • Own type universe Image: http://atheistexile.com/libertyUSign5.jpg

38. Demo ExceLinq and SignaLinq

39. Summary • LINQ != a foreign concept • Design lessons learned • Extension-by-design • Leverage language patterns • Get to know you monads! • LINQify the world • Fan-in and fan-out • Data is everywhere Code = data! Image: http://da-crystal.net/GCMS/blog/arabic-book-start-linq/

40. Q & A