C# and LINQ

1. C# and LINQ Yuan Yu Microsoft Research Silicon Valley

2. Collections and Iterators IEnumerable<T> • Very easy to use: • foreach (string name in persons) • { • Console.WriteLine(name); • } Iterator (current element) Elements of type T

3. More on IEnumerable<T> • IEnumerable<T> is a generic collection • C# generics is very similar to C++ template • T is a type parameter representing its element type • IEnumerable<double> • IEnumerable<MyClass> • .NET collections implement IEnumerable<T> • T[], List<T>, HashSet<T>, Stack<T>, Queue<T>, Dictionary<K, T>, …

4. IEnumerable<T> Examples • Example 1 • Example 2 string[] persons = { “Frank”, “Bob”, “Chandu”, “Mike”, “Dennis” } foreach (string name in persons) { Console.WriteLine(name); } int[] numbers = { 1, 5, 2, 12, 4, 5 }; int sum = 0; foreach (int x in numbers) { sum += x; }

5. LINQ: Operators on Collection Collection<T> collection; boolIsLegal(Key); string Hash(Key); var results = from c in collection where IsLegal(c.key) selectnew { Hash(c.key), c.value};

6. LINQ Operators Input Where (filter) Select (map) GroupBy OrderBy (sort) Aggregate (fold) Join

7. Lambda Expression • A nice way to represent anonymous functions • Examples: • Func<int, int> inc = x => x + 1; • Func<int, double> sqrt = x => Math.Sqrt(x); • Func<int, int, int> mul = (x, y) => x * y; • Func<T, R> represents any method that takes a argument of type T and returns a value of type R • Similar to C++ function pointer

8. Where • Filters the elements in a collection based on a predicate • Example: IEnumerable<T> Where<T>(IEnumerable<T> source, Func<T, bool> pred) int[] a = { 1, 5, 2, 12, 4, 5 }; IEnumerable<int> result = a.Where(x => x > 4);

9. Select • Transforms each element of a collection into a new form • Example: IEnumerable<R> Select<T, R>(IEnumerable<T> source, Func<T, R> selector) int[] a = { 1, 5, 2, 12, 4, 5 }; IEnumerable<int> result = a.Select(x => x * x);

10. Composing Operators • Composing computations • Or simply • You can use “var” to represent the type int[] a = { 1, 5, 2, 12, 4, 5 }; IEnumerable<int> r1 = a.Where(x => x > 4); IEnumerable<int> r2 = r1.Select(x => x *x); int[] a = { 1, 5, 2, 12, 4, 5 }; IEnumerable<int> r2 = a.Where(x => x > 4).Select(x => x * x); int[] a = { 1, 5, 2, 12, 4, 5 }; var r2 = a.Where(x => x > 4).Select(x => x * x);

11. Query Comprehension • If you really hate lambda expression, you can also use the alternate SQL-like syntax int[] a = { 1, 5, 2, 12, 4, 5 }; var r2 = from x in a where x > 4 select x * x;

12. Invoking User-Defined Functions int[] a = { 1, 5, 2, 12, 4, 5 }; var r2 = from x in a whereMyPredicate(x) selectMath.Sqrt(x); public static boolMyPredicate(int x) { // User code here }

13. SelectMany • Tranforms each element of a sequence to an IEnumerable<R>and flattens the resulting sequences into one sequence • Example: IEnumerable<R> SelectMany<T, R>( IEnumerable<T> source, Func<T, IEnumerable<R>> selector) string[] lines= { “A line of words of wisdom”, “Dryad and DryadLINQ are great”}; var result = lines.SelectMany(x => x.Split(' '));

14. OrderBy • Sorts the elements of a sequence in ascending order according to a key • Example: IEnumerable<T> OrderBy<T, K>(IEnumerable<T> source, Func<T, K> keySelector) IEnumerable<Employee> employees; varresult = employees.OrderBy(x => x.Name);

15. GroupBy • Groups the elements of a sequence according to a specified key selector function • IGrouping<K, T> represents a group of elements of type T with key K • g.Key returns the key of the group • g is IEnumerable<T> IEnumerable<IGrouping<K, T>> GroupBy<T, K>(IEnumerable<T> source, Func<T, K> keySelector)

16. GroupBy Examples • Example 1: • Example 2: int[] numbers = { 5, 4, 1, 3, 9, 8, 6, 7, 2, 0 }; var groups= numbers.GroupBy(x => x % 5); string[] items= { "carrots", "cabbage", "broccoli", "beans", "barley" }; IEnumerable<IGrouping<char, string>> foodGroups= items.GroupBy(x => x[0]);

17. Distinct • Returns distinct elements from a sequence • Example IEnumerable<T> Distinct<T>(IEnumerable<T> source) int[] numbers = { 1, 5, 2, 11, 5, 30, 2, 2, 7 }; varresult = numbers.Distinct();

18. Aggregate • Applies an accumulator function over a sequence • Example: RAggregate<T>(IEnumerable<T> source, R seed, Func<R, T, R> func) double[] doubles = { 1.7, 2.3, 1.9, 4.1, 2.9 }; doubleresult = doubles.Aggregate(1.0, (r,n) => r * n);

19. Pre-defined Aggregators • Useful pre-defineds • Count, LongCount, Sum, Max, Min, Average, All, Any, Contains, … • Examples: IEnumerable<int> numbers; long result = numbers.LongCount();

20. Exercises (1) • Keep all numbers divisible by 5 var div = x.Where(v => v % 5 == 0); • The average value varavg = x.Sum() / x.Count(); • Normalize the numbers to have a mean value of 0 var avg = x.Sum() / x.Count(); var norm = x.Select(v => v - avg); • Keep each number only once varuniq = x.GroupBy(v => v) .Select(g => g.Key); varuniq = x.Distinct();

21. Exercises (2) • Flatten lists varflatten = x.SelectMany(v => v); • The average of all even #s and of all odd #s varavgs = x.GroupBy(v => v % 2) .Select(g => g.Sum() / g.Count()); • The most frequent value varfreq = x.GroupBy(v => v) .OrderBy(g => g.Count()) .Select(g => g.Key) .First() • The number of distinct positive values varpos = x.Where(v => v >= 0) .Distinct() .Count();

22. Putting them together: Histogram public static IEnumerable<Pair> Histogram( IEnumerable<LineRecord> input, int k) { var words = input.SelectMany(x => x.line.Split(' ')); var groups = words.GroupBy(x => x); var counts = groups.Select(x => new Pair(x.Key, x.Count())); var ordered = counts.OrderByDescending(x => x.count); var top = ordered.Take(k); return top; }

23. MapReduce in LINQ public static IEnumerable<R> MapReduce<T,M,K,S>( IEnumerable<T> input, Func<T, IEnumerable<M>> mapper, Func<M, K> keySelector, Func<IGrouping<K, M>, IEnumerable<R>> reducer) { var map = input.SelectMany(mapper); var group = map.GroupBy(keySelector); var result = group.SelectMany(reducer); return result; }