LINQ , Take Two Realizing the LINQ to Everything Dream

1. LINQ, Take TwoRealizing the LINQ to Everything Dream Bart J.F. De Smet blogs.bartdesmet.net/bart bartde@microsoft.com

2. A Historical Perspective 5 years ago Little recent innovation Where’s the cloud? Censored

3. Language Integrated Query Monads Why? What? How?

4. The Monadic Bind Operator Revealed Could there be more? IEnumerable<T> IQueryable<T> new[]{ 42 } IEnumerable<R> SelectMany<T, R>(this IEnumerable<T> source,Func<T, IEnumerable<R>> selector) SelectMany Also see www.codeplex.com/LINQSQO for “Project MinLINQ”

5. Building the Maybe Monad (For Fun and No Profit) Null-propagating dot string s = name?.ToUpper(); One single library function suffices Syntactic sugar name.SelectMany(_ => _.ToUpper(), s => s) from _ in name from s in _.ToUpper() select s Compiler

6. Essential LINQ

7. Query Providers Revisited – Do We Need IQueryable<T>? Does it really have to be a runtime check? Implements IQueryable<T> varsrc = newSource<Product>(); var res = from p insrc wherep.Price > 100m groupp byp.Category; foreach(var p in res) Console.WriteLine(p); Compiles fine

8. Leveraging The Query Pattern varres = from p insrc wherep.Price > 100m groupp byp.Category; Syntactic sugar varres = src .Where(p => p.Price> 100m) .GroupBy(p => p.Category); No GroupBy“edge” Can be instance methods Where Select Source<T> Filtered<T> Projected<T>

9. Taking It One Step Further classTweetAboutFromLoc { publicFromStringFrom; publicAboutStringAbout; publicLocStringLocation; } classFromString { FilterFromoperator==( FromStringf, strings) } varres= fromtweetintwitter classTweetAboutLoc { publicAboutString About; publicLocStringLocation; } classAboutString { FilterAboutoperator==( AboutStringf, string s) } wheretweet. ==“Bart” From About From Location From wheretweet. About == “LINQ” About About Location selecttweet; Recipe Method overloads Type state machine Operator overloads Query “learns” From operator overloading “Has a” type class Twitter { publicTwitterByFromWhere(Func<TweetAboutFromLoc, FilterFrom> filter); // Other filter methods } class TwitterByFrom { publicTwitterByAboutFromWhere(Func<TweetAboutLoc,FilterAbout> filter); // Other filter methods // Fields with current filters } Custom syntax trees

10. Query Providers Revisited

11. Event Processing Systems Way simpler with Rx Rx is a library for composing asynchronous and event-basedprograms using observable sequences. Queries! LINQ! • .NET 3.5 SP1 and 4.0 • Silverlight 3, 4, and 5 • XNA 4.0 for Xbox 360 • Windows Phone 7 • JavaScript (RxJS) Download at MSDN Data Developer Centeror use NuGet

12. Push-Based Data Retrieval Application MoveNext Got next? OnNext Interactive Reactive Have next! Environment IObservable<T> IObserver<T> IEnumerable<T> IEnumerator<T>

13. Event Programming Interfaces Both interfaces ship in the .NET 4 BCL interface IObservable<out T> { IDisposable Subscribe(IObserver<T> observer);} interfaceIObserver<in T> { voidOnNext(T value); voidOnError(Exception ex); voidOnCompleted(); }

14. IQbservable<T> LINQ to WMI Events How? ToQbservable Translatable (Expression trees) IQueryable<T> LINQ to SQL ToQueryable LINQ to *.* AsObservable Homo-iconic AsEnumerable AsQbservable AsQueryable ToObservable IEnumerable<T> LINQ to Objects IObservable<T> LINQ to Events Fixed (MSIL) ToEnumerable Pull(interactive) Push (reactive) What? Duality Concurrency(IScheduler) Where? Message loops Distributed Worker pools Threads

15. IQbservable<T> – The Dual of IQueryable<T> interface IQbservable<out T> : IObservable<T> { ExpressionExpression { get; } TypeElementType { get; } IQbservableProvider Provider { get; } } interfaceIQbservableProvider { IQbservable<R> CreateQuery<R>(Expression expression); } We welcome semantic discussions Extended role for some operators No Execute method

16. LINQ to WMI Events (WQL)

17. Rx and the Visual Studio Async CTP Asynchronously computed value • The essence of the feature • Feature based on the “await-pattern” • Awaiting Task<T> • Returns the single result (of type T) • Awaiting IObservable<T>? • We return the lastresult (of type T) • Want all values? Use ToList() first! asyncTask<int> GetLength(Task<string> name) { string n = awaitname; returnn.Length; } Result is async too Can await a “future” value

18. Asynchronous Pull-Based Data Access? • Await feature is about sequential code • Great for single-valued computations • What about asynchronous pull-based datacollections? public interface IAsyncEnumerable<T> { IAsyncEnumerator<T> GetEnumerator(); } public interface IAsyncEnumerator<T> : IDisposable { T Current { get; } Task<bool> MoveNext(); } Rx definesQuery Operators Can await

19. Rx support for C# and VB “await”

20. Where Execution Happens– Introducing Schedulers Rx’sunified way tointroduceconcurrency interfaceIScheduler { DateTimeOffset Now { get; } IDisposable Schedule<T>(T state, Func<IScheduler, T, IDisposable> action); IDisposable Schedule<T>(T state, TimeSpandueTime, Func<IScheduler, T, IDisposable> action);IDisposableSchedule<T>(T state, DateTimeOffsetdueTime, Func<IScheduler, T, IDisposable> action); } ToObservable IObservable<T> IEnumerable<T> ToEnumerable Synchronous Asynchronous

21. IScheduler Specifies Where Things Happen Imported TextBoxTextChangedevent varres = from word ininput.DistinctUntilChanged() .Throttle(TimeSpan.FromSeconds(0.5)) from words in lookup(word) select words; Asynchronous web service call Indicates where things run Use of scheduler to synchronize res.Subscribe(words => { lst.Items.Clear(); lst.Items.AddRange((from word in words selectword.Word).ToArray());}); res.ObserveOn(new ControlScheduler(frm)).Subscribe(words => { lst.Items.Clear(); lst.Items.AddRange((from word in words selectword.Word).ToArray());});

22. IScheduler Parameterization Baked in notion of “where”? Entry-point for the schema Custom schedulers could be very rich (e.g. server farm) var ctx = newNorthwindDataContext(); var res = from product in ctx.Products where product.Price > 100m select product.Name; Decoupled “what” from “where” foreach (varp inres.RemoteOn(newSqlScheduler(“server”))) // Process product

23. Expression Tree Remoting Rx .NET fromtickerinstocks whereticker.Symbol == “MSFT” selectticker.Quote Observable data source Retargeting to AJAX JSON serializer RxJS stocks .Where(function (t) { returnt.Symbol == “MSFT”; }) .Select(function (t) { returnt.Quote; })

24. Remoting of Query Expressions

25. LINQ to the Unexpected Non-persistent Model[ Decisions[Reals, SA, VZ], Goals[ Minimize[20 * SA + 15 * VZ] ], Constraints[ C1 -> 0.3 * SA + 0.4 * VZ >= 2000, C2 -> 0.4 * SA + 0.2 * VZ >= 1500, C3 -> 0.2 * SA + 0.3 * VZ >= 500, C4 -> SA <= 9000, C5 -> VZ <= 6000, C6 -> SA >= 0, C7 -> VZ >= 0 ] ] fromminctx.CreateModel(new { vz = default(double), sa = default(double) }) where 0.3 * m.sa + 0.4 * m.vz >= 2000 && 0.4 * m.sa + 0.2 * m.vz >= 1500 && 0.2 * m.sa + 0.3 * m.vz >= 500 where 0 <= m.sa && m.sa <= 9000 && 0 <= m.vz && m.vz <= 6000 orderby 20 * m.sa + 15 * m.vz selectm Cost / barrel Refinement % Max # barrels Min # barrels To compute call Solve

26. LINQ to the Unexpected fromcostSAinGetPriceMonitor("SA") fromcostVZinGetPriceMonitor("VZ") from m inctx.CreateModel( new{ vz = default(double), sa = default(double) }) where 0.3 * m.sa + 0.4 * m.vz >= 2000 && 0.4 * m.sa + 0.2 * m.vz >= 1500 && 0.2 * m.sa + 0.3 * m.vz >= 500 where 0 <= m.sa && m.sa <= 9000 && 0 <= m.vz && m.vz <= 6000 orderbycostSA * m.sa + costVZ * m.vz select m Observabledata sources SelectMany Subscribe here! Parametersto decision

27. Theorem Solving using Z3

28. A Futuristic Perspective Next 5 years Remoting Async Solvers Rx and Ix are here Toolkits Scheduler Censored

29. Stay up to date with MSDN Belux • Register for our newsletters and stay up to date:http://www.msdn-newsletters.be • Technical updates • Event announcements and registration • Top downloads • Follow our bloghttp://blogs.msdn.com/belux • Join us on Facebookhttp://www.facebook.com/msdnbehttp://www.facebook.com/msdnbelux • LinkedIn: http://linkd.in/msdnbelux/ • Twitter: @msdnbelux DownloadMSDN/TechNet Desktop Gadgethttp://bit.ly/msdntngadget

30. TechDays 2011 On-Demand • Watchthis session on-demand via Channel9http://channel9.msdn.com/belux • Download to your favorite MP3 or video player • Get access to slides and recommended resources by the speakers

31. THANK YOU