Better SharePoint through Service Oriented Architecture and Development

1. Better SharePoint through Service Oriented Architecture and Development DEV346, PM346

2. Eugene Rosenfeld, MVP, MCAD • CTO, Black Blade Associates • http://www.blackbladeinc.com • erosenfeld@blackbladeinc.com • Programmer, better programmer, architect, CTO.

3. About this Session • Intended audience • Application architects • Some SharePoint dev experience is helpful • Some distributed application design experience is helpful • Very little code, mostly Visios

4. Agenda • EAI and SOA • Traditional SharePoint Application Architecture • SOA Intro • Rethink the Application as Services • Case Study: Document Section Manager application to docBlock service

5. EAI and SOA • EAI – Enterprise Application Integration • What does EAI have to do with SOA? • The goal of both EAI and SOA is to let systems communicate

6. Prime Tenet of EAI • Minimal intrusion into a system • The fewer changes made to a system, the fewer things break • The fewer changes made to a system, the fewer things to maintain • The fewer changes made to a system, the fewer things will break when the system is upgraded

7. Intrusiveness (low to high) • Client-side artifacts • Images • CSS • JavaScript • Server-side artifacts • CAML (List / site def, content types, etc) • Web Parts / application pages • Event Receivers / workflows • Any other compiled code

8. Agenda • EAI and SOA • Traditional SharePoint Application Architecture • SOA Intro • Rethink the Application as Services • Case Study: Document Section Manager application to docBlock service

9. Sample SharePoint Application

10. Aspects of the Architecture • Entire codebase must reside on each web front end server • Web front end servers bear the bulk of the load for executing the code • Most code runs in the w3wp.exe process • Most code runs in the context of an HTTP request

11. Problems with the Architecture • Entire codebase must reside on each web front end server • Deployment complexity • Configuration complexity • Potential application compatibility issues • One more thing to worry about when upgrading to next version of SharePoint

12. Problems with the Architecture • Web front end servers bear the bulk of the load for executing the code • Difficult to determine load application places on farm • May require scaling out the web front ends

13. Problems with the Architecture • Most code runs in the w3wp.exe process • Recycling the w3wp.exe process causes all applications to restart

14. Problems with the Architecture • Most code runs in the context of an HTTP request • HTTP has a 60 second timeout • Bad code can break the HTTP request pipeline

15. Agenda • EAI and SOA • Traditional SharePoint Application Architecture • SOA Intro • Rethink the Application as Services • Case Study: Document Section Manager application to docBlock service

16. SOA Intro • SOA: Service Oriented Architecture • SOA is architecture, not technology

17. SOA Myths • Myth: SOA is new • Myth: SOA needs an Enterprise Service Bus • Myth: SOA requires a particular technology • Myth: SOA requires Web Services • Myth: SOA requires XML • Myth: SOA requires text-based interface • Myth: SOA requires HTTP • Proof: Look at WCF in .Net 3.0

18. Web Service is not a SOA Service

19. What’s Left in SOA Definition? • SOA is a design principle that abstracts a business capability from the interface used to access the capability. • SOA services should: • Scale up • Scale out • Be distributable • Have remotable interfaces • Be topology-aware

20. Who are the Clients? • HTML Client: IE, Firefox, Safari, etc • RSS Client: Outlook, NewsGator, etc • SMTP Client: Outlook, Notes, Evolution, etc • Terminal Client: HyperTerminal, x3270, etc • Web Service Client: ? – Custom Code • XML Client: ? – Custom Code

21. Agenda • EAI and SOA • Traditional SharePoint Application Architecture • SOA Intro • Rethink the Application as Services • Case Study: Document Section Manager application to docBlock service

22. Design Goals • Get as much code out of the w3wp.exe process as possible • Get as much code out of SharePoint web front ends as possible

23. Why? To Mitigate Risk • Writing the application is the easy part • Operational cost of the application • 70% of app cost after completion, during operations and maintenance • Platform updates and configuration changes • Platform variations • Windows Server 2003 / 2008 • x86 / x64 • Permissions

24. Achieving the Design Goals • Separate processing into 2 categories: • Requires SharePoint infrastructure • Does not require SharePoint infrastructure • Separate processing duration into 3 categories • Fast (< 15 sec) • Medium ( > 15 sec) • long running ( > 60 sec)

25. Design Implementation

26. My code fits more than one box • Of course it does. It should. • Most non-trivial applications will use at least 2-3 boxes.

27. For Example… • Box 1: A click in a web part causes a document property in SharePoint to change. • Box 2: The property change triggers an asynchronous event receiver to call a remote web service. • Box 4: The remote web service starts a long-running operation. • Box 6: The long-running operation completes and publishes the results back to SharePoint. • Box 1: The web part displays the new state of the data when a user revisits the page.

28. Re-architecting an Application • It’s time when… foreach(SPWeb web in currentWeb.Webs) foreach(SPList list in web.Lists) //do lots of work! • Process flows are the key to re-architecture • Reuse processes, not code

29. Agenda • EAI and SOA • Traditional SharePoint Application Architecture • SOA Intro • Rethink the Application as Services • Case Study: Document Section Manager application to docBlock service

30. Document Section Manager App • What it did • Software that allowed multiple people to edit the same document at the same time • How it did it • Converted a physical document into a virtual document with multiple physical sections • Merged section changes back into main document so main document was always current

31. Document Section Manager App • Implementation • Document section changes trigger an asynchronous event receiver • Event receiver invokes a custom Windows service to process document sections • Windows service invokes MS Word through PIA / DCOM to perform the section merges • Another Windows service uploads the processed document back to SharePoint

32. DSM Process Flow

33. DSM’s Place in SharePoint Farm

34. Issues with the Architecture • All processing ran on each web front end • Deployment was a nightmare • Deploy and configure MS Word to each WFE • Deploy and configure custom Windows service to each WFE • System reliability was sub-optimal, mostly due to constant DCOM issues

35. Issues with the Architecture • Application did not work when SharePoint was installed on Windows Server 2008, due to DCOM differences • Application did not work when SharePoint was installed on Windows Server 2003 x64 • Document processing put substantial load on WFEs

36. Issues with the Architecture • Main issue: The Document Section Manager was too intrusive in the SharePoint system. This caused the deployment, configuration, stability, and maintenance issues. • This is an architectural issue that required a fundamental redesign of the application.

37. Re-architecting the DSM 1 34 4, 6 3 4 4, 6

38. Re-architecting the DSM • Document Assembler Service remains mostly unchanged but is moved from SharePoint farm to external application server on external hardware. • Uploader Service remains mostly unchanged but is moved from SharePoint farm to external application server on external hardware.

39. Re-architecting the DSM • New SOAP Service interface created for Document Assembler and Uploader services on external application server. • Event receiver remains mostly unchanged but now invokes the Document Assembler Service remotely rather than locally. • MS Word is removed from SharePoint WFEs.

40. docBlock’s Place in SharePoint 10% 90%

41. docBlock Client Process Flow

42. docBlock Service Process Flow

43. Tech Benefits of Re-architecture • Deployment is 80% easier • 90% less code running on SharePoint WFEs • MS Word no longer required on SharePoint WFEs • 95% less load on SharePoint WFEs • 75% increase in processing reliability

44. Non-Tech Benefits • Easier to find developer resources: not everyone working on the app needs to be a SharePoint expert. • Easier to sell the application: very little push-back from IT concerned about what gets installed on SharePoint farm. • Companies requesting services work to help them similarly re-architect their applications.

45. Non-Tech Benefits • Application can now be distributed as a virtual or physical appliance. • Application can now be hosted “in the cloud.”

46. Summary • If processing doesn’t require SharePoint, don’t run the processing in SharePoint. • Minimal intrusion into SharePoint yields a more stable, reliable, performant platform. • Execute slow code asynchronously with respect to the web UI. • Utilizing SOA service interfaces yields more manageable, upgradable, and supportable code.

47. Additional Resources • Document Section Manager code: http://www.codeplex.com/WSSDocMan • Things that Should be Easy http://thingsthatshouldbeeasy.blogspot.com • Black Blade Web Site http://www.blackbladeinc.com

48. Thank you for attending! Please be sure to fill out your session evaluation! Eugene Rosenfeld DEV346, PM346: Better SharePoint through Service Oriented Architecture and Development