Feeding Mash-ups with TDI 7 May 2007

1. Feeding Mash-ups with TDI7 May 2007 Eddie Hartman Johan Varno TDI Development Team

2. Highlights Using TDI as a general purpose mash-up & syndication integration server that provides RSS and Atom feeds based on data from multiple back-ends with custom transformation.

3. Agenda • Overview • Situation • Challenge • Proposed approach • TDI fundamentals • Live demonstration

4. Overview • Situation • RSS/Atom is a flexible data distribution/request mechanism. Furthermore, its simplicity has fuelled the advent of mash-ups. • Orchestrating the mash-ups and consuming RSS/Atom feeds is well underway all across IBM. • Challenge • Getting data into feeds or available as Web Services, although not complex in itself, is not as simple in a scalable manner. This is largely due to the huge permutation of data sources and API’s where available information needs to be enriched, transformed and augmented before being driven into an RSS or Atom feed. • Proposal • TDI provides a general-purpose integration service for hosting feeds. The content of these feeds is provided by back-end ‘service AssemblyLines’ that use TDI Connectors to access data sources, manipulate retrieved data and create the RSS/Atom payload. –these service AssemblyLines can also be published as Web Services.

5. TDI architecture TDI server Files LDIF, DMSL Execution run-time environment XML, CSV Logging & tracing Script, Java Event service LDAP Integrated development environment JavaScript Engine Web Services AssemblyLine API SAP, Siebel Peoplesoft ACT event processing GLA, RAC HTTP, TCP Functions & libraries Domino CCMDB, CEI Web administration and monitoring Action Manager: Automatic monitor, control, and alternate action POP, IMAP SNMP, JMX Pooling, threads & tombstones Delta detection JDBC Persistence service: SystemStore SystemQueue JMS / MQ Commandline interface Java VM Windows, Linux, AIX, iSeries, zOS, Sun, HP

6. AssemblyLine pipeline Automated behaviors and flow logic reduce the work needed to build secure, enterprise-ready integration solutions. These internal workflows can easily be modified and extended. In addition, each AssemblyLine is a custom dataflow pipeline that is built quickly and visually using the TDI editor GUI.

7. RDBMS JDBC connector LDAP connector JMS connector ”any” connector TDI: Mashup & syndication integration server TDI AssemblyLine ”service Controller” Service AssemblyLines Incoming RSS or Atom (or Web Service)request HTTP (Web Service) connector service connector Custom logic, such as registration and provisioning of services OMP Any number of ”service AssemblyLines” can provide services that may be accessed through the multi-threaded ”Service controller”. Each Service AL leverages the connectors and capabilitites of TDI to create advanced feeds and services that provide transformation, augmentation, enrichment of the data from any number of connected sources.

8. 'People' AL JDBC connector 'Changes' AL LDAP connector Domino Changeconnector Data filtering logic ’Mail’ AL LDAP connector Data filtering logic Data filtering logic Reads from a database table, joining more attributes from an LDAP server and filtering out invalid data Demonstration Overview TDI Picks up changes to Person documents in Domino that occurred since the last time this info was requested. 'Feed Server' AL Incoming RSS or Atom request HTTP connector Dispatcher fordelegating work toservice AssemblyLines Build and reply with RSS feed XML based on returned data. Returns LDAP entries found using the specified mail filter The service AssemblyLine to use is specified as a parameter in the RSS URL (http://myrssfeed/rss?AL=People) making the solution easily extendable. For Atom, you replace ”/rss” with ”/atom” and conversion is handled automatically.

9. Demo

10. Google connector REST ”mashup” server with TDI TDI AssemblyLine ”REST server” Get events from EVDB HTTP connector Incoming REST call http://<server>/events?keywords=music&location=Austin&date=Future TDI returns Keyhole Markup Language for use in HTTP connector For each EVDB event returned Get content on each event from Google Create KML and add to return payload Provided location (”Austin”), event type (”music”) and date (”future”) in a URL, this REST server will return suitable events from EVDB, augmented with content from Google, formatted in KML format for viewing with Google Earth.

11. What is TDI? • TDI is an integration framework • Graphical Development Environment to build and test solutions • Run-time Server for deployment • Web-based Administration and Monitoring Console for management (AMC) • Java API & Commandline utility for automation and embedding • Represents a unique rapid integration development methodology • Provides unparalleled control over data flow and transformation • Remote (over IP) development and test from your laptop • Supported on most major platforms, including iSeries and z/OS.

12. Evolution of Infrastructure • TDI Approach • Build & extend integration incrementally • Phase in (and out) systems in controlled stages • Enable & leverage standards • Solution Framework & Trench Tool • Accelerating POCs/POTs • Deploying software - data migration, validation, washing and conversion • Extending the reach of installed systems • Augmenting monitoring (e.g. log scraping, polling APIs/sockets, ...) • Creating new data services

13. TDI bundled with IBM products (partial list) • Lotus Domino 8 & Lotus Connections • WebSphere RFID Information Center • Tivoli Identity Manager (Enterprise & Express) • CCMDB & TADDM • Tivoli Access Manager (TAMeb) • Tivoli Directory Server • Federated Identity Manager

14. Community Resources • Community websites • Examples, video tutorials, reusable assets, documentationhttp://www.tdi-users.org(IBM internal) https://w3.webahead.ibm.com/w3ki/display/TDI • Mailing list (news, updates, events) • Newsgroups • TDI Newsgroupnews://news.software.ibm.com/ibm.software.network.directory-integrator • IBM internal newsgroupnews://ibmforums.ibm.com/s.directory !! Participate and Share !!

15. End