Adaptive Infrastructure: Identifying and Using Pattern

Adaptive Infrastructure:Identifying and Using Pattern Sjarif Abdat (sabdat@cs.ui.ac.id) Universitas Indonesia MTI - Perencanaan Infrastruktur

Reference: • The Adaptive Enterprise: IT Infrastructure Strategies to Manage Change and Enable GrowthBruce Robertson and Valentin SribarAddison Wesley, 2002 MTI - Perencanaan Infrastruktur

Instead of developing infrastructure from the ground up for each new application, businesses can leverage a limited number of reusable infrastructure patterns. Pattern-based MTI - Perencanaan Infrastruktur

Designing for the future • It is impractical and inefficient to allow infrastructure and business process to grow unchecked with separate decisions and implementations each time an applications is deployed • Recognizing patterns allows you to reuse existing technology, business process, and the skills of existing personnel • The pattern set must be variable enough to cover all major application requirements, but limited in number to deliver the benefits of a simplified environment. • Patterns must also provide sufficient detail to enable the design and implementation of high-quality, end-to-end solutions with minimal customization. MTI - Perencanaan Infrastruktur

Relating Patterns to Platforms MTI - Perencanaan Infrastruktur

Relating Patterns to Platforms • Pattern are the information, insight, and experience – the “what is” and “how to” – that are common to an existing class of application • This information is captured in a form that makes it easier to reuse with future applications of the same class. • Patterns also capture experience and best practices for projects that result from real apps requests by business • They emphasize end-to-end reuse in a logical rather than physical. • Patterns deal with applications, platforms deal with technologies • Platform requirements are IT-driven, pattern requirements are more directly driven by business needs • Patterns are designed to address application requirements by providing a proven and tested solution MTI - Perencanaan Infrastruktur

Why So Few Patterns • 80%-90% of all infrastructure needs can be consistently accommodated by a set of less than ten patterns • The specifics of these patterns may change to accommodate new technologies • It should be small • When you have too many options, complexity becomes a problem. • It’s difficult to maintain expertise in too many areas, designing a limited set of patterns will help you keep your focus. • Keep in mind the 80/20 rule and leave the last 20% of differences outside of the standards • Are patterns just for new infrastructure? MTI - Perencanaan Infrastruktur

The Starter Kit • META Group recognize a set nine starter patterns in three basic interaction types: • Transact patternsApplications in which business data is written and stored for long period of time, such as online customer orders and other transactions • Publish patternsApplications with read-only data, such as online marketing information • Collaborate patternsApplications where information contained in files and documents is shared between two or more users, such as a product design document shared by a development team. MTI - Perencanaan Infrastruktur

The Starter Kit MTI - Perencanaan Infrastruktur

Pattern Maturity MTI - Perencanaan Infrastruktur

Transact Patterns • Transact pattern include any application that “writes” structured information to a system or a data set • Transactions let you Create, Read, Update, and Delete (CRUD) • A DBMS is usually a standard component of this patterns • Require ACID properties (Atomic, Consistent, Isolated, and Durable) • Sub-classified using the following logical tiered structured: • 1-Tier Transact,or Host Transact, where all data processing is done on the host system in a monolithic application • 2-Tier Transact, which typically bring the presentation and business logic to the client side, leaving the database management on the server • 3/N-Tier Transact,which, in addition to separating data management like 2-Tier, separates the presentation logic from the business logic MTI - Perencanaan Infrastruktur

1-Tier Transact Pattern • This patterns includes batch applications or online transaction processing (OLTP) without logical abstraction between the presentation, application, and data logic. • Most mainframe implementations fall into the 1-Tier Transact Pattern • The entire application, including the database, business logic, and presentation logic, is hosted on a single, high-end server • Great for serving large user populations with massive transaction rates, fast response times and high availability MTI - Perencanaan Infrastruktur

1-Tier Transact Pattern MTI - Perencanaan Infrastruktur

1-Tier Transact Pattern • Benefits: • Availability. The host pattern’s availability is common strength familiar to infrastructure developer • Scalability. This patterns excels in the number of concurrent users it can support • Reliability. This pattern is also the most reliable. System recovery problems are often handled automatically MTI - Perencanaan Infrastruktur

1-Tier Transact Pattern • Weaknesses: • Expensive. Users have to pay a premium price for mainframe hardware and software licenses. • Inflexible. Changing the system requires a significant engineering effort. • Lacking agility. The shared nature of applications can generate significant interrelated workload dependencies. • Lack of applications. Package application support is declining • Resource-intensive. MTI - Perencanaan Infrastruktur

1-Tier Transact Pattern • Best Practices: • Put wrappers around legacy host code to encapsulate the components and make them reusable • Avoid using this pattern for new applications • Don’t use components optimized for this pattern in other patterns • Minimize the investment in upgrading this pattern • Futures: • No new applications should be based on this pattern. MTI - Perencanaan Infrastruktur

2-Tier Transact Pattern • This pattern involves a smart PC on the desktop communicating directly with a back-end database server • It also includes web servers that intertwine CGI/ASP/JSP presentation and application logic • The heavy integration of application logic and screen presentation logic make app-to-app integration difficult • Best suited for implementation on a LAN, in an environment where bandwidth are high and latency are low • SQL statements and database access middleware typically perform poorly over higher latency and low bandwidth WAN • How can you tell if an application is 2-Tier or 3-Tier? MTI - Perencanaan Infrastruktur

2-Tier Transact Pattern MTI - Perencanaan Infrastruktur

2-Tier Transact Pattern • Benefits: • Speed. The best choice for applications being deployed in a high-risk, volatile business environment, where business cycle are measured in weeks rather than months or years • Rapid development. Delivery can take place at a rapid clip MTI - Perencanaan Infrastruktur

2-Tier Transact Pattern • Weaknesses: • Lack of scalability • Security vulnerabilities • Sourcing limitation • Presentation limitation • Lack of API • Hidden Cost MTI - Perencanaan Infrastruktur

2-Tier Transact Pattern • Best Practices: • Limit this pattern to small-scale applications in a LAN-based environment • Be wary of adapting this pattern to WAN-based clients using a terminal server • Don’t use the Web version of this pattern for large-scale applications • Take advantage of this pattern for rapid prototyping MTI - Perencanaan Infrastruktur

2-Tier Transact Pattern • Futures: • Prototyping and piloting will continue to use this pattern • Smart PC version wanes • ISVs slowly migrate from this pattern to N-Tier Transact MTI - Perencanaan Infrastruktur

3/N-Tier Transact Pattern • This pattern consist of a thin client carrying presentation-logic only, communicating with a client-neutral, server-based applications, and a back-end database server • The most scalable and flexible client/server transaction pattern • Users can be highly decentralized (WAN-friendly) • Flexible to integrate with other applications or points of interaction. • Offers the greatest distribution of processing functionality in a transactional environment • Clients in this pattern tend to be thin, and shared business logic can be supported from multiple point of interaction MTI - Perencanaan Infrastruktur

3/N-Tier Transact Pattern MTI - Perencanaan Infrastruktur

3/N-Tier Transact Pattern • Benefits: • Scalability. Distributing the processing load on multiple server tiers provides significant scalability • Requires more design effort to partition application functions and limit dependencies • Security. Web SSO products provide multi tier security with role-based resource authorization • Presentation. All user interfaces can leverage the same application logic, both logical and physical • API. Ability to use APIs independently of the application’s presentations is the key advantage • Reduces the cost of EAI because of sharing APIs between applications reduces the time and expense required to integrate the apps MTI - Perencanaan Infrastruktur

3/N-Tier Transact Pattern • Weaknesses: • Need customization. Requires significant customization at application and tool levels • Misunderstood. Web server based is not always 3/N-Tier • Expensive. System integrators maintain that integration and implementation expenses will exceed hardware and software by a 4-to-1 ratio. MTI - Perencanaan Infrastruktur

3/N-Tier Transact Pattern • Best Practices: • Develop Transactional Integration services to match 3/N-Tier applications • Work closely with application developers in the early stages, as they build or buy tools • Use scale-out stateless farm architecture • Use experience gained from existing OLTP DBMS and server OS/hardware platforms, even if other server tier platforms are different • Do not replicate OLTP data to scale • Take advantage of 3/N-tier pattern benefits when scaling and integrating with other applications • Develop a Test Lab, including equipment and procedures for investigating the infrastructure impact of various patterns and services MTI - Perencanaan Infrastruktur

3/N-Tier Transact Pattern • Futures: • Continue to be the preferred choice for transactional applications • ISVs slowly migrate to this pattern • Better app development tools supporting this pattern MTI - Perencanaan Infrastruktur

Stateless Farm Architecture and Scale-Up Design • Scalability and availability can be approached using: • Scale-up • Scale-out • Stateless Farm Design: • Application designer understand how to build statelessness into Web applications • HTTP makes it possible -> IP redirection, to distribute the load to number of Web servers • For transactional apps, requirement still exists to maintain user session information. • Application state is achieved using cookies, decorated URLs, or persistence mechanisms provided by the load balancers. • Storing state in the database is the most safe and reliable design, even it it limits performance MTI - Perencanaan Infrastruktur

Stateless Farm Architecture and Scale-Up Design • Clustering Database • General-purposes commercial database apps are more difficult to parallelize, given the need to maintain state centrally • Best practice is to use clusters primarily to enhance DBMS server availability, not to extend scalability • Scalability issues will be addressed by new software products, such as Oracle 9i Real Application Cluster (RAC), and new hardware technologies. MTI - Perencanaan Infrastruktur

Stateless Farm Architecture and Scale-Up Design • Clustering Other Servers • To scale Web applications can be addressed by implementing DNS round-robin load balancing. • Simple DNS policies are not sophisticated enough to avoid unhealthy servers in cluster • Vendor have developed server load balancer (SLB) and network load balancer (NLB) products to improve reliability, performance, and manageability of server clusters • They distribute the processing load based on various policies, including round-robin, least connections, and CPU load • The three core advantages to NLB solutions are: • Increased redundancy • Better performance • Easier management MTI - Perencanaan Infrastruktur

Publish Patterns • Publish patterns include most applications that provide read-only and data analysis access • Scaling these infrastructures is usually a matter of replicating the data as much as needed to support volume and reliability • Publish applications are the most easily and commonly outsourced (when the goal is to serve very large number of users over large geographies) • Sub-classified as: • Client/Server Publish, usually leveraging complex GUI analysis tools • Web Publish, where the data-access is initiated in a browser • Stream Publish, an emerging pattern that covers near real-time audio and video downloads MTI - Perencanaan Infrastruktur

Client/Server Publish Patterns • This pattern is defined by the use of a smart PC, such as a sophisticated business intelligent client, with session-oriented protocols. • Best used for implementing sophisticated data analysis capabilities for small, well-defined user base. • Applications that fit this pattern are online analytical processing (OLAP) tools and reporting-intensive application • This client/server pattern is the most popular Publish pattern inside the enterprise MTI - Perencanaan Infrastruktur

Client/Server Publish Patterns • Fig 3.8 MTI - Perencanaan Infrastruktur

Client/Server Publish Patterns • Benefits: • Highly adaptive. It reduces the number of interface and the number of extractions that have to be performed on the operational system • Weaknesses: • Resource-intensive. Client code size and bandwidth requirements make this pattern less easily scalable and deployable (tend to be restricted to LAN use) • Unpredictable demand. It is not uncommon for the answer to a query to be 100 to 200 megabytes or more in size MTI - Perencanaan Infrastruktur

Client/Server Publish Pattern • Best Practices: • Developers must pay close attention to modeling in order to maintain system performance • Manage data access closely, both educated and uneducated users may be accessing these applications, and a tiny mistake in an SQL command can bring a data mart to its knees • Consider using a separate OLAP server to optimize performance by pre-calculating and storing results for common queries • Use this pattern only for data-intensive, read-only applications • Due to the bandwidth requirements, this type of pattern should typically be used in a LAN environment only • You should use one of the Transaction pattern to handle the write-back/change requirements resulting from analyses • Create a unified analytic service using a Data Warehouse/Operation Data Store (DW/ODS) • Use publish-specific tools such as OLAP analysis tools rather than VB or PowerBuilder MTI - Perencanaan Infrastruktur

Client/Server Publish Patterns • Futures: • Client-based slicing and dicing of data will remain popular, especially for disconnected users • Convergence of Web and Client/Server publish with ability to move transparently between them MTI - Perencanaan Infrastruktur

Web Publish Patterns • This pattern is defined by the use of an HTML browser and HTTP protocol to enable read-only access to structured XML or HTML documents • More flexible than Client/Server Publish pattern in supporting large, less well-defined user groups • Most popular for applications reaching beyond the enterprise boundaries • Many Web sites today use a combination of Publish and Transact patterns • However, infrastructure planner should separate these distinct functions to get a better picture of the infrastructure that is needed for each activity MTI - Perencanaan Infrastruktur

Web Publish Patterns MTI - Perencanaan Infrastruktur

Web Publish Patterns • Benefits: • Due to the maturity of this pattern, many solid tools are available for developing read-only Web publishing apps • Speed of initial development • Wide availability of Web hosting service provider • Scalability is very simple to achieve by scale out, rather than scale up (stateless farm architecture) • Performance-enhancement solution: • Network load balancer and multi-site balancing • Compression • Caching (reverse caching in DC, client caching, intermediate ISP caching) • Content delivery network (CDN) services like Akamai • Traffic or rate shapping MTI - Perencanaan Infrastruktur

Web Publish Patterns • Weaknesses: • Difficult to manage scale or load on e-Business Web sites that fit this pattern • Difficult to change the presentation logic from Web to non-Web presentations without completely rebuilding the application • Publishing to multiple points of interaction is still a work in progress MTI - Perencanaan Infrastruktur

Web Publish Pattern • Best Practices: • Focus on availability first, then on throughput and response time • Plan for at least two-way redundancy, even for internal systems • Examine Web-hosting service providers for designs and sourcing • Consider using Content Delivery Network (CDN) services for static content, but only for very large audiences • Scale out (duplicate and replicate) Web and data servers with load balancers • Support multiple point of interactions (or at least plan on this in the future • Manage, limit, or disallow direct access to transactional data • Avoid running transactional applications on infra-structure designed for the Web Publish pattern • Create a unified analytical integration service via DW/ODS MTI - Perencanaan Infrastruktur

Web Publish Pattern • Best Practices (cont): • Don’t use expensive Online Transaction Processing database • Leverage less expensive hardware and software in general for data storage component. • Don’t neglect security issues. Be sure to guard against data defacement and denial of service attacks • Coordinate with marketing personnel and other business decision-makers, to estimate the impact of business cycles, marketing campaigns and other business variables on application workloads • Design for outsourcing. Even if you decide not to outsource now, this is an increasingly cost-effective option for many businesses, and may be advantageous in the future • Install response time and load monitoring services for business-critical applications • Take a service-centric approach. Once you have optimized this pattern, it can be widely reused with minimal or no customization. MTI - Perencanaan Infrastruktur

Web Publish Patterns • Futures: • N-Tier publishing based on XML, XSLT, and XQL • Convergence of Web and Client/Server publish with the ability to move transparently between them MTI - Perencanaan Infrastruktur

Stream Publish Patterns • This pattern is used for real-time publishing of streaming content to “multimedia player” • Streaming plays the file in near real-time as it download • The latency requirements of real-time multimedia delivery are critical • Stream publish is only one-way • Streaming multimedia requires great changes in user behavior, Web site loads, and network configuration MTI - Perencanaan Infrastruktur

Stream Publish Patterns • Benefits • Usability. Streaming content is much easier to use simply because little interactivity is involved • Business value.Streaming content can make web sites more dramatic and interesting (increase customer retention) • Clear choiceVendor solution are coalescing slowly, which makes specifying solutions easier to do MTI - Perencanaan Infrastruktur

Stream Publish Patterns • Weaknesses: • Bandwidth-intensivewhich also can mean expensive • Unpredictable resultUnpredictable and variable network can affect the streaming quality • Unpredictable demandE-Business or consumer-focused sites normally have an unknown external user base. MTI - Perencanaan Infrastruktur

Stream Publish Pattern • Best Practices: • Find a qualified service provider to host large-scale content streaming. Be sure they are equipped to handle dynamic bandwidth needs quickly and efficiently during peak demand • Don’t use gratuitous streaming content on web pages, since it complicate infrastructure requirements while perhaps providing little business value • Monitor usage carefully. Traffic spikes are common with streaming content, and poor quality of service can largely negate the business benefits • Use traffic shaping to control the impact of streaming content on other Web applications • Use multicasting whenever possible for internal viewing of popular streaming content • Internally, consider using scheduled replication of content to servers that are logically and physically closer to large user populations, instead of running streams over WAN. • Monitor corporate intranet sites for overuse of streaming media links MTI - Perencanaan Infrastruktur

Stream Publish Patterns • Futures: • Increasingly advanced compression algorithms will improve the streaming quality • Streaming publishing will evolve into multicasting • CDN are adding streaming support to their distributed caching system MTI - Perencanaan Infrastruktur

Collaborate Patterns • The collaborate pattern involves peer-to-peer communication, usually centered on shared documents or groups of documents • Driven by the need to share and modify calendars, text documents, and design drawings, along with growth of more advanced collaboration techniques using streaming audio and video • True collaboration requires multiple human beings working directly together, through the asynchronous time between their activities • It is essential to have unified, highly centralized, and singular infrastructure solutions • Differentiated patterns in this category include: • Real-time collaborate pattern • Store-and-forward collaborate pattern • Structured collaborate pattern MTI - Perencanaan Infrastruktur

Adaptive Infrastructure: Identifying and Using Pattern