1 / 44

Application and System Monitoring Using JBoss Application Server

Application and System Monitoring Using JBoss Application Server. Nicholas Whitehead Robert Quinn Patrick Urban March 1, 2005. Agenda. Holistic Application and System Performance Monitoring Challenges In Collecting and Analyzing Performance Data

duer
Download Presentation

Application and System Monitoring Using JBoss Application Server

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Application and System Monitoring Using JBoss Application Server Nicholas Whitehead Robert Quinn Patrick Urban March 1, 2005

  2. Agenda • Holistic Application and System Performance Monitoring • Challenges In Collecting and Analyzing Performance Data • A Proposed Architecture for a Performance Management Data Platform • An Implementation of the Platform • Questions (and tentative answers)

  3. Performance Monitoring & Analysis

  4. Performance Monitoring & Analysis • Heterogeneous Monitoring Targets • Network (LAN, WAN, Routers, Switches) • Hosts (CPU, Memory, Disk, Threads) • Various Operating Systems • Application Server (JMX, Instrumentation, Pooling) • Middleware (Message Throughput) • JVM (Memory, GC) • Database (Exec Plans, Memory, Concurrency, Locks) • Files (ad nauseum) • Applicaton Specific Structures

  5. Challenges • Phone Call: “System is slow” Why and what do you do about it ? • Prevention: Where are your weak spots ? • Planning: How many servers do I need to get us through the next n months ? • What is the relationship between system load and: • Number of users • Day of the week • Month of the year • My astrological sign ?

  6. So Many Monitor Points, So Little Time

  7. Techniques For Performance Monitoring • Commercial Enterprise Frameworks • Expensive • Not Always Complete • Manual Data Collection & Merging • No Real Time Vizualization • Inflxexible • High Maintenance • Adhoc Monitoring • No Cross Referencing • Multiple Monitors Load Monitored System

  8. A Cascading Performance Emergency • Users Report System is slow. Where’s Waldo ?

  9. Anatomy of a Perfomance Hit There He Is !

  10. A Consolidated Platform For Peformance Data Acquisition

  11. Key Attributes • Central, globally available clearing house for performance and system status data. • Scale to large numbers of data sources and client dashboards. • Supply dashboards, reports and status summaries to dashboards. • Provide platform for post processing acquired performance and status data. • Monitor Proxy: Multiplex multiple monitor clients to one set of predictable and controllable data acquisition streams. • Support Correlation of Heterogeneous Metrics

  12. JMX – Perfect For The Job • Extensible and Standardized Container for In-Memory State • Robust, Flexible API • Rich and Dynamic Meta Data • Organized Domain Structure • Built In Monitors • Attribute Change and other Notifications • Wide Support for 3rd Party Tools • Rich Lifecycle Control

  13. Services • Simple State: Holds the last reading for a specific metric. e.g. The last CPU utilization reading on webserver1. • Data History Buffer: The last n readings of a specific metric. Shows flat, upward or downward trend. • Persistence: Store performance data for reporting, historical analysis and forecasting. • Charting: Real time data visualization. • Alerts: Alerting services for specific conditions, monitor triggers or failed readings. • Boundary Tracking: Track high, low and average values for performance statistics. • Data Summarization: Aggregation of large quantities of data into understandble summarized reportlets.

  14. Simple State & State History Buffer • JMX Attributes Contain the Latest Scanned Value of a Performance Metrics • History Buffer Show the Last n Readings.

  15. Persistence & Reporting • Store performance data for: • Reporting • Forecasting • Capacity Planning • Trend Analysis

  16. Charting • Visualization of real time performance data. • Side by side or large and rich displays containing many graphs for visual correlation.

  17. Alerts • Alerting by email and pager when established threshholds are crossed. Nagios Monitoring System Service Status Screen www.nagios.org

  18. Boundary Tracking • Track boundaries of selected metrics • High • Low • Average

  19. Summarization & Aggregation • Large quantities of performance data can be aggregated and summarized to an understandable reportlet. \\IISWEBSERVER01\Thread(inetinfo/0)\Thread State: 5 \\IISWEBSERVER01\Thread(inetinfo/0)\Thread Wait Reason: 0 \\IISWEBSERVER01\Thread(inetinfo/1)\Thread State: 5 \\IISWEBSERVER01\Thread(inetinfo/1)\Thread Wait Reason:6 \\IISWEBSERVER01\Thread(inetinfo/2)\Thread State: 5 \\IISWEBSERVER01\Thread(inetinfo/2)\Thread Wait Reason: 0 \\IISWEBSERVER01\Thread(inetinfo/3)\Thread State: 5 \\IISWEBSERVER01\Thread(inetinfo/3)\Thread Wait Reason:6 \\IISWEBSERVER01\Thread(inetinfo/4)\Thread State: 5 \\IISWEBSERVER01\Thread(inetinfo/4)\Thread Wait Reason: 0 \\IISWEBSERVER01\Thread(inetinfo/5)\Thread State: 5 \\IISWEBSERVER01\Thread(inetinfo/5)\Thread Wait Reason:6 \\IISWEBSERVER01\Thread(inetinfo/6)\Thread State: 5 \\IISWEBSERVER01\Thread(inetinfo/6)\Thread Wait Reason: 0 \\IISWEBSERVER01\Thread(inetinfo/7)\Thread State: 5 \\IISWEBSERVER01\Thread(inetinfo/7)\Thread Wait Reason:6

  20. Helios – An Implementation • An Open Source Implementation • Powered by JBoss Application Server • Performance Data Service & Clearing House • Heterogeneous Data Sources • Extensible Monitors and Post Processing • Customizable and Easy to Configure Dashboards

  21. Evolution of Helios Original State of Affairs • Multi-tool data collection • Manual Data Merge • No Proxy. All monitor clients put load on servers.

  22. Evolution of Helios Simple Automation • Automated multi-tool data collection • Daily merge • Manual Reports and Charts

  23. Evolution of Helios Simple Dashboard • Numeric and character data only. • Web page console. • Limited data.

  24. Evolution of Helios Evolving Architecture • Implementation of MBean Proxies. • MBean generated graphics. • Single console still requires rebuild and redeploy to change.

  25. Helios Version 0.1 New Architecture • Dynamic Dashboards that can be saved and shared. • Dashboard display widgets can be resized and moved. • Configurable, extensible and hot deplyable data source monitors. • Extensible post-processing using plugins. • External client API for push data. • More…..

  26. Helios Architecture • Extensible and Robust JMX (Remote Notification Really Work) • Hibernate Managed Persistence (& HARs !!) • Scalable and Clusterable Services • Multi-Protocol Remoting • Great Platform To Work On • Free Platform To Work On

  27. Monitors - Overview • Implemented as a JMX MBean • Acts as a proxy to a performance data source • Performance data is available as attributes • Configured using XML in jboss-service.xml • Can be configured to pass results to a collection of plugins • Broadcasts notification of new scans and attribute changes • Active Monitors scan a performance data source (Pull) • Passive Monitors receive performance data from an External Client Agent (Push)

  28. Monitors - Types • Currently Available • JMXMonitor - Monitors JMX Attributes • WindowsMonitor - Monitors Windows Performance Manager Stats • SQLMonitor - Monitors JDBC Data Sources • YahooFinancialMonitor - Monitors Yahoo Finance Securities • PassiveMonitor – Creates attributes for holding results from a remote agent • JRat Java Byte Code Instrumentation – Acquires Java Method Elapsed Times • In Progress • LinuxMonitor: Monitors Linux /proc file performance data • JSR77 Autoconfiguring Monitor • Text Stream Reader • File Scraper (Log Analyzer) • Telnet Data Acquisition • SSH Data Acquisition • Planned • OpenArm Helios Mediator: Application Response Measurement Data Acquisition • GkRellm Data Acquisition • NetPerf Data Acquisition • SNMP Data Acquisition • PING Monitor

  29. Monitors – Sample Configuration

  30. Plugins - Overview • Configured for each Monitor • Receives a copy of each scan result • Extends a monitor’s functionality • Configured using XML in jboss-service.xml • Can create a new result and register it an attribute in the Monitor (back register)

  31. Plugins - Types • Currently Available • PersistScalar – persists scalar results • PersistSimpleArray – persists simple arrays • RollingHistory – Tracks ‘n’ previous versions of a scan • VariaGraphPlugin – Creates line graphs (JPG, PNG or SVG) • TrackAverageNumber- Computes averages • BasicSQLHTMLFormatter – Formats ResultSets as HTML • J2EEStatsHTMLFormatter – Formats JSR77 results as HTML • TrackHighNumber – Maintains a maximum value • TrackLowNumber – Maintains a minimum value • TrackAverageNumber – Maintains the average value • In Progress • Graphing Library • Nagios External Event Updates • File Logging Event • Planned • Customizable Event Broadcaster For Integrating to 3rd Parties • Customizable HTML Renderer

  32. Plugins – Sample Configuration

  33. Subscription Service - Overview • Allows a client to subscribe to and receive updates for Monitor attributes • Implemented as an MBean • HTTP Façaded provide for web clients • Clients establish a “session” • Server Side Proxy manages client’s subscriptions • Subscription listen on the notification bus and filter attributes notifications based on client’s subscription • Client can subscribe/unsubscribe to attributes • Client “polls” and receives all available updates

  34. Subscription Service - Components

  35. Subscription Service – Basic Use Cases

  36. Visualizing The Data Dashboard Manager • A WYSIWYG tool for managing and sharing views of your information real-time via “Dashboards”.

  37. The Goals Of Helios Dashboard Manager • Easy Creation • Easy Viewing • Easy Sharing

  38. Easy Creation • All available “Monitors” displayed for selection in a tree style view called the Dictionary

  39. Easy Creation Cont. • Drag and Drop style dashboard creator • Supports resizing and overlapping

  40. Easy Viewing • View dashboards using a browser • Currently supports IE 5.5+ and Firefox 1.0 • Supports full-screen viewing and slide show style for multiple dashboards. • Live Data Updates. Supported output type include… • Single Value • Array • Rich HTML • Image .gif, .jpg, .png • SVG • View on any platform via Firefox • No additional client installs or plugins required • Clean presentation ready look

  41. Easy Sharing • Dashboards support custom HTML content via “Labels” that allow the creator to annotate a dashboard as necessary. • Underlying XML descriptor allows for dashboard definitions to be imported and exported between Helios server instances.

  42. Future Plans • Library of Client-Side Rendered Graphs via SVG • More Dashboard Customization Options • Look and Feel • Behavior • Security/Dashboard Ownership Integration

  43. Client API • Protocols: • RMI • JMS • Simple XML over HTTP • XML over HTTP allows non-Java scripting languages such as Perl and TCL to perform remote data acquisition for Helios. • Remote API support for: • Creating New Passive Monitors. • Creating New Monitor Attributes • Updating Monitor Attributes

  44. Thanks ! Helios: Greek God Of The Sun. He sees everything that happens and is often called upon to shed light on events. https://helios.dev.java.net/index.html Helios. See All. Hear All. Understand All.

More Related