A pplication R edundancy T ool

1. Application Redundancy Tool A.R.T. CS 495 Fall 2005 Kristi Olson

2. Description • A.R.T. is an system of hot standby for applications. • Designed for applications which need to run continuously. • Intended for use with applications which require socket connections.

3. Internship • Internship with GCI’s Network Support Group, Operations System Support. • NSG OSS responsibilities: • Provision phone service, calling cards, internet services. • Internal Support (data collection) • Network Monitoring

4. Applications supporting these services • Homegrown. • Most run around the clock. • Some applications are mission critical. • Loss of productivity when applications are down. • No formal system of redundancy. • Lack of existing product to buy. • Lack of funds.

5. Applications continued • Methods of transmission vary widely. • Fiber optic • Reliable, fast • Satellite • Prone to weather related outages, inherent 600 ms delay. • Microwave • Even more prone to weather: fog, rain, or even a hot day causes outages.

6. Applications continued • Socket connections: • Some applications establish one or more “permanent” socket connections. • Others repeatedly establish multiple “temporary” socket connections.

7. How to provide redundancy? • V.R.R.P. • Virtual Routing Redundancy Protocol • System of dynamic redundancy. • One router is designated master. • Other routers are backups. • Uses multicasting.

8. V.R.R.P. and A.R.T. • Establish “Master” and “backup” instances of the application. • Identical except for a configuration file. • The backups loop continuously listening for status polls from the Master. • If the Master stops sending polls, the backup comes online. • Put the application instances on the same multicast group.

9. Requirements • Minimal modifications to existing code. • Configurable for any type of application and transmission protocol. • Reliability: • A backup can not come online prematurely, nor can it come online too late. • The actual switch over should be minimally service affecting. • Applications should take control or release sockets accordingly.

10. Configuration • Priority: • The instance with the highest priority becomes the master. • Broadcast Interval: • How often the application sends and listens for status polls. • Allowable Missed Polls: • Multicasting uses UDP. • Less reliable transmission technologies.

11. Status Polls • Application Name • Each application only reads messages pertaining to itself. • Timestamp: • Latency is possible. • A.R.T. ignores old messages. • Priority

12. Putting it all together • Configuration file: • Contains instance information. • A.R.T. Perl module: • Defines multicast groups. • Evaluates status polls. • Other related overhead • Modifications to existing code: • Where to poll. • Don’t want to poll too often, or not often enough. • Encapsulate socket connections. • Re-factoring opportunity

13. High level data flow

14. A.R.T. Monitoring • Email Alerts: • Notify admins should a switch over occur. • Web Page: • Traffic Light: • Green Light - Master is working. • Yellow Light – Backup is listening. • Red Light - application is down.

15. A.R.T. Monitoring Web Page

16. A.R.T. • Questions?