Evaluating web software reliability
1 / 35

Evaluating Web Software Reliability - PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

CSI518 – Group 1. Evaluating Web Software Reliability. By Zumrut Akcam, Kim Gero, Allen Chestoski, Javian Li & Rohan Warkad . Agile Development. What is Agile ? Focus on business value Collaboration within team; meetings Communication is key Sprints, prototypes, feedback

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Evaluating Web Software Reliability

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

CSI518 – Group 1

Evaluating Web Software Reliability

By Zumrut Akcam, Kim Gero, Allen Chestoski,

Javian Li & Rohan Warkad

Agile Development

  • What is Agile?

    • Focus on business value

    • Collaboration within team; meetings

    • Communication is key

    • Sprints, prototypes, feedback

    • Changing requirements

The Agile Development Process

  • How does Agile work?

  • How did our class use Agile?

    • Three Sprints

    • “Stand up” meetings at beginning of each class

    • Retrospective at the end of each sprint

The Agile Development Process


What is reliability for Web applications?

  • The reliability for Web applications can be defined as the probability of failure-free Web operation completions.[2]

  • Failure is “the event of a system deviating from its specified behavior like obtaining or delivering information”.[1]

Definition of Reliability

  • Failures are caused from the following sources:

    • Host, network or browser failures: computer systems, network or software failures, etc.

    • Source content failures: missing, inaccessible files, JavaScript errors, etc.

    • User errors: improper usage, mistyped URLs.[2]

Failure Sources

  • Attempt to extend previous work on testing the reliability of websites.

  • Gain experience doing a research project

Project Goal

Sprint 1

  • Read relevant research papers

  • Identify factors that may effect reliability analysis

  • Gather status logs

Sprint 1 Goals

  • Web Workload Characteristics

    • Byte Count

    • User Count

    • Session Count

    • Error Count

Factors That May EffectReliability Analysis

  • Reliability analysis via status logs

  • Variety of reliability requirements

  • Commercial and non-commercial

  • We will try to record the technologies the websites employ (Apache, DNN, ISS, PHP, ColdFusion, etc..)

System to Analyze Reliability On

Sprint 2

  • Collect log files for calculation

  • Automate processes to extra data (user,

    session, byte, and error counts)

  • Convert them into excel format

  • Log Parser

Sprint 2 Goals

  • DNN Logs (10 Websites)

  • PHP Logs

Sprint 2 Progress

  • .NET version of Drupal

  • An open source platform for building websites and web applications based on Microsoft .Net technology.

  • Leading open source ASP.NET web content management

  • Has been downloaded over 6 million times

  • ~100 employees

  • 5th Version

  • Founded 2006

What is DotNetNuke (DNN)

  • Logs from 10 Websites

  • Window Server (Same Server)

  • SQL Server 2008

  • ~1000 unique visitors per day

  • Logs contain

    • User count

    • Limited Error count

Our DNN Logs

  • The DNN Logs do not contain

    • Session count

    • Byte count

Major Problem

  • Generate our own DNN logs


Sprint 3

  • Technologies Used

    • Windows XP Professional

    • Microsoft Internet Information Services (IIS)

    • Microsoft SQL Server 2008

    • DotNetNuke (DNN)

Server Side

  • Clients

    • Web-Crawlers

    • DotNetNuke Client API

  • Inducing Errors

  • Logs contain:

    • Client IP’s

    • Byte Counts (Uploaded & Downloaded)

    • Time-Taken

    • Status Code

  • Generating Logs

    Description of Success Status Codes

    Description of Error Codes


    • Server log data consisted of 23 consecutive days of data.

    • Page Not Found (Error 404) is the most common type of error in our logs, with 46% of total recorded errors.

    • Accessing forbidden data (Error 403) follows with 41%.

    • 72 unique IPs, 33 thousand hits total, and each hit associated with about 5kb.

    Workload Measurement Facts

    Status Code-Bytes Graphic













    Error and Success Ratios

    *HTTP Status Codes defined

    By W3C-World Wide Web


    500-Internal Server Error Profile

    Number of Status Codes

    Average Time Taken By Different Status Codes

    • The Nelson Model

      • R: Reliability

      • f : Total number of failures

      • n: Number of workload units

      • r: Failure rate

  • Mean Time Between Failures (MTBF)

    • MTBF = n/f

  • Identify a metric to analyze reliability

    • By Nelson Model, the site software reliability is R = 0.966, or that 96.6% of access to website is successful.

    • This model also shows that MTBF=29.6 hits or the site averages one error for every 29.6 hits.

    • From the number of errors chart, we can see that Server errors are very few among the other errors which shows what the reliability of the DNN server is.


    [1] J.Tian, S.Rudraraju, Z.Li, “Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs”,2004.


    • [1] Jeff Tian, SunitaRudraraju, and Zhao Li. 2004. Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs. IEEE Trans. Softw. Eng. 30, 11 (November 2004), 754-769. DOI=10.1109/TSE.2004.87 http://dx.doi.org/10.1109/TSE.2004.87

    • [2] Toan Huynh and James Miller. 2009. Another viewpoint on "evaluating web software reliability based on workload and failure data extracted from server logs". Empirical Softw. Engg. 14, 4 (August 2009), 371-396. DOI=10.1007/s10664-008-9084-6 http://dx.doi.org/10.1007/s10664-008-9084-6

    • [3] G. Albeanu, A. Averian, I. Duda, “Web Software Reliability Engineering”,2009.


  • Login