slide1
Download
Skip this Video
Download Presentation
Automated SIP Interoperability Testing

Loading in 2 Seconds...

play fullscreen
1 / 24

Automated SIP Interoperability Testing - PowerPoint PPT Presentation


  • 153 Views
  • Uploaded on

Automated SIP Interoperability Testing. Archana Rao and Henning Schulzrinne Department of Computer Science Columbia University http://www.cs.columbia.edu/IRT/voip-testbed/. SIP 2009 (Paris, January 2009). Overview. The problem of SIP interoperability Far from perfect

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Automated SIP Interoperability Testing' - gyala


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1
Automated SIP Interoperability Testing

Archana Rao and Henning Schulzrinne

Department of Computer Science

Columbia University

http://www.cs.columbia.edu/IRT/voip-testbed/

SIP 2009 (Paris, January 2009)

overview
Overview
  • The problem of SIP interoperability
    • Far from perfect
    • Damages “brand”, harms customers, encourages single-vendor deployments
  • Testbed Overview
    • Architecture
    • Components
  • Experiments
    • Interoperability study
    • End-client characteristics
  • Summary
interoperability
Interoperability

Is it a real problem ?

Hi Henning,You may remember me from IETFs past -- I haven't attended any in some time because I couldn't find any really interesting projects. I'm finallygetting into SIP. I've got Speakeasy VoIP service, two sipphone accounts, a Cisco 7960 and a copy of x-ten on my Mac. And I still can't make it work. Voice flows in one direction only. I'm not even behind a NAT or firewall -- both machines have global addresses, with no port translations or firewalls.

I've been working with Internet protocols for over 20 years. I've implemented and contributed to them. And if *I* can't figure out how to make this stuff work, how is the average grandmother expected to do so?

SIP is unbelievably complex, with extraordinarily confusing terms. There must be half a dozen different "names" -- Display Name, User Name, Authorization User Name, etc -- and a dozen "proxies". Even the word "domain" is overloaded a half dozen different ways. This is ridiculous!Sorry. I just had to get this off my chest. Regards, Phil Karn.

Excerpts from an email posted on IETF RAI mailing list –

Reference: http://www1.ietf.org/mail-archive/web/rai/current/msg00082.html

interoperability issues
Interoperability issues

Why do we have interoperability problems?

  • SIP is complex
    • More than 150 RFCs and 500 active Internet Drafts
    • Efforts like – “draft-ieft-sip-hitchhikers-guide” help to a certain extent, but not sufficient
  • SIP has no proposed architecture/profile
    • SIP usage in variety of environments and domains make it impossible
    • Standardization from IETF is different from that of ISO/ITU
  • SIP is continually evolving
    • Non trivial for implementers to follow the life-cycle from IDs to RFCs
  • Walled gardens, non-standard environments
    • Vendors need to make products that work in their customer’s environment
  • Poor implementations, intentional non-interoperability etc.
improving interoperability
Improving interoperability

Current efforts to combat SIP interoperability issues

  • SIP Interoperability (SIPit) events
    • Week long gatherings of SIP implementers to test interoperability
    • Coordinated by SIP Forum
  • SIP Forum
    • SIP Connect 1.0 (and 1.1)  outsourced enterprise VoIP
  • IETF Basic Level of Interoperability for SIP Services (BLISS)
    • Focus on resolving interpretability issues in SIP features
    • Line sharing, call parking, automated call handling and call queuing
  • University of New Hampshire – InterOperability Laboratory
    • Vendor-neutral lab dedicated to testing data networking technologies
    • 20 different testing programs, each costing about $20,000 per year
interoperability1
Interoperability

So, isn’t the problem solved?

  • These forums focus on details of advanced SIP features
  • Basic-level of interoperability between innumerable SIP devices in the market
    • Can I take any SIP phone and make a VoIP call, through any SIP service provider?

How do we go about it?

1. Identify the basic real-world scenarios for SIP registration and session establishment

2. Use our VoIP testbed to configure a variety of SIP infrastructure

3. Study the behavior of SIP end-clients and servers for interoperability

studying interoperability
Studying interoperability
  • Define a minimum set of call flows constituting basic interoperability
    • RFC 3665, RFC 3666, RFC 4317
  • Use a VoIP testbed to realize a variety of real-world SIP infrastructure setups
    • Columbia VoIP testbed
  • Study the behavior of SIP end-clients and servers in each of these scenarios
    • Interoperability matrices
    • Categorization of common issues 
      • Lack of clarity in the specification
      • Implementation of an older specification
      • Incomplete implementation of the specification
      • Incorrect implementation of the specification
      • Failure against robustness tests
columbia voip testbed
Columbia VoIP testbed

What?

  • VoIP infrastructure for experimentation, analysis, testing, prototyping and deployment of SIP/VoIP components in a variety of environments.
  • Part of a multi-university research project supported by NSF
    • University of North Texas, Purdue University and University of California, Davis

Why?

  • There’re no telecom testbeds available for research & experimentation
  • VoIP implementations & experiments have lacked scientific rigor
    • Continuously emerging standards
    • Need to develop repeatable, accurate test frameworks
  • Realistic VoIP experiments require a distributed testbed
slide9
http://www.cs.columbia.edu/IRT/voip-testbed/

http://www.cs.columbia.edu/IRT/voip-testbed/

your equipment here :-)

columbia voip testbed servers
Columbia VoIP testbed: servers
  • 5 SIP servers
    • Asterisk, Pbxnsip, Sipd, SER, OpenSER
  • 3 Platforms
    • Microsoft Windows XP
    • Linux Fedora Core 6
    • Sun Solaris X
  • 3-way Connectivity
    • the Internet
    • VPN
    • PSTN
columbia voip testbed uas
Columbia VoIP testbed: UAs
  • 20+ SIP hard-phones, soft-phones, WI-FI phones with different capabilities
  • More details – www.columbia.edu/~ahr2114/VoIPtestbed/end-clients
nsf voip testbed
PSTN

Network

UNIVERSITY-A

UNIVERSITY-B

UNIVERSITY-D

The

Internet

UNIVERSITY-C

NSF VoIP testbed

Image courtesy: http://secnet.csci.unt.edu/7.15.funding_application.ppt

columbia voip testbed experiments
Columbia VoIP testbed experiments

Planned or ongoing

  • Interoperability study
    • Can any two SIP devices talk to each other?
  • SIP end-client characteristics
    • Signaling and codec features in SIP devices
    • Robustness
    • VoIP with/despite NAT
  • Performance issues
    • Signaling delay, voice mouth-ear delay, packet loss resilience
    • Servers – Scalability, maximum capacity
  • Security
    • Separate testbed for DOS and other attacks

work in progress

interoperability study call flows
Interoperability study: call flows

Registration (RFC 3665)

  • Successful new registration
  • Unsuccessful registration
  • Cancellation of registration
  • Update of contact list

Session Establishment (RFC 3665)

  • Successful session establishment
  • Session establishment through two proxies
  • Session establishment with multiple proxy authentication
  • Successful session with proxy failure
  • Unsuccessful no answer
  • Unsuccessful busy
  • Unsuccessful no response from user agent
  • Unsuccessful temporarily unavailable

Codec Negotiation (RFC 4317)

  • Audio and DTMF
  • Audio, video and DTMF
  • Audio and video codec reordering
interoperability study matrix
Interoperability study: matrix

Interoperability Matrix

Test case #1 - Successful new registration

interoperability study auth name
Interoperability study: auth name

Issue 1 – Use of different formats for authentication name

  • Authorization: Digest username=“user1”,realm=“proxy01.sipphone.com”,nonce=“xxx”, uri=“sip:proxy01.sipphone.com”,response=“yyy”, algorithm=MD5

Authorization: Digest username=“sip:[email protected]”, realm=“proxy01.sipphone. com”,nonce=“xxx”,uri=“sip:proxy01.sipphone.com”,response=“yyy”, algorithm=MD5

  • Implication
    • Registration and call setup failure, if both formats are not supported by the UAs
  • Instances
    • Polycom PVX/VSX, Wengophone don’t register with 3Com / sipd proxies
  • Workaround
    • Use Asterisk between Polycom PVX and 3Com proxy
interoperability study rport
SIP server

UAC

INVITE (Dest port:5060)

180 Ringing (Src port:12345)

200 OK (Src port:12345)

ACK (Dest port:12345)

200 OK (Src port:12345)

ACK (Dest port:12345)

Interoperability study: rport

Issue 2 – Behavior in the absence of rport parameter

  • Implication
    • Incorrect/incomplete signaling resulting in multiple retransmissions and failed transaction
  • Instances
    • Calls to Cisco 7940 via Sipphone proxy are always unsuccessful
    • Calls from Polycom VSX via 3com or sipd proxies are always unsuccessful
  • Workaround
    • Nothing
interoperability study codecs
Interoperability Study: codecs

Issue 3 – Codec Negotiation

  • Implication – Audio/Video failure (mostly unidirectional, but bidirectional sometimes)
  • Problem Instances
    • Audio failure when Cisco calls voicemail service of sipphone provider
    • Video failure when Polycom PVX calls Xlite via Asterisk
  • [Offer]

v=0

o=Cisco-SIPUA 28422 0 IN IP4 128.59.17.67

s= SIP Call

t=0 0

m=audio 31030 RTP/AVP 0 8 18 101

c=IN IP4 128.59.17.67

a=rtpmap:0 PCMU/8000

a=rtpmap:8 PCMA/8000

a=rtpmap:18 G729/0

a=fmtp:18 annexb=no

a=rtpmap:101 telephone-event/8000

a=fmtp:101 0-15

a=sendrecv

  • [Answer]

v=0

o=Cisco-SIPUA 28422 0 IN IP4 128.59.17.67

s= SIP Call

t=0 0

m=audio 31030 RTP/AVP 18 0 8 101

c=IN IP4 128.59.17.67

a=rtpmap:18 G729/0

a=fmtp:18 annexb=no

a=rtpmap:0 PCMU/8000

a=rtpmap:8 PCMA/8000

a=rtpmap:101 telephone-event/8000

a=fmtp:101 0-15

a=sendrecv

ua characteristics features
UA characteristics: features
  • Basic
    • Configuration, number of lines, call hold, call forwarding, DND, NAT support
  • Advanced
    • Encryption, symmetric RTP, conferencing, audio/video codecs, PoE
    • Audio quality – silence suppression, echo cancellation, comfort noise
    • Transport, signaling and support protocols – TCP, TLS, HTTP, DHCP, DNS, TFTP, NTP
  • More details at – http://www.columbia.edu/~ahr2114/VoIPTestbed/EndClients.htm
ua characteristics robustness
UA characteristics: robustness

Robustness

  • SIP Torture Tests (RFC4475)
    • A set of SIP messages aimed at stressing the SIP parser, based on experiences at SIPit.
  • PROTOS
    • Test suite with 4500+ INVITE requests, developed by University of Oulu

Example Results

ua characteristics nats
UA characteristics: NATs

SIP and NAT

  • Problematic for both signaling (SIP) and media transfer (RTP)
  • Circumventing NAT effect – solutions exist for both end-clients and proxies
  • Quest to overcome this, could lead to techniques uglier than NAT itself!

NAT test setup

  • NAT realization - Linux iptables, CLICK router
  • Studying the behavior in different scenarios
    • NAT support enabled in both the proxy server and end-client
    • NAT support enabled in only one of them etc
  • Aiming to analyze the existing practices – the good, the bad and the ugly!
ua with nats
UA with NATs

SIP devices in NAT environment

what now
What now?

Summary

  • Tests indicate that basic-level interoperability is far from perfect
  • Instances of interoperability failures illustrated in our IETF workshop paper
  • Achieving Interoperability shouldn’t be left just to vendors
    • non-interoperability damages SIP brand, causes grief for third parties and harms customers

Going ahead – some ideas

  • Establish designated interop liaison for each vendor
    • calling customer support unlikely to be useful
  • Encourage vendors to publish interoperability reports
    • in standard format
  • Provide self-certification tests, supported by a remotely accessible test rig
summary
Summary

Current status

  • The VoIP testbed & VPN connectivity is operational
  • Systematic study of real-world interoperability issues
    • Basic-level interoperability is nowhere near 100%
    • IETF 70 – The 1st SIP Forum Workshop on SIP Interoperability
  • Evaluating SIP end client characteristics

Testbed plans

  • Experiments on NAT, performance, security
  • Make the testbed accessible and useful for engineers & deployments
  • Create a knowledge repository about SIP devices & tools for interoperability testing
ad