Internet2 & perfSONAR-PS

August 23rd 2012, OSG & perfSONAR-PS Meeting Jason Zurawski – Senior Research Engineer Internet2 & perfSONAR-PS

Agenda • Definitions • Historical Overview • Present Day Activities • “Wish List” • Future Directions

What is perfSONAR? • Measurement Middleware • Enables the sharing and management of data collected from external sources (e.g. SNMP, Iperf) • Web Services based front ends – simple requests dump back XML/JSON formatted raw measurement data • Location/Discovery services (find what you need) • (Our attempt at) GUIs • Deployed community infrastructure, available on 100s of networks/domains • Independent software implementations • Protocol to handle communication between components

What isn’t perfSONAR? • A silver bullet • Performance debugging is still a dark art. • pS tools can assist in the process of identifying a problem, but it is by no means fully automated • A complete product • Active development effort on protocol and software • An effort to redefine measurement tools • Incorporate existing tools (e.g. SNMP monitoring, Ping, Bandwidth etc.) when available. Middleware is there to share (and encourage the sharing) of data • Ex: Think of a ‘protected’ MRTG web page. Only useful to those with a password. perfSONAR allows sharing of the raw data to be used in other systems

Historical Overview • Internet2 End-2-End Performance Initiative (E2EPI) • Fall of 2000 • Founding premise for Internet2 is that QoS will solve reliability problems experienced through the commercial internet. It is presumed that, until QoS is going, bandwidth overload will solve problems. • Realization that bandwidth alone doesn't solve all problems. M. Mathis of PSC (now Google) presents the Wizard Gap; those with technical know-how can always use a system to full potential while the average user is experiencing end-to-end (E2E) problems or an insignificant improvement • Performance Initiatives Performance Environment System (PIPES) • Regular testing on paths of interest • Dedicated machines to do the task • Discontinued as of 2005 in favor of perfSONAR

Historical Overview • Global^H^H^H^H^H^H Open Grid Form • ~2003 • Network Measurement working group – proposed a ontology of common network measurements. Common representations and categories • Created a format to enable programmatic sharing of this data. • Reference implementation needed to strengthen the standard … • perfSONAR • ~2004 • Joint effort between Internet2 E2EPI, Members of GGF-NMWG, GEANT2 JRA-1 Activity (Europe) • Focus on developing interoperable software for measurement exchange (using well defined protocols) • Coordinate on international testing procedures

Historical Overview • perfSONAR (cont) • Included ESnet and RNP (Brazilian NREN) in effort • First software – 2006 (Java based wrapper around RRD files; enabled web service sharing of collected data). • perfSONAR-PS • ~2007 • Forked effort by U.S. Partners to produce interoperable software based on sounder operational footing (e.g. emphasis on integration with existing Linux based measurement systems) • ESnet, Internet2, University of Delaware, Georgia Tech, Indiana University, Fermilab, SLAC • Producing software updates to current time

Historical Overview • pSPerformance Toolkit • ~2009 • Observation that the largest barrier to adoption of an end to end monitoring system is deployment • End to end monitoring needs observation points along an entire path • Package the tools into a nearly 0 configuration package: • Initially Knoppix, then CentOS based LiveCD/Netinstall Linux distribution • Point and click to set up regular monitoring • Integration with outside alarming tools (NAGIOS/ZENOS)

Historical Overview - Conclusion • 11+ Years of experience for some participants • Goal is simplify the identification and solution of end to end performance problems • Software deployment bar has been lowered to improve adoption – increased adoption facilitates faster solution space • Related Efforts: • LHCOPN/LHCONE/USATLAS/USCMS Throughput Group • GEANT eduPERT • Fasterdata.es.net • Internet2 Research Support • 25% of support requests (32 since April 2012) are related to E2E performance

Present Day Activities • Software Development • Internet2 Core team = 3 developers, with fractional FTE allotments • Core products • perfSONAR-PS Software - TL1/SNMP Status monitoring, SNMP Measurement Archive, perfSONAR-BUOY Regular Testing system, Topology Service, Lookup Service • ESnet supports other current services • pS Performance Toolkit (Linux Distro) – Support of the system guts and efforts to package software into RPMs • Joint effort with Esnet

Present Day Activities • Software Support • “Community” Mailing lists. We encourage everyone to be self supporting due to the man-power allotments • Research Support via the CTO’s office (e.g. more personalized assistance in enabling a full deployment at a campus or VO) • Soft Money – comes and goes • 2 DOE projects (OSC, GATech) • 1 GENI project (IU) • Internet2 pursuing NSF money – no ETA

Present Day Activities • Network Observatory • Distributed set of test instances at 9 internet2 PoPs • 2 x Throughput (bwctl.$CITY.net.internet2.edu) • 1 x Latency (owamp.$CITY.net.internet2.edu) • 1 x User tester (ndt.$CITY.net.internet2.edu) • Used to collect baseline measurements of Internet2 network • Release network data to researchers • Serves as a testing point for network debugging (e.g. isolate the problem/divide and conquer).

Present Day Activities • Related (Internal) Efforts • OWAMP • Stable @ v 3.3 • Additional releases not expected beyond bugfixes • BWCTL • Stable @ 1.3 • Additional releases are expected to support Iperf v3, no ETA • NDT • Stable, but still changing (MLab tie in). • Currently at 3.6.5. • Lots of questions on the future of this (see Web10G in next section)

Present Day Activities • Related (External) Efforts • IPERF • Recent release of version 3 (complete re-write) • ESnet working to integrate into BWCTL • pS effort still using older version 2 of IPERF • Web10G • New effort to redefine a TCP “MIB” for the Linux kernel • Follow on to Web100 – a necessary component for the NPAD/NDT tools • pS Performance Toolkit uses Web100 kernels. Expects to migrate to Web10G when: • Packages are available for CentOS Linux • A user tool (Iperf, NPAD, etc.) can consume the data

“Wish List” • Not Ranked, not Comprehensive. Based on recent discussions and open issues • “Mesh” Configuration (e.g. management of multiple nodes without having to configure each) • Framework Stabilization • Lookup/Topology Infrastructure is flakey. Re-thinking information management and sharing • Data retrieval is slow and limited by XML technology – considering REST API to encourage faster development of clients/GUIs

“Wish List” • Cont. • More Open “Open Source”. Recent interest from other communities about contributing (Medical, as well as OSG). How can we make that easier? • Additional tools – pursuing soft funding from NSF/DOE/NIH to alter NDT/create new Web10G tool, or get funding to complete aforementioned items • OS Upgrades – Support 32/64 Bit versions of CentOS 6 (we keep things simple, no real effort to support Debian based distros, or others). • GUIS – although we want to get out of this game … Encourage others do do the GUI work, tell us what is needed to do it right.

Future Directions • Internet2 does not want to see pS fade away, but … • Development does cost $ and resources (that we are short on) • pS does not generate revenue • Community has a sense of ownership when it comes to wanting free software, but not when it comes to supporting/advancing it. Change in perception needed • Largest users are VOs, and many requests are not generalizable beyond the VO. VO resources (e.g. OSG) could be exercised on certain tasks

Future Directions • Expected outcomes: • Increased community involvement in the direction and action of development • Possible introduction of services on top of measurement infrastructure (e.g. ‘RHEL model’, perhaps we offer a ‘Net+’ service out of measurement for institutions that would rather have a managed solution) – proceeds would advance the open source development • Stepping away from GUIs and more into hardening the infrastructure (e.g. GUI development belongs with the users that need it most, not network engineers that can’t please everyone)

Questions? • ?

Internet2 & perfSONAR-PS August 23rd 2012, OSG & perfSONAR-PS Meeting Jason Zurawski – Senior Research Engineer For more information, visit http://psps.perfsonar.net

Internet2 & perfSONAR-PS