“
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

“ MEHARI: A System for Analysing the Use of the Internet Services ” PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on
  • Presentation posted in: General

“ MEHARI: A System for Analysing the Use of the Internet Services ” Presented by: Arturo Azcorra, Josep Solé-Pareta MEHARI Partners: UC3M, UPC and UPM. MEHARI Project Objectives. Traffic Capture Subsystem High Speed AAL5 Reasembly Modular and scalable Low cost

Download Presentation

“ MEHARI: A System for Analysing the Use of the Internet Services ”

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Mehari a system for analysing the use of the internet services

“MEHARI: A System for Analysing the Use of the Internet Services”

Presented by: Arturo Azcorra, Josep Solé-Pareta

MEHARI Partners: UC3M, UPC and UPM


Mehari project objectives

MEHARI Project Objectives

  • Traffic Capture Subsystem

    • High Speed

    • AAL5 Reasembly

    • Modular and scalable

    • Low cost

  • Support for many Traffic Analysis tools:

    • Detailed analyisis (including contents for AUP audits)

    • Identification and agreggation of bidirectional flows

    • Traffic classification by usage

    • Traffic classification by origin / destination

    • Internet header verification

    • ...


Mehari functional architecture

Analysis Subsystem

Capture Subsystem

Analysis Platform(s)

Auto-

regulation

PPS

IP Biflows

+ symptoms

PPS

PreprocessingModule

Capture Platform(s)

ApplicationModules

ATM 1

ATM 1

ATM 0

ATM 1

TrafficSamples

ATM 0

ATM 0

Data base

-patterns

- addresses

- ...

ATM

Cells

Statistics

and Reports

Operator

ATM

Backbone

Capturepoint

MEHARI Functional Architecture

MEHARI System


Capture subsystem

Capture Subsystem

  • Modular and scalable

    • N units over the same or different trunk links

    • Requires high speed connection to the analysis subsystem

  • Senses ALL VPI/VCI in the fiber

    • Captures in promiscuous or filtered mode over VPI/VCI list

  • Capture capacity for each unit

    • Sustained Average of 8 Mbit/s for a 6,000 Euros unit

    • 3,000% better price/performance than commercial protocol analyzers

    • Capture rate controled by analysis rate


Information registered

length (bytes)

timestamp UNIX

(seg.µseg)

Truncated AAL5

info field

VPI/VCI

0:893083746.654070:100/1:1064 :45000428E81B40002F062E36C600B...

1:893083746.654090:100/1:44:4500002C00AC400037069CF5CC4B3C...

2:893083746.654101:100/1:40:45000028455840003606052FCF4F2C1...

3:893083746.654280:103/224:1500:450005DC6C4B4000FD06142640...

4:893083746.654288:103/224:40:45000028240440007B06401E829FD...

5:893083746.654517:103/224:400:45000190B30340001D06B516238A...

......

1668:893083746.813551:100/1:281:4500011976710000FB04BFFCE40...

#

init_time=893083746.652986

final_time=893083746.813582

cap_time=0.160596

frame

seq_num

Files with programmable granularity

Information Registered


Pre processing module

Pre-processing Module

  • Main functions

    • pseudo-packet agreggation to flows

    • pseudo-packet analysis

    • count of symptoms associated to each flow

  • Produces flow list with associated information:

    • flow desc with packet and byte count

    • weighted list of symptoms

  • Highly configurable:

    • symptom definition and inter-relation

    • aggregation period


Classification module

Classification Module

  • Current categories:

    • LEISURE, COMMERCIAL, ACADEMIC, UNKNOWN

  • Current heuristics (human auditing):

    • 1º ‘known’ addresses

      • e.g.: banks (COM), playboy (LEI), sports newspapers (LEI)

    • 2º dominant symptoms

      • e.g.: HTTP=2, PASSWD=3, VISA=1 (COM)

      • e.g.: MAIL=1, CHAT=4, SEX=3 (LEI)

    • 3º non standard ports

      • e.g.: ftp over ports other than 20/21 (UNK)

    • 4º ‘known’ ports

      • e.g.: 6969 (LEI)

Academic by default


Traffic origin destination analysis module

Traffic origin/destination analysis module

Traffic Origin/Destination Analysis Module (TODM)

Official IRR

Data Bases

Processor

NRN BGP

other...

Subnetwork,CIDR, ASs, ...

Databases

Summary

Report Files

Identification

Pre-processing

of AS

Module (TCM)

IP Biflows


Internet headers analysis module

Internet headers analysis module

Internet Header Analysis Module (IHM)

Data base

Summary Report Files

with header patterns

- % Verified traffic

- % Pending traffic

Capture Files

Summary Report Files

Internet Header

Pre-

analysis

-Remote and local servers

processing

(session oriented)

Unknown Traffic

Summary Report Files

Processor

(unknown traffic)


Modularity and scalability of mehari

P 1.2

P 1.1.2

P 1.1

P 1.1.1

P 1.1.3

P 1.3

P 1.3.1

Modularity and Scalability of MEHARI

  • Process tree structure for information flow

  • Interprocess Comunication using shared files

  • May be distributed among several machines using NFS


Some applications of these tools

Some applications of these tools

  • Traffic monitoring

    • Billing and charging models for NRN and Corporate Networks

    • Network configuration

      • Resources dimensioning

      • Placing Proxies, ...

  • Service usage control

    • Control that the services are used responsibly, i. e. auditing the academic networks AUP (Acceptable Use Policy)

    • Security


Conclusions

Conclusions

  • Modular, scalable and extensible architecture

  • Capture systems with excelent price/performance

  • Flow information aggregation with symptoms and bidirectional flow correlation

  • Intermediate data base of patterns and addresses

  • Application modules currently implemented:

    • Classification by usage (AUP)

    • Classification by origin/destination

    • Internet header analysis


Future work

Future work

  • Further improvements in capture capacity

  • Applications to detect security attacks

  • Graphical user interface

  • Automatic reaction to incidents:

    • Alarms (mail, pager, SNMP, ...)

    • Flow blocking or re-routing

    • Flow logging for off-line human analyisis

  • Other type of statistics:

    • Traffic statistics, as those provided by the NetFow

    • Top 100 lists of hosts/servers

    • Main origins/destinations of traffic

    • Most popular sites (webs, ftps, chat servers, ...)


Trial on spanish nrn rediris

Trial on Spanish NRN: RedIris

RedIRIS: the Spanish NRN

Splitters

RedIRIS

RedIRIS

GIGACOM

Core

Regional

Telefónica

ATM

Router

Nodes

Network

ATM Access Switch

100

BaseT

1

STM-1

ATM

Ethernet

Internet

Optical

0

(

RedIris

)

Interfaces

NFS

Remote Access

Analysis PC

Traffic Capture PC

(

LINUX)

(

FreeBSD)


Sample of results traffic classification by usage i

% Bytes (Input traffic)

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

User Groups (17)

Academic

Leisure

Commercial

Unknown

Sample of Results: Traffic classification by usage (I)


Sample of results traffic classification by usage ii

Total Input traffic to RedIRIS (% Bytes)

Total Output traffic to RedIRIS (% Bytes)

Unknown

Commercial

Unknown

Commercial

2%

2%

2%

3%

Leisure

Leisure

12%

17%

Academic

Academic

84%

78%

Sample of Results:Traffic classification by usage (II)


Sample of results main traffic origin destination i

100%

% Bytes (Input traffic)

90%

80%

70%

RedIRIS

60%

TEN-34/155

Ibernet

50%

Rest of Internet (through USA)

40%

30%

20%

10%

0%

User Groups (17)

Sample of Results: Main traffic origin/destination (I)


Sample of results main traffic origin destination ii

Total Output traffic from RedIRIS

Total Input traffic to RedIRIS

26%

27%

RedIRIS

36%

TEN-34/155

41%

Ibernet

Rest of Internet (through USA)

21%

21%

16%

12%

Sample of Results: Main traffic origin/destination (II)


Mehari a system for analysing the use of the internet services

Input traffic

60 %

50 %

40 %

30 %

% of captured traffic

20 %

10 %

0 %

User Groups (17)

Sample of Results: % of academic traffic in the link with USA (according with the IRR description)


Sample of results top 25 most visited commercial sites in one of the user groups

Sample of Results: Top 25 most visited commercial sites in one of the user groups

% Bytes (Input traffic to one of the user groups)

45%

40%

35%

30%

25%

20%

15%

10%

5%

0%

RS

FUT

ABF

OLE

RAN

TSAI

GRN

INFASE

ICTNET

JETNET

SPRITEL

CONEXIS

REDESTB

INTERCOM

CAIXA-RED

IBERNETCOM

ES-TTD-951020

ES-CTV-980527

ES-FCR-950607

IP-MULTIMEDIA

ABCTELEMATIC

SERVICOM2-NETS

SERVICOM1-NETS

DAUCOM2MEG-ES

CANAL-PLUS-SPAIES

Other Sub-Networks: 958


Sample of results january february 99 top 25 most visited ten 155 ass in one of the user groups

Sample of Results (January-February´99): Top 25 most visited TEN-155 ASs in one of the user groups

% Bytes (Input traffic to one of the user groups)

20%

18%

16%

14%

12%

10%

8%

6%

4%

2%

0%

AS1239

AS513 CERN

AS3215 RAIN

Other Ass: 433

AS1103 SURFnet

AS1717 RENATER

AS5556 Telenordia AB

AS5470 AUTH-NET-AS

AS1290 PSINet UK Ltd.

AS3301 TeliaNet Sweden

AS3269 TELECOM ITALIA

AS1853 ACOnet Backbone

AS2529 Demon Internet Ltd

AS786 The JANET IP Service

AS2856 BTnet UK Regional network

AS1741 FUNET autonomous system

AS2852 CESNET z.s.p.o. - TEN34-CZ

AS8761 RETENET Autonomous System

AS8743 HighwayOne Autonomus System

AS6805 mediaWays Autonomous System

AS1653 SUNET Swedish University Network

AS8209 A2000 / Kabeltelevisie Amsterdam bv

Education

AS1835 DENet - Danish Network for Research and

AS1275 DFN-IP service and DFN customer networks

Network

AS224 UNINETT, The Norwegian University & Research

AS559 SWITCH, Swiss Academic and Research Network


Sample of results internet headers verification

1.5 %

13.5 %

0.1 %

Pending

Verified

Unknown

Rejected

84.9 %

Sample of Results : Internet Headers Verification


  • Login