p2p apps ii l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
P2P Apps (II) PowerPoint Presentation
Download Presentation
P2P Apps (II)

Loading in 2 Seconds...

play fullscreen
1 / 66

P2P Apps (II) - PowerPoint PPT Presentation


  • 270 Views
  • Uploaded on

P2P Apps (II). CS 598 IG 23 rd February, 2006 Mehedi Bakht. END-TO-END ARGUMENTS IN SYSTEM DESIGN. J. H. Saltzer, D.P. Reed and D. D. Clark M.I.T Laboratory for Computer Science. Presented by Mehedi Bakht. About the paper. One of the most widely cited/read papers in systems

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'P2P Apps (II)' - daniel_millan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
p2p apps ii

P2P Apps (II)

CS 598 IG

23rd February, 2006

Mehedi Bakht

end to end arguments in system design

END-TO-END ARGUMENTS IN SYSTEM DESIGN

J. H. Saltzer, D.P. Reed and D. D. Clark

M.I.T Laboratory for Computer Science

Presented by Mehedi Bakht

about the paper
About the paper
  • One of the most widely cited/read papers in systems
  • No number, graphcis,etc !
  • The reasoning in this paper is behind many of the design decisions in today’s network
the argument
The Argument
  • Define when it is applicable:
    • “The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the endpoints of the communication system…”
  • Regardless of what happens in the communication systems, correct operation can only be verified by endpoints.
the argument5
The Argument
  • Define when it is applicable:
    • “The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the endpoints of the communication system…”
  • Is it possible to completely ensure integrity due to probabilistic nature of integrity checks?
the argument6
The Argument
  • Define the consequence:
    • “… Therefore, providing that questioned function as a feature of the communication system itself is not possible…”
  • If you can’t do it properly, don’t do it at all.
the argument7
The Argument
  • Define the exception:
    • “… (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement.)”
  • Its not a hard and fast rule, there are special cases where the benefit outweighs the cost.
careful file transfer

2

4

3

1

5

Careful File Transfer
  • Copy/Move file from HD on Computer A to HD on Computer B
careful file transfer9
Careful File Transfer
  • Possible threats to an accurate transfer:
    • Disk error
    • Software error (OS, File transfer program, Network driver)
    • Hardware error
    • Communication system
    • System crash
careful file transfer10
Careful File Transfer
  • Solution 1: Point-to-Point
    • Reinforce each step of process (timeout, retry, etc.)
    • Goal: Reduce probability of each threat to an acceptably small value
    • Could be hard to do, each step must be full-proof
    • Could be inefficient, extra checking
  • Solution 2: End-to-End
    • Store file with a checksum, transfer file, read transferred file back from disk, compute checksum, send checksum to originator to compare the two checksums.
    • If check fails, redo from beginning
careful file transfer11
Careful File Transfer
  • Solution 3: Both
    • Point-to-Point checks in communication system (such as link level, IP, and/or TCP)
    • End-to-End checks must still be performed, since only one of the threats is handled
    • Does not reduce the overall burden to the application, but may reduce the frequency of problems
  • Lesson:
    • Application must supply the guarantee in the end
performance aspects
Performance Aspects
  • Performing the function at the lower level may cost more!
    • Job can’t be done as efficiently since information is more limited
    • May be more efficient if it can be performed with minimal perturbartion
  • Applications that don’t need the function have to pay for it
performance aspects13
Performance Aspects
  • Performing the function at the lower level may cost more!
  • Applications that don’t need the function have to pay for it
  • But what if the total cost shared by all applications(when implemented in comm. System) is less than what incurred by individual applications at application layer?
example delivery guarantees
Example: Delivery Guarantees
  • Acknowledgement of Delivery of a message
  • Knowing that a message got delivered is not important for an application
  • Knowing that the other host acted on the message is important
  • This can only be done by the application on the other host, not the comm system
example secure transmission of data
Example: Secure Transmission of Data
  • Encryption in the network requires trust to manage encryption keys
  • Data is in the clear as it passes into the target node
  • Authenticity still has to be checked by application
  • On the other hand, Network encryption ensures that a node cannot deliberately transmit information that should not be exposed
  • So we need a combination of both?
example duplicate message transmission
Example: Duplicate Message Transmission
  • Some network designs allow(ed) a message to be delivered twice
  • The application might do that, too
  • Thus, suppression must be accomplished by the application anyway
  • Function can be omitted from lower levels
example transaction management
Example: Transaction Management
  • Distributed data storage system
    • Accessing data via identifier, version, type of access: read/write (, value to be written)
  • Network is simplified and does not suppress duplicate messages
    • Identifier plus version suffices to detect duplicate writes
    • Duplicate read only leads to duplicate response
  • Acknowledgement needed is that data is stored safely
    • Can only be provided by the higher levels
  • By eliminating ACKs, the number of messages is halved!
identifying the ends
Identifying the Ends
  • Analysis of application requirements is essential
  • For voice traffic, unusually strong version of the argument applies
    • Bit-perfect communication leads to delays in packet delivery
    • It is better to accept slightly damaged but not delayed packets
  • However, this changes if the conversation is not real-time (voice mail)
    • Accuracy is more important than performance suddenly
  • Carefully identify the end points!
does it still hold
Does it still hold?
  • What about middleboxes?
    • Firewalls
    • NATs
    • Transparent Caches
emerging requirements
Emerging Requirements
  • Operation in untrustworthy world
    • Are the end-points in willing cooperation to achieve goals?
    • attacks on networks, spam, etc
  • More demanding applications
    • Simple service model of internet makes no guarantee
    • Streaming audio and video demand more sophisticated service
emerging requirements21
Emerging Requirements
  • Rise of 3rd Party Involvement
    • Third parties impose themselves between communicating end points
    • ISPs, governments, etc
  • Less sophisticated users
    • End tto end arguments may be source of complexity to the user
    • Software must be installed, configured, upgraded and maintained
preserving peer replicas by rate limited sampled voting

Preserving Peer Replicas By Rate-Limited Sampled Voting

Petros Maniatis

Mema Roussopoulos

TJ Giuli

David S. H. Rosenthal

Mary Baker

Yanto Muliadi

Stanford University

Presented by Mehedi Bakht

slide23

Motivation behind Digital Preservation System

  • Academic publishing is migrating to web
  • Websites are becoming the version of record for many scientific journals.
  • Academic libraries are faced with the urgent problem of creating online collections with the staying power of traditional hardcopy books and journals.
  • Information stored on paper can survive for millennia; information stored digitally today may not be recoverable next week !!
characteristics of digital preservation systems
Characteristics of Digital Preservation Systems
  • Long-term large-scale
  • Lack of central control
  • Unusual Requirements
    • Example : Need to avoid long-term secrets like encryption keys
  • Unusual Oppurtunities
    • Example : Some operations can be made very time-consuming without sacrificing usability
design requirements
Design Requirements
  • Must be very cheap to build and maintain
    • No high-performance hardware (RAID)
  • Need not to operate quickly
    • Should prevent rather than expedite changes
  • Must properly operate for decades without central control
lockss
LOCKSS
  • Lots Of Copies Keep Stuff Safe
  • What is it?
    • software for digital preservation
    • provides librarians with an easy and inexpensive way to collect, store, preserve, and provide access to their own, local copy of authorized content they purchase
how does lockss work
How does LOCKSS work?
  • Have many copies to ensure the long-term survival of the documents
    • Same as for hard copies
  • Use Peer-to-peer opinion polls to guarantee the authenticity of the documents
design principles
Design Principles
  • Cheap storage is unreliable:
    • Write-once media are a least as unreliable as disks
  • No long-term secrets:
    • Too hard to preserve; too hard to recover from leak
  • Use inertia:
    • Prevent change, do not make it too easy
design principles continued
Design Principles (Continued)
  • Avoid third party reputation:
    • Too vulnerable to slander or subversion
  • Intrusion detection is intrinsic:
    • System with bimodal behavior can provide intrinsic intrusion detection
  • Assume a strong adversary:
    • Attackers will be able to use very large numbers of hosts over a long time
the new opinion poll protocol
The New Opinion Poll Protocol
  • Large Archival Units (AUs)
  • Different types of peers
    • Malign peer: one tries to subvert the system
    • Loyal peer: one that follows the LOCKSS protocol at all times
    • Damaged peer: a loyal peer with a damage AU
    • Healthy peer: a loyal peer with the correct AU
  • Goal
    • high probability of healthy peers despite failures and attacks
    • Low probability that a powerful adversary can inflict sufficient damage without detection
the idea of polling
The Idea of Polling
  • The basic idea is to ensure authenticity of AUs by taking advantage of bimodality of the replicas
bimodal system behavior
Bimodal System Behavior
  • For any given article, it is the case that
    • with high probability either most of the replicas are good
    • or with high probability most of the replicas are bad.
    • That is, the probability of half of the replicas being good and half of the replicas being bad is almost zero

Probability

the idea of polling33
The Idea of Polling
  • If the votes overwhelmingly agree, the poll initiator’s copy is ok, do nothing
  • If the caller of the pool receives votes that overwhelmingly disagree
    • Ask for a copy to repair its own
    • Vote again
  • If the result of the poll is neither a landslide win nor a landslide loss (i.e. it violates the bimodal property), then the caller raises an alarm to attract human attention to the situation
voting membership
Voting Membership
  • Inner circle
    • Decide the poll outcome
  • Outer circle
    • Nominated by inner circle
    • May become members of the inner circle in the future
details
Details
  • Each peer maintains two lists
    • Reference list
      • Recently encountered peers
    • Friends list
      • Peers with out-of-band relationship
protocol description bootstrappiing
Protocol Description - Bootstrappiing
  • For each document (or Archival Unit - AU) a library maintains a Friend List.

Friend List

protocol description bootstrappiing37
Protocol Description - Bootstrappiing
  • For each document (or Archival Unit - AU) a library maintains a Friend List.
  • How this friend list is created in the first place by libraries which do not have much collaboration with other libraries?

Friend List

bootstrapping continued
Bootstrapping (Continued)
  • Copy all entries from its current friends list into its reference list
  • Each reference has a random expiration time

Friend

Reference

Copy

poll initiation
Poll Initiation

Poll ?

Poll ?

Poll ?

Reference

Inner

random # of copies

Poll : [Poll ID, Public Key]

  • Choose N random peers from the reference list (inner circle)
poll initiation40
Send encrypted poll messages

Remove peers that cannot answer the challenge-response questions within a specified time frame from the inner circle

If too few inner circle members, invites additional peers from the reference list

Abort when the reference list is exhausted

INITIATOR

VOTER

Poll ?

Poll Initiation
poll solicitation
In response, the peer sends back a PollChallenge message of the form [Poll ID, Diffie-Hellman Public Key, challenge, YES]

YES/NO denotes whether the peer is willing to vote or note

Challenge is a random value used for “effort proof”

INITIATOR

VOTER

Poll ?

Challenge (Yes/No)

Poll Solicitation
poll effort
For every received affirmative message the initiator produces some computational effort that is provable via poll effort proof

The effort and its proof are cryptographically derived from

The poll identifier

Potential voter’s challenge

INITIATOR

VOTER

Poll ?

Challenge (Yes/No)

Proof

Poll Effort
computational effort
Computational Effort
  • Originated from the idea to fight spam
  • Basic idea:
    • the sender must make some sort of computational effort specific to the message or the receiver
  • It should take much less time to verify the effort than to compute it
  • LOCKSS uses memory-bound computations to account for CPU power disparity
poll effort verification nomitaion
A voter verifies the poll effort proof it receives in a PollProof message using the poll identifier and challenge it sent to the initiator

Chooses I other peers at random from its own reference list and nominates them for inclusion into the poll

INITIATOR

VOTER

Poll ?

Challenge (Yes/No)

Proof

Nominate

Poll Effort Verification & Nomitaion
vote construction
Consists of a hash of AU and interleaved with provable computational effort

Vote computation is divided in rounds, each with computational effort and the hashed portion double in size

A subsequent challenge is dependent on the previous challenge

INITIATOR

VOTER

Poll ?

Challenge (Yes/No)

Proof

Nominate

Vote

Vote Construction
vote verification
Vote Verification
  • If the proof of effort is incorrect, the vote is invalid, and the peer if black listed
  • If the proof is correct, and the hash matches, it is valid and agreeing
  • If the proof is correct, and the hash mismatches, it is valid and disagreeing
vote tabulation
Vote Tabulation
  • Agreeing votes are smaller than a threshold (landslide loss), the initiator needs to repair its copy
  • Agreeing votes are greater than a threshold (landslide win), the initiator updates its reference list and schedules the next poll
  • Otherwise, raise an alarm
repair
Repair
  • Need to detect inconsistencies between the voting information and the repaired AU
  • If initiator cannot complete the repair process, raise the corresponding alarm
protocol analysis
Protocol Analysis
  • Need to achieve the following
    • Prevent one from gaining a foothold
    • Make it expensive for the adversary to waste another peer’s resources
    • Make it likely for attacks to be detected
timeliness of effort
Timeliness of Effort
  • Only proofs of recent effort can affect the system
  • Need to expend resources to maintain foothold
rate limiting
Rate Limiting
  • Loyal peers call polls autonomously and infrequently
  • The rate of progress for an attack is limited by victims, not by attackers
reference list update
Reference List Update
  • After a successful poll..
    • Remove all disagreeing peers
    • Remove some randomly chosen agreeing peers
    • Insert outer circle peers whose votes were valid
    • Insert randomly chosen entries from the friends list
    • Remove all peers that have not voted in the last E polls
  • Motivation
    • Reduce dependency on a particular group of peers
obfuscation of protocol state
Obfuscation of Protocol State
  • Encrypt all but the first protocol message exchanged by a poll initiator and each potential voter
  • Make all loyal peers invited into a poll, even those who decline to vote
  • Can’t deduce the number of loyal peers who are involved in deciding the outcome of a poll
alarms
Alarms
  • Raising an alarm is expensive
    • Involve human examinations
  • Three kinds of alarms proposed
  • Use human intervention to detect and fix the problem
alarms55
Alarms
  • Raising an alarm is expensive
    • Involve human examinations
  • Three kinds of alarms proposed
  • Use human intervention to detect and fix the problem
  • In the paper, it seems that alarms somehow manage to make everything right again, but does not specify how!
adversary analysis
Adversary Analysis
  • Complete parameter knowledge
  • Exploitation of common peer vulnerability
    • Take over a fraction of populations running the same implementation
  • Unconstrained identities
    • Infinite IP addresses
  • Stealth
    • One cannot discern loyal peers from compromised ones
adversary analysis57
Adversary Analysis
  • Total information awareness
    • Identities of all malign peers
  • Perfect work balancing
  • Perfect digital preservation
    • Incorruptible copies of good and bad Aus
  • Local eavesdropping
  • Local spoofing
    • One end of the communication needs to be in the local network
adversary attacks
Adversary Attacks
  • Platform attacks
    • Can take over a fraction of peers instantaneously
  • Protocol attacks
    • Play against the LOCKSS protocol
protocol attacks
Protocol Attacks
  • Stealth modification
    • Replace good AUs with bad ones
  • Nuisance
    • Raise many alarms
  • Attrition
    • Prevent loyal peers from repairs
  • Theft
    • Obtain published content without paying
protocol attacks60
Protocol Attacks
  • Free-loading
    • Obtain services without supplying services in return
counter attack techniques
Counter-Attack Techniques
  • Adversary foothold in a reference list
    • Need to wait for invitation to vote
    • Need to behave well for a long time before the attack (without raising alarms)
  • Vote base on good AU, supply the bad AU for repair
    • Ask random sample bits (verified) before each poll
    • The repair AU must match the initial bits
stealth modification attack strategy
Stealth Modification Attack Strategy
  • Two phases
    • Lurk to build a foothold in loyal peers’ reference lists
    • Attack
  • Need to have the majority of votes
  • Need to have loyal peers < the alarm threshold
simulation
Simulation
  • Running LOCKSS for 30 years
  • 1000 peers
    • Clusters of 30 peers
    • 29 peers in the initial friends list
      • 80% from the local cluster
  • 20 years of lurking
  • 10 years of attacking
results
Results
  • Low rates of false alarms in the absence of attacks
  • Can sustain up to 1/3 of the peers subverted (with 10% churn)
conclusion
Conclusion
  • LOCKSS can produce a peer-to-peer system with remarkable ability to resist attacks over decades.
conclusion66
Conclusion
  • LOCKSS can produce a peer-to-peer system with remarkable ability to resist attacks over decades.
  • Application of the concept of probabilistic security
  • Can we think of any other case where the the basic ideas of LOCKSS can be applied?