1 / 22

Improving Metrics for Simplicity in Networked Systems Design

This paper explores the challenge of capturing and quantifying simplicity in designing networked systems. While the community values "simple" and "clean" designs, existing metrics often fail to align with this aesthetic. We propose a new approach to measure design simplicity by analyzing the dependencies inherent in system state. By rigorously comparing different design options through the lens of these dependencies, we aim to better align algorithmic and system design goals. Our findings suggest that understanding and measuring complexity can enhance system design practices in the networking domain.

ayasha
Download Presentation

Improving Metrics for Simplicity in Networked Systems Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Capturing Complexity in Networked Systems Design: The Case for Improved Metrics Sylvia Ratnasamy Intel Research

  2. Motivation Our community values “simple”, “clean” system designs “The advantage of Chord is that it is less complicated…” – Chord, Sigcomm’01 “The complexity of the architectural design…” – Sprint, IEEE Network 2000 “A design for multicast that is simple…” – Perlman. Simple Multicast, IETF Draft. “Complex routing algorithms may have difficulty scaling… simple floods are sufficient” -- TinyOS, NSDI’04 “This paper proposed a simple and effective approach …” – SOSR, OSDI’04 “Thus, simplicity is our primary design requirement …” – BVR, NSDI’05

  3. Motivation Our community values “simple”, “clean” system designs • But lacks metrics that rigorously capture this aesthetic • Best practice: metrics borrowed from the theory community • e.g., total_messages, total_state • Problem: at times incongruent with our notion of “simplicity” • e.g., flooding: inefficient but simple • e.g., dijkstra’s: O(n) state, but simple • e.g., leaderID vs. neighborID

  4. This Paper • Can we quantify design “simplicity”? • more rigorously compare-and-contrast design options • align design goals of algorithms, systems communities • First stab: complexity metrics for the algorithmic component of a networked system • Note • just one aspect to system complexity • intent: complement, not replace, existing metrics

  5. Outline • Strawman • Simple evaluation scenarios • Limitations and Future Directions

  6. Strawman Two observations: • much of system design centers around state • what state is required? • how is it constructed/maintained? • how is it used? • what distinguishes wide-area state: • derived from remote nodes  dependencies are distributed • relayed via intermediate nodes  more dependencies Current metrics mostly treat all state as equal

  7. Proposal For a given piece of state s, measure the ensemble of distributed dependencies that yield s • First cut: measure  counting • akin to existing metrics: #msgs, #state • amenable to evaluation by simple examination, simulation

  8. Value vs. Transport Dependencies s is value dependent on x, s is transport dependent on next(B) state at A, R1, R2, R3. vs= #pieces of state on which s is value dependent tsx= #pieces of state relied on to transport x to s next(B) next(B) next(B) next(B) A R1 R2 R3 B x=30° s=x

  9. Counting Dependencies next(B) next(B) next(B) next(B) A R1 R2 R3 B x=30° s=x vx = txx = 0 vs = 1 ; tsx = 4

  10. Counting Dependencies Note: value dependencies accumulate A R1 R2 R3 B C R4 s=x y=20° x=30+y° vx = 1 txy = 2 vy = tyy = 0 vs = 2 tsx = 4

  11. Dependencies  Complexity single input: state s, derived from x • complexity of s, cs = (vx + 1)tsx + cx A R1 R2 R3 B C R4 s=x y=20° x=y+30° vs = 2 , tsx = 4 vy = tyy = 0 vx = 1 txy = 2 cx = 1.2+0 = 2 cs = 2.4+2 = 10 cy = 0

  12. Dependencies  Complexity single input: state s, derived from x • complexity of s, cs = (vx + 1)tsx + cx … B A s=2 s=i s=d x=0 s=1 vs = d , tss-1 = 1 vx = i tii-1 = 1 cs = O(d2) c= 0 cs = 1 cs = 3

  13. Dependencies  Complexity multiple inputs: state s, derived from x1, x2, …, xm two basic variants: • s = ALL (xi) cs = Σ ((vxi + 1)tsxi + cxi) • if vxi = cxi = 0, tsxi = 1, then cs = m • s = ANY (xi) cs = 1/m Σ ((vxi + 1/m)tsxi + cxi) • if vxi = cxi = 0, tsxi = 1, then cs = 1/m

  14. Dependencies  Complexity multiple paths: state s derived from x, multiple paths from x to s simple case: m disjoint paths, each incurring d transport dependencies tsx = d/m, cs = vs tsx + cx • if vx = cx = 0, then cs = d/m

  15. Complexity: toy scenarios

  16. Preliminary Evaluation (summary)

  17. Limitations, Future Directions • scope: no validation of assumptions, correctness, quality, performance, etc. • correlated inputs • discriminating transport dependencies • capturing absent dependencies • capturing robustness • count “reverse” dependencies? • role of time • evaluating the metric: future studies

  18. Summary • design simplicity: valued, but poorly measured • question: is design complexity measurable? • conjecture: capturing dependencies in state is key (w.r.t. algorithmic component of a system)

  19. Thanks! Questions?

  20. … B A s=2 s=i s=d x=0 s=1 complexity of operations • an operation acts on different pieces of state • each piece of state has a complexity • compute complexity of an operation by (appropriately) combining complexity of state it acts on cs = 0 1 ... O((d-1)2) O(d2) complexity of forwarding operation: cfwd = O(d3)

  21. complexity of operations • an operation acts on different pieces of state • each piece of state has a complexity • compute complexity of an operation by (appropriately) combining complexity of state it acts on cs = k 1 cs = k 2 cs = k 3 fetch(obj) … cs = k m complexity of fetch: cfetch = O(k/m)

  22. Future Studies • centralized solutions simpler? • e.g., RCP vs. DV • designing for low dependency • e.g., soft state-based approaches • layered vs. customized • e.g., PHTs [sigcomm’05] vs. Mercury [sigcomm’04] • different routing options • mesh (RON, ESM), DHT-inspired (VRR/ROFL), centralized (RCP), compact routing-based, landmark routing, coordinate-based, …

More Related