Yogi part 3
This presentation is the property of its rightful owner.
Sponsored Links
1 / 58

Yogi – Part 3 PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

Yogi – Part 3. Engineering the tool and parallelization. Windows Device Drivers. do { //get the write lock KeAcquireSpinLock (& devExt -> writeListLock ); nPacketsOld = nPackets ; request = devExt -> WriteListHeadVa ; if(request && request->status){

Download Presentation

Yogi – Part 3

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Yogi part 3

Yogi – Part 3

Engineering the tool and parallelization


Windows device drivers

Windows Device Drivers

do {

//get the write lock

KeAcquireSpinLock(&devExt->writeListLock);

nPacketsOld = nPackets;

request = devExt->WriteListHeadVa;

if(request && request->status){

devExt->WriteListHeadVa = request->Next;

KeReleaseSpinLock(&devExt->writeListLock);

irp = request->irp;

if(request->status > 0){

irp->IoStatus.Status = STATUS_SUCCESS;

irp->IoStatus.Information = request->Status;

}

else{

irp->IoStatus.Status = STATUS_UNSUCCESSFUL;

irp->IoStatus.Information = request->Status;

}

SmartDevFreeBlock(request);

IoCompleteRequest(irp, IO_NO_INCREMENT);

nPackets++;

}

} while (nPackets != nPacketsOld);

KeReleaseSpinLock(&devExt->writeListLock);

Unlocked

U

L

Locked

U

L

Error


Yogi part 3

Read for

understanding

Drive testing

tools

API Usage Rules

(SLIC)

New API rules

Defects

Software Model

Checking

100% path

coverage

Rules

Static Driver Verifier

Development

Testing

Source Code


Yogi part 3

Read for

understanding

Drive testing

tools

API Usage Rules

(SLIC)

New API rules

Defects

Software Model

Checking

100% path

coverage

Rules

Static Driver Verifier

Development

Testing

Source Code


Architecture of yogi

Architecture of Yogi

C Program

slamcl

Slam.li database

SLIC property

slicc

Initial functionsummaries

Instrumented

Slam.li database

Alias and mod-ref information

li2yir

polymorphic

region graphs

Yogi IR (yir)

YSim

YAbsMan

Proof that program satisfies property

Test case that exposes a bug

Z3 theorem prover


Implementation

Implementation

  • ~6K lines of Ocaml

  • Z3theorem prover

  • Integrated with SDVfor Windows

  • Engineering effort: 3 person years

  • F# version available with SDVRP

    • http://research.microsoft.com/yogi


Optimizations

Optimizations

  • Share our experiences in making Yogi robust, scalable and industrial strength

  • Several of the implemented optimizations are folklore

    • Very difficult to design tools that are bug free evaluating optimizations is hard!

    • Our empirical evaluation gives tool builders information about what gains can be realistically expected from optimizations

    • Details in “An empirical study of optimizations in Yogi, ICSE ’10“

  • Vanilla implementation of algorithms:

    • (flpydisk, CancelSpinLock) took 2 hours

  • Algorithms + engineering + optimizations:

    • (flpydisk, CancelSpinLock) took less than 1 second!


O ptimizations

Optimizations

  • Initial abstraction from property predicates

  • Relevance heuristics for predicate abstraction

    • Suitable predicates (SP)

    • Control dependence predicates (CD)

  • Summaries for procedures

  • Thresholds for tests


Evaluation setup

Evaluation setup

  • Benchmarks:

    • 30 WDM drivers and 83 properties (2490 runs)

    • Anecdotal belief: most bugs in the tools are usually caught with this test suite

  • Presentation methodology:

    • Group optimizations logically such that related optimizations are in the same group

    • Total time taken, total number of defects found for every possible choice of enabling/disabling each optimization in the group


Initial abstraction

Initial abstraction

state {

enum {Locked = 0, Unlocked = 1}

state = Unlocked;

}

KeAcquireCancelSpinlock.Entry

{

if (state != Locked) {

state = Locked;

}

else

abort;

}

KeReleaseCancelSpinlock.Entry

{

if (state == Locked) {

state = Unlocked;

}

else

abort;

}

0

0

0

1

1

1


Empirical results

Empirical results

16%


Relevance heuristics sp

Relevance heuristics (SP)

Irrelevant?

  • Avoid irrelevant conjuncts

A

B

C

D

A

B

C

C

D


Relevance heuristics cd

Relevance heuristics (CD)

  • Abstract assume statements that are not potentially relevant by skip statements

  • If Yogi proves that the program satisfies property, we are done.

  • Otherwise, validate the error trace and refine the abstraction by putting back assume statements, if the error trace is spurious


Example sp heuristic

Example: SP heuristic

int x;

void foo() {

bool protect = true;

if (x > 0)

protect = false;

if (protect)

KeAcquireCancelSpinLock();

for (i = 0; i < 1000; i++) {

a[i] = readByte(i);

}

if (protect)

KeReleaseCancelSpinLock();

}

A

B

C

D

A

B

C

C

D


Example sp heuristic1

Example: SP heuristic

int x;

void foo() {

bool protect = true;

if (x > 0)

protect = false;

if (protect)

KeAcquireCancelSpinLock();

for (i = 0; i < 1000; i++) {

a[i] = readByte(i);

}

if (protect)

KeReleaseCancelSpinLock();

}

A

B

C

D

A

B

C

C

D


Example cd heuristic

Example: CD heuristic

int x;

void foo() {

bool protect = true;

if (x > 0)

protect = false;

if (protect)

KeAcquireCancelSpinLock();

for (i = 0; i < 1000; i++) {

a[i] = readByte(i);

}

if (protect)

KeReleaseCancelSpinLock();

}


Empirical results1

Empirical results

10%


Empirical results2

Empirical results

16%


Empirical results3

Empirical results

25%


Interprocedural analysis

Interprocedural analysis

  • Yogi performs a compositional analysis

    • : Is it possible to execute starting from state and reach state ?

  • Global modification analysis

  • Compositional May-Must analysis (POPL 2010)


Example

Example

A

B

C

D

A

B

C

C

foo(…)

D


Empirical results4

Empirical results

32%


Empirical results5

Empirical results

28%


Empirical results6

Empirical results

42%


Testing

Testing

  • Yogi relies on tests for “cheap” reachability

  • Long tests

    • avoiding several potential reachability queries

    • results in too many states and thus memory consumption

  • Test thresholds: time vs. space tradeoff


Empirical evaluation

Empirical evaluation


Modeling the environment

Modeling the environment

if (DestinationString)

{

DestinationString->Buffer = SourceString;

// DestinationString->Length should be set to the

// length of SourceString. The line below is missing

// from the original stub SDV function

DestinationString->Length = strlen(SourceString);

}

if (SourceString== NULL)

{

DestinationString->Length = 0;

DestinationString->MaximumLength = 0;

}


Summary

Summary

  • Described optimizations implemented in Yogi

  • Evaluated optimizations on the WDM test suite

  • Empirical data used to decide which optimizations to include in Yogi

  • We believe that this detailed empirical study of optimizations will enable tool builders to decide which optimizations to include and how to engineer their tools

  • Download:http://research.microsoft.com/yogi


Yogi part 3

Bolt

  • Bolt: a generic framework that uses MapReduce style parallelism to scale top-down analysis

Intraprocedural

analysis

Bolt

Queries

Yogi

Querying

summaries

sumDB


Interprocedural analysis1

Interprocedural analysis

Y

not-may summary: perform refinement

must summary : generate test and

extend frontier

can we parallelize this algorithm?


Main idea

Main idea

intmain (int y){

if(*)

x = foo(y);

else

x = bar(y);

if (x < 0)

error();

}

Analyze queries in parallel!


Is it worth it

Is it worth it?


Introducing bolt

Introducing Bolt

Intraprocedural

analysis

Bolt

Queries

Yogi

Querying

summaries

sumDB


Modification to intraprocedura l yogi

Modification to intraprocedural Yogi

Q1

Done

(add summary to sumDB)

Q1

Q1

Yogi

Blocked

(add new sub-queries)

Q1

Ready

(may add new sub-queries)

sumDB


Lifecycle of a query

Lifecycle of a query

Reduce

blocked

ready

Reduce

done


In a nutshell

In a nutshell …

  • Bolt uses a MapReduce-style parallelization:

    • Map stage: Run each ready query on a separate instance of Yogi

    • Reduce stage: Resolve dependencies between queries

  • Terminate when the main query has an answer in sumDB


Example1

Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Yogi

Map

sumDB

initially empty


Example2

Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Yogi

Map

Reduce

sumDB


Example3

Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Map

Yogi

Yogi

Yogi

sumDB


Example4

Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Map

Yogi

Yogi

Yogi

Reduce

sumDB


Implementation1

Implementation

let bolt () =

while (initQuery.isNotDone()) do

worklist := Async.AsParallel

[for q in worklist -> async{return yogi q}];

reduce ();

done;


Evaluation

Evaluation

  • Benchmarks

    • 45 Windows device drivers and 150 safety properties

    • Picked 50 driver+property pairs taking 1000+ seconds on sequential version

  • Experimental setup

    • 8 processor cores

    • Artificial throttle: number of threads per Map stage


Evaluation1

Evaluation

  • Cumulative results on 50 checks, 8-cores, 64 threads


In depth analysis

In-depth analysis

Driver:toastmon(25KLoC)


In depth analysis1

In-depth analysis

Driver:toastmon(25KLoC)


In depth analysis2

In-depth analysis

2

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis3

In-depth analysis

4

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis4

In-depth analysis

8

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis5

In-depth analysis

8

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis6

In-depth analysis

16

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis7

In-depth analysis

32

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis8

In-depth analysis

64

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


Observations

Observations

  • Boltachieves an

    • average speedup of 3.7x

    • maximum speedup of 7.4x

  • Boltverified drivers that were previously out of reach for Yogiand other tools


Summary1

Summary

  • Bolt: a general framework for parallelizing a large class of program analysis

  • A may-must instantiation of Boltbased onYogi

    • Evaluation on a large set of Windows device drivers

    • Speedups up to 7.4x on 8-cores

    • Solving problems sequential version cannot solve

Parallel

Interprocedural

analysis

Intraprocedural analysis

Bolt


Recap and conclusions

Recap and conclusions…

  • So what did we learn, and what next?


Yogi main ideas

Yogi: main ideas

  • Combining verification and testing is interesting in theory and practice

    • Theory:

      • map out the “space” between testing and verification. Can use tests to do proofs, if we observe how the tests run!

      • interesting interplay between “may” and “must”, can write beautiful rules that bring out the duality between “may” and “must” (see POPL 2010 paper)

      • can write down interprocedural may-must algorithms using Map-Reduce parallelism

    • Practice:

      • can verify properties by running tests, and reducing load on theorem prover

      • can use tests and must-alias analysis to come up with hypothesis for doing proofs

      • can persist must and may summaries and scale precise analysis to large pieces of code, and use parallelism to scale even more!


Future directions

Future directions

  • Integrate with interpolants/abduction. So far Yogi has concentrated on “where to refine”, and not much on “how to refine” or “which predicates to use to refine” (CAV ‘12)

  • How much parallelism is inherently there for analyzing very large programs? Can parallelism help?


Yogi part 3

PLDI 2012 tutorial

http://research.microsoft.com/yogi/pldi2012.aspx

{adityan, [email protected]


  • Login