Yogi part 3
Download
1 / 58

Yogi – Part 3 - PowerPoint PPT Presentation


  • 72 Views
  • Uploaded on

Yogi – Part 3. Engineering the tool and parallelization. Windows Device Drivers. do { //get the write lock KeAcquireSpinLock (& devExt -> writeListLock ); nPacketsOld = nPackets ; request = devExt -> WriteListHeadVa ; if(request && request->status){

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Yogi – Part 3' - uma-terrell


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Yogi part 3
Yogi – Part 3

Engineering the tool and parallelization


Windows device drivers
Windows Device Drivers

do {

//get the write lock

KeAcquireSpinLock(&devExt->writeListLock);

nPacketsOld = nPackets;

request = devExt->WriteListHeadVa;

if(request && request->status){

devExt->WriteListHeadVa = request->Next;

KeReleaseSpinLock(&devExt->writeListLock);

irp = request->irp;

if(request->status > 0){

irp->IoStatus.Status = STATUS_SUCCESS;

irp->IoStatus.Information = request->Status;

}

else{

irp->IoStatus.Status = STATUS_UNSUCCESSFUL;

irp->IoStatus.Information = request->Status;

}

SmartDevFreeBlock(request);

IoCompleteRequest(irp, IO_NO_INCREMENT);

nPackets++;

}

} while (nPackets != nPacketsOld);

KeReleaseSpinLock(&devExt->writeListLock);

Unlocked

U

L

Locked

U

L

Error


Read for

understanding

Drive testing

tools

API Usage Rules

(SLIC)

New API rules

Defects

Software Model

Checking

100% path

coverage

Rules

Static Driver Verifier

Development

Testing

Source Code


Read for

understanding

Drive testing

tools

API Usage Rules

(SLIC)

New API rules

Defects

Software Model

Checking

100% path

coverage

Rules

Static Driver Verifier

Development

Testing

Source Code


Architecture of yogi
Architecture of Yogi

C Program

slamcl

Slam.li database

SLIC property

slicc

Initial functionsummaries

Instrumented

Slam.li database

Alias and mod-ref information

li2yir

polymorphic

region graphs

Yogi IR (yir)

YSim

YAbsMan

Proof that program satisfies property

Test case that exposes a bug

Z3 theorem prover


Implementation
Implementation

  • ~6K lines of Ocaml

  • Z3theorem prover

  • Integrated with SDVfor Windows

  • Engineering effort: 3 person years

  • F# version available with SDVRP

    • http://research.microsoft.com/yogi


Optimizations
Optimizations

  • Share our experiences in making Yogi robust, scalable and industrial strength

  • Several of the implemented optimizations are folklore

    • Very difficult to design tools that are bug free evaluating optimizations is hard!

    • Our empirical evaluation gives tool builders information about what gains can be realistically expected from optimizations

    • Details in “An empirical study of optimizations in Yogi, ICSE ’10“

  • Vanilla implementation of algorithms:

    • (flpydisk, CancelSpinLock) took 2 hours

  • Algorithms + engineering + optimizations:

    • (flpydisk, CancelSpinLock) took less than 1 second!


O ptimizations
Optimizations

  • Initial abstraction from property predicates

  • Relevance heuristics for predicate abstraction

    • Suitable predicates (SP)

    • Control dependence predicates (CD)

  • Summaries for procedures

  • Thresholds for tests


Evaluation setup
Evaluation setup

  • Benchmarks:

    • 30 WDM drivers and 83 properties (2490 runs)

    • Anecdotal belief: most bugs in the tools are usually caught with this test suite

  • Presentation methodology:

    • Group optimizations logically such that related optimizations are in the same group

    • Total time taken, total number of defects found for every possible choice of enabling/disabling each optimization in the group


Initial abstraction
Initial abstraction

state {

enum {Locked = 0, Unlocked = 1}

state = Unlocked;

}

KeAcquireCancelSpinlock.Entry

{

if (state != Locked) {

state = Locked;

}

else

abort;

}

KeReleaseCancelSpinlock.Entry

{

if (state == Locked) {

state = Unlocked;

}

else

abort;

}

0

0

0

1

1

1



Relevance heuristics sp
Relevance heuristics (SP)

Irrelevant?

  • Avoid irrelevant conjuncts

A

B

C

D

A

B

C

C

D


Relevance heuristics cd
Relevance heuristics (CD)

  • Abstract assume statements that are not potentially relevant by skip statements

  • If Yogi proves that the program satisfies property, we are done.

  • Otherwise, validate the error trace and refine the abstraction by putting back assume statements, if the error trace is spurious


Example sp heuristic
Example: SP heuristic

int x;

void foo() {

bool protect = true;

if (x > 0)

protect = false;

if (protect)

KeAcquireCancelSpinLock();

for (i = 0; i < 1000; i++) {

a[i] = readByte(i);

}

if (protect)

KeReleaseCancelSpinLock();

}

A

B

C

D

A

B

C

C

D


Example sp heuristic1
Example: SP heuristic

int x;

void foo() {

bool protect = true;

if (x > 0)

protect = false;

if (protect)

KeAcquireCancelSpinLock();

for (i = 0; i < 1000; i++) {

a[i] = readByte(i);

}

if (protect)

KeReleaseCancelSpinLock();

}

A

B

C

D

A

B

C

C

D


Example cd heuristic
Example: CD heuristic

int x;

void foo() {

bool protect = true;

if (x > 0)

protect = false;

if (protect)

KeAcquireCancelSpinLock();

for (i = 0; i < 1000; i++) {

a[i] = readByte(i);

}

if (protect)

KeReleaseCancelSpinLock();

}





Interprocedural analysis
Interprocedural analysis

  • Yogi performs a compositional analysis

    • : Is it possible to execute starting from state and reach state ?

  • Global modification analysis

  • Compositional May-Must analysis (POPL 2010)


Example
Example

A

B

C

D

A

B

C

C

foo(…)

D





Testing
Testing

  • Yogi relies on tests for “cheap” reachability

  • Long tests

    • avoiding several potential reachability queries

    • results in too many states and thus memory consumption

  • Test thresholds: time vs. space tradeoff



Modeling the environment
Modeling the environment

if (DestinationString)

{

DestinationString->Buffer = SourceString;

// DestinationString->Length should be set to the

// length of SourceString. The line below is missing

// from the original stub SDV function

DestinationString->Length = strlen(SourceString);

}

if (SourceString== NULL)

{

DestinationString->Length = 0;

DestinationString->MaximumLength = 0;

}


Summary
Summary

  • Described optimizations implemented in Yogi

  • Evaluated optimizations on the WDM test suite

  • Empirical data used to decide which optimizations to include in Yogi

  • We believe that this detailed empirical study of optimizations will enable tool builders to decide which optimizations to include and how to engineer their tools

  • Download:http://research.microsoft.com/yogi


Bolt

  • Bolt: a generic framework that uses MapReduce style parallelism to scale top-down analysis

Intraprocedural

analysis

Bolt

Queries

Yogi

Querying

summaries

sumDB


Interprocedural analysis1
Interprocedural analysis

Y

not-may summary: perform refinement

must summary : generate test and

extend frontier

can we parallelize this algorithm?


Main idea
Main idea

intmain (int y){

if(*)

x = foo(y);

else

x = bar(y);

if (x < 0)

error();

}

Analyze queries in parallel!



Introducing bolt
Introducing Bolt

Intraprocedural

analysis

Bolt

Queries

Yogi

Querying

summaries

sumDB


Modification to intraprocedura l yogi
Modification to intraprocedural Yogi

Q1

Done

(add summary to sumDB)

Q1

Q1

Yogi

Blocked

(add new sub-queries)

Q1

Ready

(may add new sub-queries)

sumDB


Lifecycle of a query
Lifecycle of a query

Reduce

blocked

ready

Reduce

done


In a nutshell
In a nutshell …

  • Bolt uses a MapReduce-style parallelization:

    • Map stage: Run each ready query on a separate instance of Yogi

    • Reduce stage: Resolve dependencies between queries

  • Terminate when the main query has an answer in sumDB


Example1
Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Yogi

Map

sumDB

initially empty


Example2
Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Yogi

Map

Reduce

sumDB


Example3
Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Map

Yogi

Yogi

Yogi

sumDB


Example4
Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Map

Yogi

Yogi

Yogi

Reduce

sumDB


Implementation1
Implementation

let bolt () =

while (initQuery.isNotDone()) do

worklist := Async.AsParallel

[for q in worklist -> async{return yogi q}];

reduce ();

done;


Evaluation
Evaluation

  • Benchmarks

    • 45 Windows device drivers and 150 safety properties

    • Picked 50 driver+property pairs taking 1000+ seconds on sequential version

  • Experimental setup

    • 8 processor cores

    • Artificial throttle: number of threads per Map stage


Evaluation1
Evaluation

  • Cumulative results on 50 checks, 8-cores, 64 threads


In depth analysis
In-depth analysis

Driver:toastmon(25KLoC)


In depth analysis1
In-depth analysis

Driver:toastmon(25KLoC)


In depth analysis2
In-depth analysis

2

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis3
In-depth analysis

4

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis4
In-depth analysis

8

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis5
In-depth analysis

8

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis6
In-depth analysis

16

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis7
In-depth analysis

32

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In depth analysis8
In-depth analysis

64

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


Observations
Observations

  • Boltachieves an

    • average speedup of 3.7x

    • maximum speedup of 7.4x

  • Boltverified drivers that were previously out of reach for Yogiand other tools


Summary1
Summary

  • Bolt: a general framework for parallelizing a large class of program analysis

  • A may-must instantiation of Boltbased onYogi

    • Evaluation on a large set of Windows device drivers

    • Speedups up to 7.4x on 8-cores

    • Solving problems sequential version cannot solve

Parallel

Interprocedural

analysis

Intraprocedural analysis

Bolt


Recap and conclusions
Recap and conclusions…

  • So what did we learn, and what next?


Yogi main ideas
Yogi: main ideas

  • Combining verification and testing is interesting in theory and practice

    • Theory:

      • map out the “space” between testing and verification. Can use tests to do proofs, if we observe how the tests run!

      • interesting interplay between “may” and “must”, can write beautiful rules that bring out the duality between “may” and “must” (see POPL 2010 paper)

      • can write down interprocedural may-must algorithms using Map-Reduce parallelism

    • Practice:

      • can verify properties by running tests, and reducing load on theorem prover

      • can use tests and must-alias analysis to come up with hypothesis for doing proofs

      • can persist must and may summaries and scale precise analysis to large pieces of code, and use parallelism to scale even more!


Future directions
Future directions

  • Integrate with interpolants/abduction. So far Yogi has concentrated on “where to refine”, and not much on “how to refine” or “which predicates to use to refine” (CAV ‘12)

  • How much parallelism is inherently there for analyzing very large programs? Can parallelism help?


PLDI 2012 tutorial

http://research.microsoft.com/yogi/pldi2012.aspx

{adityan, [email protected]


ad