Yogi part 3
This presentation is the property of its rightful owner.
Sponsored Links
1 / 58

Yogi – Part 3 PowerPoint PPT Presentation


  • 54 Views
  • Uploaded on
  • Presentation posted in: General

Yogi – Part 3. Engineering the tool and parallelization. Windows Device Drivers. do { //get the write lock KeAcquireSpinLock (& devExt -> writeListLock ); nPacketsOld = nPackets ; request = devExt -> WriteListHeadVa ; if(request && request->status){

Download Presentation

Yogi – Part 3

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Yogi – Part 3

Engineering the tool and parallelization


Windows Device Drivers

do {

//get the write lock

KeAcquireSpinLock(&devExt->writeListLock);

nPacketsOld = nPackets;

request = devExt->WriteListHeadVa;

if(request && request->status){

devExt->WriteListHeadVa = request->Next;

KeReleaseSpinLock(&devExt->writeListLock);

irp = request->irp;

if(request->status > 0){

irp->IoStatus.Status = STATUS_SUCCESS;

irp->IoStatus.Information = request->Status;

}

else{

irp->IoStatus.Status = STATUS_UNSUCCESSFUL;

irp->IoStatus.Information = request->Status;

}

SmartDevFreeBlock(request);

IoCompleteRequest(irp, IO_NO_INCREMENT);

nPackets++;

}

} while (nPackets != nPacketsOld);

KeReleaseSpinLock(&devExt->writeListLock);

Unlocked

U

L

Locked

U

L

Error


Read for

understanding

Drive testing

tools

API Usage Rules

(SLIC)

New API rules

Defects

Software Model

Checking

100% path

coverage

Rules

Static Driver Verifier

Development

Testing

Source Code


Read for

understanding

Drive testing

tools

API Usage Rules

(SLIC)

New API rules

Defects

Software Model

Checking

100% path

coverage

Rules

Static Driver Verifier

Development

Testing

Source Code


Architecture of Yogi

C Program

slamcl

Slam.li database

SLIC property

slicc

Initial functionsummaries

Instrumented

Slam.li database

Alias and mod-ref information

li2yir

polymorphic

region graphs

Yogi IR (yir)

YSim

YAbsMan

Proof that program satisfies property

Test case that exposes a bug

Z3 theorem prover


Implementation

  • ~6K lines of Ocaml

  • Z3theorem prover

  • Integrated with SDVfor Windows

  • Engineering effort: 3 person years

  • F# version available with SDVRP

    • http://research.microsoft.com/yogi


Optimizations

  • Share our experiences in making Yogi robust, scalable and industrial strength

  • Several of the implemented optimizations are folklore

    • Very difficult to design tools that are bug free evaluating optimizations is hard!

    • Our empirical evaluation gives tool builders information about what gains can be realistically expected from optimizations

    • Details in “An empirical study of optimizations in Yogi, ICSE ’10“

  • Vanilla implementation of algorithms:

    • (flpydisk, CancelSpinLock) took 2 hours

  • Algorithms + engineering + optimizations:

    • (flpydisk, CancelSpinLock) took less than 1 second!


Optimizations

  • Initial abstraction from property predicates

  • Relevance heuristics for predicate abstraction

    • Suitable predicates (SP)

    • Control dependence predicates (CD)

  • Summaries for procedures

  • Thresholds for tests


Evaluation setup

  • Benchmarks:

    • 30 WDM drivers and 83 properties (2490 runs)

    • Anecdotal belief: most bugs in the tools are usually caught with this test suite

  • Presentation methodology:

    • Group optimizations logically such that related optimizations are in the same group

    • Total time taken, total number of defects found for every possible choice of enabling/disabling each optimization in the group


Initial abstraction

state {

enum {Locked = 0, Unlocked = 1}

state = Unlocked;

}

KeAcquireCancelSpinlock.Entry

{

if (state != Locked) {

state = Locked;

}

else

abort;

}

KeReleaseCancelSpinlock.Entry

{

if (state == Locked) {

state = Unlocked;

}

else

abort;

}

0

0

0

1

1

1


Empirical results

16%


Relevance heuristics (SP)

Irrelevant?

  • Avoid irrelevant conjuncts

A

B

C

D

A

B

C

C

D


Relevance heuristics (CD)

  • Abstract assume statements that are not potentially relevant by skip statements

  • If Yogi proves that the program satisfies property, we are done.

  • Otherwise, validate the error trace and refine the abstraction by putting back assume statements, if the error trace is spurious


Example: SP heuristic

int x;

void foo() {

bool protect = true;

if (x > 0)

protect = false;

if (protect)

KeAcquireCancelSpinLock();

for (i = 0; i < 1000; i++) {

a[i] = readByte(i);

}

if (protect)

KeReleaseCancelSpinLock();

}

A

B

C

D

A

B

C

C

D


Example: SP heuristic

int x;

void foo() {

bool protect = true;

if (x > 0)

protect = false;

if (protect)

KeAcquireCancelSpinLock();

for (i = 0; i < 1000; i++) {

a[i] = readByte(i);

}

if (protect)

KeReleaseCancelSpinLock();

}

A

B

C

D

A

B

C

C

D


Example: CD heuristic

int x;

void foo() {

bool protect = true;

if (x > 0)

protect = false;

if (protect)

KeAcquireCancelSpinLock();

for (i = 0; i < 1000; i++) {

a[i] = readByte(i);

}

if (protect)

KeReleaseCancelSpinLock();

}


Empirical results

10%


Empirical results

16%


Empirical results

25%


Interprocedural analysis

  • Yogi performs a compositional analysis

    • : Is it possible to execute starting from state and reach state ?

  • Global modification analysis

  • Compositional May-Must analysis (POPL 2010)


Example

A

B

C

D

A

B

C

C

foo(…)

D


Empirical results

32%


Empirical results

28%


Empirical results

42%


Testing

  • Yogi relies on tests for “cheap” reachability

  • Long tests

    • avoiding several potential reachability queries

    • results in too many states and thus memory consumption

  • Test thresholds: time vs. space tradeoff


Empirical evaluation


Modeling the environment

if (DestinationString)

{

DestinationString->Buffer = SourceString;

// DestinationString->Length should be set to the

// length of SourceString. The line below is missing

// from the original stub SDV function

DestinationString->Length = strlen(SourceString);

}

if (SourceString== NULL)

{

DestinationString->Length = 0;

DestinationString->MaximumLength = 0;

}


Summary

  • Described optimizations implemented in Yogi

  • Evaluated optimizations on the WDM test suite

  • Empirical data used to decide which optimizations to include in Yogi

  • We believe that this detailed empirical study of optimizations will enable tool builders to decide which optimizations to include and how to engineer their tools

  • Download:http://research.microsoft.com/yogi


Bolt

  • Bolt: a generic framework that uses MapReduce style parallelism to scale top-down analysis

Intraprocedural

analysis

Bolt

Queries

Yogi

Querying

summaries

sumDB


Interprocedural analysis

Y

not-may summary: perform refinement

must summary : generate test and

extend frontier

can we parallelize this algorithm?


Main idea

intmain (int y){

if(*)

x = foo(y);

else

x = bar(y);

if (x < 0)

error();

}

Analyze queries in parallel!


Is it worth it?


Introducing Bolt

Intraprocedural

analysis

Bolt

Queries

Yogi

Querying

summaries

sumDB


Modification to intraprocedural Yogi

Q1

Done

(add summary to sumDB)

Q1

Q1

Yogi

Blocked

(add new sub-queries)

Q1

Ready

(may add new sub-queries)

sumDB


Lifecycle of a query

Reduce

blocked

ready

Reduce

done


In a nutshell …

  • Bolt uses a MapReduce-style parallelization:

    • Map stage: Run each ready query on a separate instance of Yogi

    • Reduce stage: Resolve dependencies between queries

  • Terminate when the main query has an answer in sumDB


Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Yogi

Map

sumDB

initially empty


Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Yogi

Map

Reduce

sumDB


Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Map

Yogi

Yogi

Yogi

sumDB


Example

void main (inti)

{

if (i> 0)

x = foo(i);

else if (j > -10)

x = bar(i);

else

x = baz(j);

y = x + 5;

if (y <= -5)

error();

}

Map

Yogi

Yogi

Yogi

Reduce

sumDB


Implementation

let bolt () =

while (initQuery.isNotDone()) do

worklist := Async.AsParallel

[for q in worklist -> async{return yogi q}];

reduce ();

done;


Evaluation

  • Benchmarks

    • 45 Windows device drivers and 150 safety properties

    • Picked 50 driver+property pairs taking 1000+ seconds on sequential version

  • Experimental setup

    • 8 processor cores

    • Artificial throttle: number of threads per Map stage


Evaluation

  • Cumulative results on 50 checks, 8-cores, 64 threads


In-depth analysis

Driver:toastmon(25KLoC)


In-depth analysis

Driver:toastmon(25KLoC)


In-depth analysis

2

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In-depth analysis

4

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In-depth analysis

8

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In-depth analysis

8

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In-depth analysis

16

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In-depth analysis

32

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


In-depth analysis

64

Driver:toastmon(25KLoC)

Property: PnpIrpCompletion


Observations

  • Boltachieves an

    • average speedup of 3.7x

    • maximum speedup of 7.4x

  • Boltverified drivers that were previously out of reach for Yogiand other tools


Summary

  • Bolt: a general framework for parallelizing a large class of program analysis

  • A may-must instantiation of Boltbased onYogi

    • Evaluation on a large set of Windows device drivers

    • Speedups up to 7.4x on 8-cores

    • Solving problems sequential version cannot solve

Parallel

Interprocedural

analysis

Intraprocedural analysis

Bolt


Recap and conclusions…

  • So what did we learn, and what next?


Yogi: main ideas

  • Combining verification and testing is interesting in theory and practice

    • Theory:

      • map out the “space” between testing and verification. Can use tests to do proofs, if we observe how the tests run!

      • interesting interplay between “may” and “must”, can write beautiful rules that bring out the duality between “may” and “must” (see POPL 2010 paper)

      • can write down interprocedural may-must algorithms using Map-Reduce parallelism

    • Practice:

      • can verify properties by running tests, and reducing load on theorem prover

      • can use tests and must-alias analysis to come up with hypothesis for doing proofs

      • can persist must and may summaries and scale precise analysis to large pieces of code, and use parallelism to scale even more!


Future directions

  • Integrate with interpolants/abduction. So far Yogi has concentrated on “where to refine”, and not much on “how to refine” or “which predicates to use to refine” (CAV ‘12)

  • How much parallelism is inherently there for analyzing very large programs? Can parallelism help?


PLDI 2012 tutorial

http://research.microsoft.com/yogi/pldi2012.aspx

{adityan, [email protected]


  • Login