By: Drew Wicke Jointly with: Sri Hari Krishna Narayanan and Paul Hovland

1 / 19

# By: Drew Wicke Jointly with: Sri Hari Krishna Narayanan and Paul Hovland - PowerPoint PPT Presentation

Identifying Active Variables to Improve the Performance of Operator Overloading Automatic Differentiation. By: Drew Wicke Jointly with: Sri Hari Krishna Narayanan and Paul Hovland. What is Automatic Differentiation?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'By: Drew Wicke Jointly with: Sri Hari Krishna Narayanan and Paul Hovland' - luka

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Identifying Active Variables to Improve the Performance of Operator Overloading Automatic Differentiation

By: Drew Wicke

Jointly with:

Sri Hari Krishna Narayanan and Paul Hovland

What is Automatic Differentiation?

• A means of computing the derivative of a function within a computer program
• Operator Overloading AD – Utilizes features of the programming language to alter the meaning of mathematical operators to compute the derivative

• Made by Sandia National Laboratories
• A package in the Trilinos framework

DERIV_TYPE operator* (DERIV_TYPE other) {

this->val = this->val() * other.val();

this->dx = this->val() * other.dx() + other->val() * this->dx();

return *this;

}

Example Code

//Function where x is the independent f

// is the dependent

voidfoo(double *x, double *f){

doubletheConst = 3.14 * 2 – 6;

double div = ((*x) / 2);

doublesquarePoly = (*x) * (*x) + 45;

if (squarePoly >= 100) {

(*f) = div * theConst;

}

else{

doublealtInput = (*x) * theConst;

(*f) = div * altInput + theConst;

}

}

int main()

{

doublex, f;

foo(&x, &f);

return0;

}

voidfoo(DERIV_TYPE_double x, DERIV_TYPE_double f);

int main()

{

DERIV_TYPE_double x, f;// independent and dependent vars

// init value of x and f is set to zero and deriv

// value is set to zero

x= 2.0; // set the initial

x.diff(0, 1); // set that x is independent

foo(&x, &f); // call foo with x and f as args

// print the value of x and the value of f

// by calling the method val which returns

// the non-derivative value

printf("f(%f) = %f\n", x.val(), f.val());

// print the derivative by calling the dx method

// and pass zero as the argtodxto specify what

// independent variable we want the derivative

// with respect to

printf("df/dx = %f\n", f.dx(0));

return0;

}

Example Continued

//Function where x is the input f is the output

voidfoo(DERIV_TYPE_double *x, DERIV_TYPE_double *f){

DERIV_TYPE_doubletheConst = 3.14 * 2 – 6;

DERIV_TYPE_double div = ((*x) / 2);

DERIV_TYPE_doublesquarePoly = (*x) * (*x) + 45;

if (squarePoly >= 100) {

(*f) = div * theConst;

}

else{

DERIV_TYPE_doublealtInput = (*x) * theConst;

(*f) = div * altInput + theConst;

}

}

Why That Code is Not Optimal

• Vary variables are those whose value is computed using an independent variable.
• Useful variablesare used to compute the value of the dependent variable
• Active variablesare both vary and useful.

voidfoo(DERIV_TYPE_double *x, DERIV_TYPE_double *f){

DERIV_TYPE_doublesquarePoly= (*x) * (*x) + 45;

DERIV_TYPE_doubletheConst= 3.14 * 2 – 6;

DERIV_TYPE_double div = ((*x) / 2);

if (squarePoly >= 100)

(*f) = div * theConst;

else{

DERIV_TYPE_doublealtInput= (*x) * theConst;

(*f) = div * altInput + theConst;

}

}

What Does Our Tool Do?

• Replaces the manual process, which is slow and overestimates the number active variables.

voidfoo(DERIV_TYPE_double *x, DERIV_TYPE_double *f){

double squarePoly= (*x) * (*x) + 45; // vary

double theConst= 3.14 * 2 – 6; // useful

DERIV_TYPE_double div = ((*x) / 2); // active

if (squarePoly >= 100) {

(*f) = div * theConst;

}

else{

DERIV_TYPE_doublealtInput= (*x) * theConst;

(*f) = div * altInput + theConst;

}

}

Activity Analysis Tool Flow

ROSE AST

OpenAnalysis ICFG

Input code

convert

int main(){

doubleval;

val = 4.3;

return0;

}

convert

Useful Analysis

Vary Analysis

Activity Analysis

Output

code

Change type

of active variables

int main(){

DERIV_TYPE_doubleval;

val = 4.3;

return0;

}

Challenges in Type Changing

• Typedefs – must extract base type and all other types such as pointers to set to the derivative type.
• Structs – must copy struct and change all variables with the body of the struct to the Sacado derivative type.

struct mine { doublemdou; };

struct mine smin; // active (before)

typedefdouble* fir;

typedef fir* sec;

sec test;//active (before)

structmine { doublemdou; };

structDERIV_CLASS_mine {

DERIV_TYPE_doublemdou;

}

typedefstructDERIV_CLASS_mineDERIV_TYPE_mine;

DERIV_TYPE_minesmin;// (after)

// after:

DERIV_TYPE_double **test;

Initial Code

//Function where x is the independent f is the dependent

voidfoo(double *x, double *f){

# pragma \$adic_dep,f// do some math

doubletheConst = 3.14 * 2 – 6;

double div = ((*x) / 2);

doublesquarePoly = (*x) * (*x) + 45;

if (squarePoly >= 100) {

(*f) = div * theConst;

}

else{

doublealtInput = (*x) * theConst;

(*f) = div * altInput + theConst;

}

}

Resulting Code

//Function where x is the input f is the output

voidfoo(DERIV_TYPE_double *x, DERIV_TYPE_double *f){

doubletheConst = 3.14 * 2 – 6; // .28

DERIV_TYPE_double div = ((*x) / 2);

doublesquarePoly = (*x) * (*x) + 45;// ERROR must downcast!

if (squarePoly >= 100) {

(*f) = div * theConst;

}

else{

DERIV_TYPE_doublealtInput = (*x) * theConst;

(*f) = div * altInput + theConst;

}

}

Add a downcast if the left hand side is vary and the right hand side has an active variable.

Resulting Code With Downcast

//Function where x is the input f is the output

voidfoo(DERIV_TYPE_double *x, DERIV_TYPE_double *f){

doubletheConst = 3.14 * 2 – 6; // .28

DERIV_TYPE_double div = ((*x) / 2);

doublesquarePoly = ADValue((*x) * (*x) + 45); // Problem solved

if (squarePoly >= 100) {

(*f) = div * theConst;

}

else{

DERIV_TYPE_doublealtInput = (*x) * theConst;

(*f) = div * altInput + theConst;

}

}

Tool Limitations

• Typedefed structs: typedef struct{…} myStruct;
• C++ structs, classes, templates
• Function pointers: double (*func)(double, double)
• Array parameters due to not being made active: void foo(double myParam[]);

Conclusion and Future Work

• Benchmarking the tool

Thank You

How Does the Tool Work?

• Creates Abstract Syntax Tree (AST) of code using ROSE
• int main(){
• double val;
• val = 4.3;
• return0;
• }
• Converts AST to an

OpenAnalysis form

definition

Creates Interprocedural Control Flow Graph (ICFG)

Images from:

M. J. Harrold, G. Rothermel, and S. Sinha, “Computation of Interprocedural Control Dependence”, Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ‘98), pp. 11-20, March 1998

What Analyses are Used?

• Vary Analysis – Identifies variables that vary with the independent variable, but are not used in the output
• Useful Analysis – Identifies variables that are not vary but contribute to the output
• Activity Analysis – Identifies active variables, those that depend on the value of an input variable to compute the output variable. The intersection of vary and useful variables.

What Does Our Tool Do?

• Replaces the manual process that is slow and overestimates, with our tool.
• Identifies active variables using the source code analysis toolkit OpenAnalysis
• Changes the type of active variables to a typedef type using the source transformation tool ROSE

voidfoo(DERIV_TYPE_double *x, DERIV_TYPE_double *f){

double theConst= 3.14 * 2 – 6;

DERIV_TYPE_double div = ((*x) / 2);

double squarePoly= (*x) * (*x) + 45;

if (squarePoly >= 100) {

(*f) = div * theConst;

}

else{

DERIV_TYPE_doublealtInput= (*x) * theConst;

(*f) = div * altInput + theConst;

}

}