Neural Network Verification Part 2: Formulation

1. Neural Network VerificationPart 2: Formulation

2. Self-Driving Cars Human drivers replaced by deep neural networks

3. Road Sign Classification Post Training Data Parameters W Parameters W perform multiclass classification Estimate W using training data

4. Road Sign Classification Post Test Image Parameters W Parameters W select the class of a new input image

5. Road Sign Classification Post Test Image Parameters W Small deformations can cause fatal errors Input Image Prediction Input Image Prediction Blurring, Saturation Aung et al., 2017 Evtimov et al., 2017 Pixel Errors

6. Spot the Difference “pig” (91%) “airliner” (99%)

7. Difference 0.005 x

8. Glasses Sharif, Bhagavatula, Bauer and Reiter, 2016

9. 3D Object Athalye, Engstrom, Ilyas and Kwok, 2017

10. Audio Carlini and Wagner, 2018

11. Outline • Robust Deep Learning • Formulation • Black Box Solvers

12. Robust Deep Learning Post Training Data Parameters W Identify Deformation With Error Augment Training Data Set

13. Condition on inputs Robust Deep Learning Post Is there an erroneous output? Safe Error Classifications Image Deformations Re-estimate parameters W

14. Condition on inputs Robust Deep Learning Post Is there an erroneous output? Safe Error Classifications Image Deformations Re-estimate parameters W

15. Condition on inputs Robust Deep Learning Post Is there an erroneous output? Safe Error Classifications Image Deformations Terminate when all putative outputs are safe

16. Erroneous Output Post Set of possible inputs x ∈ X E.g. X is an l∞ ball around an input x* Ground-truth class is y* Score for input x and class y is s(y;x)

17. Erroneous Output Post Find x ∈ X such that miny≠y*(s(y*;x) – s(y;x)) < 0 I.e. an input that is misclassified Or prove that there does not exist such an x An instance of Neural Network Verification

18. Outline • Robust Deep Learning • Formulation • Black Box Solvers

19. Assumption Post Piecewise linear non-linearities ReLU, MaxPool… Covers many state of the art networks Intuitions can be transferred to more general settings

20. Neural Network Verification Post Neural network f Scalar output z = f(x) E.g. in binary classification, z = s(y*;x) – s(y;x) for y ≠ y* Property: f(x) > 0 for all x∈ X Formally prove the property, or provide counter-example

21. Complex Properties Post OR clause e.g. (z1 > 0) ∨(z2> 0) ∨(z3> 0) max(z1, z2, z3) > 0 Implement using a MaxPool layer at the end

22. Complex Properties Post AND clause e.g. (z1 > 0) ∧(z2> 0) ∧(z3> 0) min(z1, z2, z3) > 0 -max(-z1, -z2, -z3) > 0 Linear + MaxPool + Linear layer at the end

23. Complex Properties Post Boolean formulas OR over linear inequalities AND over linear inequalities Any Boolean formula over linear inequalities

24. Example Post min z s.t. -2 ≤ x1≤ 2 a 1 -2 ≤ x2 ≤ 2 -1 x1 1 [-2, 2] ain = x1 + x2 z x2 [-2, 2] 1 -1 bin = x1 - x2 -1 aout = max{ain,0} b Prove that z > -5 bout = max{bin,0} z = - aout - bout

25. Example Post min z Linear constraints s.t. -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 Easy to handle ain = x1 + x2 bin = x1 - x2 aout = max{ain,0} bout = max{bin,0} z = - aout - bout

26. Example Post min z s.t. -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout = max{ain,0} Non-linear constraints bout = max{bin,0} NP-hard problem z = - aout - bout

27. Outline • Robust Deep Learning • Formulation • Black Box Solvers Cheng et al., 2017; Lomuscio et al., 2017; Tjeng et al., 2017

28. Reformulation Post aout = max{ain,0} Large constant Ma(greater than any possible ain) Binary variable δa ∈ {0,1} aout ≥ ain aout ≥ 0 aout ≤ ain + (1-δa)Ma aout ≤ δaMa

29. Reformulation Post aout = max{ain,0} Large constant Ma(greater than any possible ain) Binary variable δa ∈ {0,1} Case I: δa = 0 aout ≥ ain aout ≥ 0 aout ≤ ain + (1-δa)Ma aout ≤ δaMa

30. Reformulation Post aout = max{ain,0} Large constant Ma(greater than any possible ain) Binary variable δa ∈ {0,1} Case I: δa = 0 aout ≥ ain aout ≥ 0 aout = 0 aout ≤ ain + (1-δa)Ma aout ≤ δaMa

31. Reformulation Post aout = max{ain,0} Large constant Ma(greater than any possible ain) Binary variable δa ∈ {0,1} Case II: δa = 1 aout ≥ ain aout ≥ 0 aout ≤ ain + (1-δa)Ma aout ≤ δaMa

32. Reformulation Post aout = max{ain,0} Large constant Ma(greater than any possible ain) Binary variable δa ∈ {0,1} Case II: δa = 1 aout ≥ ain aout ≥ 0 aout = ain aout ≤ ain + (1-δa)Ma aout ≤ δaMa

33. Example Post min z s.t. -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout = max{ain,0} bout = max{bin,0} z = - aout - bout

34. Example Post min z s.t. -2 ≤ x1≤ 2 -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout ≥ ain aout ≥ 0 aout ≤ ain + (1-δa)Ma aout ≤ δaMa bout = max{bin,0} δa ∈ {0,1} z = - aout - bout

35. Example Post min z s.t. -2 ≤ x1≤ 2 Mixed Integer Linear Program -2 ≤ x2 ≤ 2 ain = x1 + x2 bin = x1 - x2 aout ≥ ain aout ≥ 0 aout ≤ ain + (1-δa)Ma aout ≤ δaMa bout ≥ bin bout ≥ 0 bout ≤ bin + (1-δb)Mb bout ≤ δbMb δa ∈ {0,1} δb ∈ {0,1} z = - aout - bout

36. MILP Formulation Post Neural network structure can help determine Ma Standard solvers e.g. Gurobi, Mosek, CPLEX Does not scale (we will see results later) Standard architectures have order of 107 parameters

37. Questions?