00:00

Understanding LLVM Line Coverage Profiling

Learn how line coverage profiling is implemented in LLVM using built-in instrumentation techniques such as PGO probes, IR rewriting, and coverage mapping. Explore the process of inserting probes, interpreting coverage measurements, and generating coverage reports to assess the coverage of a test suite. Discover how to make profile files human-readable and generate reports for better analysis.

fota
Download Presentation

Understanding LLVM Line Coverage Profiling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EXERCISE #27 LLVM INSTRUMENTATION REVIEW Write your name and answer the following on a piece of paper How is line coverage profiling implemented in LLVM? 1

  2. ADMINISTRIVIA AND ANNOUNCEMEN TS

  3. LLVM INSTRUMENTATION EECS 677: Software Security Evaluation Drew Davidson

  4. 4 WHERE WE’RE AT EXPLORING PROGRAM INSTRUMENTATION An approach to dynamic analysis

  5. 5 PREVIOUSLY: STATIC INSTRUMENTATION REVIEW: LAST LECTURE PRACTICAL TOOLS FOR PROGRAM INSTRUMENTING GCC – Internal transformation passes LLVM opt – IR rewriting USAGE OF LLVM BUILT-IN INSTRUMENTATION ANALYSIS SETUP FOR A CUSTOM LLVM ANALYSIS

  6. 6 THIS LESSON: WRITING INSTRUMENTATION OUTLINE / OVERVIEW PRACTICAL TOOLS FOR PROGRAM INSTRUMENTING GCC – Internal transformation passes LLVM opt – IR rewriting Providing more detail USAGE OF LLVM BUILT-IN INSTRUMENTATION ANALYSIS Presenting an example SETUP FOR A CUSTOM LLVM ANALYSIS

  7. 7 SETUP / ASSUMPTIONS LLVM BUILT-IN INSTRUMENTATION THIS PORTION OF THE LECTURE USES A CLANG++ INSTALLATION. Should work for many versions of LLVM (tested on clang++-9) Works on clang++14 (which is installed on the cycle servers) INSTALLATION (ON A LOCAL MACHINE) sudo apt install clang++ llvm-dev

  8. 8 LLVM COVERAGE INSTRUMENTATION LLVM BUILT-IN INSTRUMENTATION GOAL: ASSESS THE COVERAGE OF A TEST SUITE APPROACH: USE LLVM’S BUILT-IN INSTRUCTION INSTRUMENTATION Piggyback on LLVM’s PGO facilities 1) Insert PGO probes 2) Interpret probes as coverage measurements 3) Generate a coverage report

  9. 9 LLVM: INSERTING PGO PROBES LLVM BUILT-IN INSTRUMENTATION -fprofile-instr-generate Generate profile information at the source instruction level -fprofile-generate Generate profile information at the IR level We can actually see the instrumented code live!

  10. 10 LLVM: INSERTING PGO PROBES LLVM BUILT-IN INSTRUMENTATION Let’s write a simple LLVM program, then observe the probes… clang++ prog.ll –o prog-instr.ll -fprofile-generate –emit-llvm

  11. 11 LLVM COVERAGE INSTRUMENTATION LLVM BUILT-IN INSTRUMENTATION GOAL: ASSESS THE COVERAGE OF A TEST SUITE APPROACH: USE LLVM’S BUILT-IN INSTRUCTION INSTRUMENTATION Piggyback on LLVM’s PGO facilities 1) Insert PGO probes 2) Interpret probes as coverage measurements 3) Generate a coverage report

  12. 12 LLVM: COVERAGE MAPPING LLVM BUILT-IN INSTRUMENTATION For understanding line coverage, we need to map changes to source code clang++ prog.ll –o prog -fprofile-instr-generate –emit-llvm -fcoverage-mapping This will cause the program to output an additional coverage file in the location of the environment variable LLVM_PROFILE_FILE export LLVM_PROFILE_FILE=test1.prof

  13. 13 LLVM COVERAGE INSTRUMENTATION LLVM BUILT-IN INSTRUMENTATION GOAL: ASSESS THE COVERAGE OF A TEST SUITE APPROACH: USE LLVM’S BUILT-IN INSTRUCTION INSTRUMENTATION Piggyback on LLVM’s PGO facilities 1) Insert PGO probes 2) Interpret probes as coverage measurements 3) Generate a coverage report

  14. 14 LLVM: COVERAGE REPORT LLVM BUILT-IN INSTRUMENTATION The profile file is useful for a variety of things (i.e. PGO). As such, it is not (immediately) human-readable We’ll use some extra tools to generate a readable report llvm-profdata merge -sparse test1.prof -o final.profdata llvm-cov-9 show badcalc -instr-profile=final.profdata >& profile.report

  15. 15 PUTTING IT ALL TOGETHER REVIEW: LAST LECTURE THESE COMMANDS WORK FINE FOR 1 TEST RUN, BUT WE CARE ABOUT TEST SUITES clang++ prog.ll –o prog -fprofile-instr-generate -fcoverage-mapping export LLVM_PROFILE_FILE=test1.prof ./prog export LLVM_PROFILE_FILE=test2.prof ./prog llvm-profdata merge -sparse test*.prof -o final.profdata llvm-cov-9 show badcalc -instr-profile=final.profdata >& profile.report

  16. 16 THIS LESSON: WRITING INSTRUMENTATION OUTLINE / OVERVIEW PRACTICAL TOOLS FOR PROGRAM INSTRUMENTING GCC – Internal transformation passes LLVM opt – IR rewriting Providing more detail USAGE OF LLVM BUILT-IN INSTRUMENTATION ANALYSIS Presenting an example SETUP FOR A CUSTOM LLVM ANALYSIS

  17. 17 EXAMPLE: CUSTOM LLVM INSTRUMENTATION PROGRAM INSTRUMENTATION: APPROACH LET’S REMOVE AND ADD SOME INSTRUCTIONS! Consider a simple “add2” program: #include <stdio.h> int main(int argc, const char** argv) { int num; scanf("%i", &num); printf("%i\n", num + 2); return 0; }

  18. 18 EXAMPLE: LLVM COVERAGE INSTRUMENTATION PROGRAM INSTRUMENTATION: APPROACH LET’S TAKE IT TO THE TERMINAL!

  19. 19 WRAP-UP WE’VE DESCRIBED THE THEORY AND PRACTICE OF PROGRAM INSTRUMENTATION Next time: Consider how we generate test cases

More Related