1 / 19

QR Code Reading Benchmark and Comparison

QR Code is widely used in our everyday life. As a two-dimensional barcode, it can store more data than 1D barcodes.There are many open-source and commercial libraries or SDKs which can read QR codes. A benchmark is needed to evaluate which one is more robust or suitable for a specific use case.<br>To know more go through this PDF or visit blog <br>https://www.dynamsoft.com/codepool/qr-code-reading-benchmark-and-comparison.html<br>

anvilewis1
Download Presentation

QR Code Reading Benchmark and Comparison

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Best QR Code Reading SDKs

  2. What is QR code? § QR Code is a type of two-dimensional barcode § Consists of black squares arranged in a square grid on a white background § Stores more data than a 1D barcodes § QR Code has full (360 degrees) reading features § QR code has a finder pattern located in three of its corners

  3. QR Code Reading Benchmark and Comparison In the following slides we are going to run a benchmark based on an image data set to find out the best QR Code reading SDKs.

  4. Dataset A comprehensive benchmark was made by the author of boofcv to compare boofcv with other 4 libraries § Performance test is made on the dataset to evaluate 5 open-source libraries § Dataset contains 536 images containing 1232 QR codes and has 16 categories § blurred § glare § pathological § bright_spots § high_version § perspective § brightness § lots § rotations § close § monitor § shadows § curved § nominal § damaged § noncompliane

  5. Performance test tool In order to run the benchmark, a performance test tool is written § The tool uses Python to run the test and provides a web interface. The Flask web framework is used. § Code can be accessed here § It is going to run on a PC device with an Intel i5-10400 CPU and 16GB memory.

  6. Evaluated libraries • Dynamsoft Barcode Reader (DBR, version: 8.6, commercial) • Commercial SDK A (version: Java 13.5, commercial) • Commercial SDK B (version: Windows 5.19.3.10, commercial) • BoofCV (version: pyboof 0.36.1, open-source) • Zxing (version: 3.4.1, open-source) • Zbar (version: pyzbar 0.1.8, open-source) • WeChat QR code detector in OpenCV (version: OpenCV 4.5.3, open-source)

  7. Evaluation metrics Performance is evaluated by reading rate and runtime. Reading rate can be achieved by dividing detected QR codes by total QR codes in images Reading rate = Detected QR codes / Total QR codes If the value of IoU (Intersection over Union) between the detected QR code’s polygon and the ground truth’s is over 0, then the QR code is considered detected. *More details about evaluation metrics can be found here

  8. Detection results Reading rate in percent Commercial SDK A Commercial SDK B OpenCV Wechat Categories Dynamsoft Zxing Zbar BoofCV blurred 69.23 38.46 26.15 23.08 38.46 40.00 55.38 bright_spots 38.14 30.93 6.19 21.65 19.59 23.71 30.93 brightness 81.18 52.94 32.94 54.12 51.76 78.82 68.24 close 92.50 25.00 35.00 5.00 12.50 77.50 65.00 curved 73.33 38.33 33.33 30.00 35.00 50.00 43.33 damaged 53.49 27.91 23.26 20.93 25.58 16.28 48.84 glare 86.79 20.75 22.64 20.75 35.85 32.08 64.15 high_version 97.30 35.14 59.46 5.41 27.03 40.54 18.92

  9. Detection results Reading rate in percent Commercial SDK A 97.86 Commercial SDK B 1.90 OpenCV Wechat 0.00 Categories Dynamsoft Zxing Zbar BoofCV lots 60.48 82.86 18.10 99.76 monitor 100 5.88 5.88 0.00 0.00 64.71 94.12 nominal 91.03 62.82 51.28 51.28 66.67 89.74 80.77 noncompliant 88.46 3.85 15.38 19.23 50.00 3.85 92.31 pathological 100 78.26 78.26 34.78 65.22 43.48 91.30 perspective 68.57 34.29 51.43 37.14 42.86 82.86 42.86 rotations 99.25 67.67 63.91 42.86 49.62 96.99 82.71 shadows 100 95.00 80.00 65.00 90.00 85.00 85.00 Total average 81.23 44.69 36.69 32.13 39.27 57.83 60.24

  10. Detection results Reading rate in percent Dynamsoft Barcode Reader ranks 1st in most of the categories. It ranks 2nd in the noncompliant and perspective categories and 4th in the lots category.

  11. Detection results Runtime Results Runtime per image (in milliseconds): Engine Result DBR 141.10 Commercial SDK A 314.04 Commercial SDK B 48.31 Zxing 181.68 Zbar 199.26 BoofCV 205.48 OpenCV Wechat 175.06

  12. Detection results Runtime per image by categories (in milliseconds): Commercial SDK A Commercial SDK B OpenCV Wechat Categories Dynamsoft Zxing Zbar BoofCV blurred 298.00 335.16 39.67 206.89 177.58 138.16 125.31 bright_spots 252.84 625.47 47.56 311.59 319.81 221.81 112.25 brightness 192.75 600.14 47.86 320.32 349.79 273.43 134.79 close 169.40 607.27 80.08 295.18 381.65 150.70 676.00 curved 188.76 300.82 43.40 173.44 220.28 103.42 129.54 damaged 200.03 205.49 42.05 84.41 96.76 46.14 138.35 glare 104.66 217.60 45.40 130.32 157.30 70.80 95.38 high_version 106.06 349.21 79.88 294.97 141.70 268.36 566.03

  13. Detection results Runtime per image by categories (in milliseconds): Commercial SDK A Commercial SDK B OpenCV Wechat Categories Dynamsoft Zxing Zbar BoofCV lots 275.43 851.14 110.14 1073.00 1477.57 6149.00 139.57 monitor 176.59 812.53 105.88 233.29 515.47 142.47 422.76 nominal 74.05 203.65 32.12 141.49 111.68 109.49 54.54 noncompliant 58.88 81.63 34.44 47.81 57.06 31.94 43.31 pathological 8.52 4.09 65.87 3.00 1.83 14.61 19.39 perspective 25.74 46.34 14.91 27.66 24.31 26.74 24.60 rotations 72.11 173.84 35.80 106.84 134.66 178.45 86.91 shadows 102.21 243.50 42.29 150.00 163.29 117.50 86.36 total average 144.13 353.62 54.21 225.01 270.67 502.69 178.44

  14. Detection results Although Commercial SDK B has low reading rate on the dataset, it is the fastest. The Dynamsoft Barcode Reader ranks second on average.

  15. Update Runtime Settings to Improve the Results § The benchmark is done without updating the SDKs’ default settings, which can be updated to improve the reading accuracy or speed § Dynamsoft Barcode Reader provides rich parameters that users can customize and optimize for different usage scenarios for the best scanning performance § The runtime settings of DBR can be represented and modified using a JSON template *Learn more about parameters of DBR here

  16. Modify the Template to Decode the Lots Category There are several files in the lots category that DBR cannot detect all the QR codes § Reason - QR codes are relatively small and the image has a large resolution (3024x4032) while in DBR’s template, there is a ScaleDownThreshold parameter which is set to 2300 so that the image will be scaled down for a better decoding speed. § If we set the ScaleDownThreshold to 99999, we can decode all the QR codes § The reading rate for the lots category can increase from 60.48 to 100 using the modified template

  17. Modify the Template to Decode the Blurred Category DBR comes with many image processing algorithms § For blurred QR codes as below, GRAY_SMOOTH image processing method can be used before deblurring to decode it § If the QR codes are blurred too much, not much can be done § DBR can only decode one extra image after using the modified template in the blurred category. However, SDKs like the Dynamsoft Camera Enhancer can help filter the blurred image

  18. Conclusion § The Dynamsoft Barcode Reader has the best QR code reading accuracy on the test set § DBR speed is good and has rich runtime settings to adapt to different scenarios § If conditions allow, it is better to choose DBR as the barcode reading SDK

  19. Links Links related to Dynamsoft Barcode Reader Download Offline SDK Get 30-day FREE Trial License

More Related