Checkvir Realtime Anti-Malware Testing and Certification

1. Checkvir Realtime Anti-Malware Testing and Certification Dr. Ferenc Leitold, Veszprog Ltd. fleitold@veszprog.hu www.checkvir.com

2. Purpose of Checkvir testing Testing methodology Technical background Testing procedures Current state Difficulties Questions Contents

3. Purpose of Checkvir testing • Problems: • Big number of updates • Cloud technology • Solutions are continually changing • Testing all versions are impossible Number of updates / day source: AV-Test.org

4. Purpose of Checkvir testing • Testing all versions are impossible • Executes tests as frequently as possible • Automatic methods have to be developed • Big number of computers have to be used

5. Purpose of Checkvir testing • The main purposes: • Provide reliable, correct and exact information mainly about: • effectiveness • performance • in a balanced way • (AMTSO’s principle) • Provide naming cross-reference information effectiveness performance

6. Testing methodology update test Unpack previous image Unpack last image Initialize testing AV update Execute test(s) no New version? Save results and reports yes Pack and save the new image Analyze results Publish results

7. Testing methodologyTechnical background firewall “malware proxy” server webserver controller firewall & router archiver clients

8. Testing methodologyTesting procedures • Malware knowledge (detection, disinfection) • against known, unknown malware and clean files • on-demand, on-access and proactive executions • “Container” checking capabilities • archives, email clients’ data files, … • Speed • on-demand, on-access • boot time • Functionality • Stability • … knowledge speed

9. Testing methodologyTesting procedures Why the speed is so important?

10. Testing methodologyTesting procedures

11. Testing methodologyTesting procedures Testing bootup time What is more important? BOOTUP TIME or SECURE BOOTING DEMO

12. Testing methodologyTesting procedures

13. Testing methodologyTesting procedures Bootup protection test Avast AVG Avira Bitdefender Eset e-Trust F-Prot F-Secure Fortinet Ikarus Kaspersky Microsoft Rising Sophos Symantec Trend Micro VirusBuster

14. Testing methodologyTesting procedures Bootup protection test Avast AVG Avira Bitdefender Eset e-Trust F-Prot F-Secure Fortinet Ikarus Kaspersky Microsoft Rising Sophos Symantec Trend Micro VirusBuster

15. Testing methodologyTesting procedures Bootup protection test Avast AVG Avira Bitdefender Eset e-Trust F-Prot F-Secure Fortinet Ikarus Kaspersky Microsoft Rising Sophos Symantec Trend Micro VirusBuster

16. Testing methodologyTesting procedures Bootup protection test Avast AVG Avira Bitdefender Eset e-Trust F-Prot F-Secure Fortinet Ikarus Kaspersky Microsoft Rising Sophos Symantec Trend Micro VirusBuster

17. Testing methodologyProactive tests vs. AM cloud technology Problems: • AM products use cloud technology • > traffic should be allowed • Malware use cloud technology • > traffic should be allowed • > How can we protect the world? • > How can we provide exactly the same environment for solutions?

18. Testing methodologyProactive tests vs. AM cloud technology firewall “malware proxy” server webserver controller firewall & router archiver clients

19. Testing methodologySettings • By default, DEFAULT settings are used • Minimal functionality is required: • Execute tests without user interaction • Automatically clean the infected file (if not possible -> delete) • Report file generation

20. Current state What is working now? • The frame system • The website • Automatic procedures of some products • Preliminary selection and validation of the samples

21. Current state

22. Current state

23. Current state

24. Difficulties • Viewpoint of the average user Automatic methods • Testing environment • Funcionality problems • Truncate report file • Stability problems

25. Questions