File Upload Validator

1. File Upload Validator Zach Moshe RotemNaar

2. Agenda • File upload vulnerabilities overview • FUV – detailed overview • Live demonstration • In the future…

3. Background • Many applications take advantage of the band width available today and allow users to upload file, either for storage or usage within the flow of the software. This allows the software to be more appealing and interactive with the user • The uploaded file is a “jack in the box”. It may convey all sorts of trouble within, from viruses to extremely large sizes

4. Avoid vulnerabilities Safe file upload principals

5. Safe file upload principals • Check file type • Avoid dangerous extensions • Validate MIME-type Module

6. Safe file upload principals • Use random filename • Avoid XSS attacks • Avoid file inclusion attacks Utility Module

7. Safe file upload principals • Keep upload directory security Module

8. Safe file upload principals • Scan file with AntiVirus Module

9. Safe file upload principals • Limit file size • Avoid DoS attack Utility Module

10. FUV package Design and Details

11. Spec • Java package, which exposes an API that allows file validation through a single validate(file) method • The application is configured by an XML file that the caller supplies. Only relevant modules will be enabled • Utilities for application developer • Using Java 1.6

12. Design FUV package Validation modules Validation modules Utils After the file is uploaded Before/While uploading the file

13. Design – validation modules FileValidator << interface>> boolean Validate(File) Module << interface>> boolean Validate(File) * FileValidatorImpl File Type Module File Name Module UNIX File Permissions Module Anti virus Module

14. Design - FileValidator • The primary interface of the system • public boolean validate(File file) • Holds set of modules • Returns true if all configured modules approved the file according to their configuration • If at least one of the modules rejected the file, the method returns false

15. Design - FileValidator • Open archive/compressed files and check the inner files using the modules • In case one of the inner files is archive/compressed file too, the same operation is done recursively • The maximum file depth allowed is configured in the XML configuration file • Opens archive/compressed files using Apache-Commons-Compress package • Supported formats: ZIP, TAR, GZIP, BZIP2

16. Design - Module • The main operation: • public boolean validate(File file) • All modules have: • “scanInnerFiles” attribute (“true” by default) • unique configuration • In case “scanInnerFiles” is “true” and the validated file is archive/compressed file, the module will scan the inner files too

17. Design – File Type Module • Validates file types according to a predefined set of accepted MIME types (white-list validation) • Uses Apache-Tikapackage for content analysis of the file • Configuration: • Allowed types • Force extension check

18. Design - UNIX Permissions Module • Can be enabled only in UNIX environment • Validates that the file on the server has the appropriate permissions • The module is configured by 3 “maximal” allowed permissions for the user, group and all (similar to UNIX file permissions) • Using ls UNIX command

19. Design – Filename Module • Validates filename strings • Configuration: • Filename length • Allowed character strips – from the strips configured in the system (white-list validation)

20. Design - AntiVirus Module • Uses an external program as an AntiVirus • Approves/Rejects the file according to its return code • Configuration: • AntiVirus path • Success return code • We’re using Clam-AV

21. Design - Demonstration File File Module FileValidator Module Module False True False

22. Design - Demonstration File File Module FileValidator Module Module False True True False If archive/comressed: Foreach inner file: send to validation

23. Design FUV package Validation modules Utils Utils After the file is uploaded Before/While uploading the file

24. Design - utils • FileNameGenerator • String generateNewRandomFilename() • String censorFilename(String filename) • SizeBoundedInputStream extends InputStream • Read() • hasReachedLimit()

25. File Name Generator • Allow the user generate safe filenames • Contains 2 methods: • censorFilename(String fileName)Censors given filename: limits the filename length and removes not-allowed charactersConfiguration: • filename length • Allowed characters strips • generateNewRandomFilename() Generates random filename according to the configured patternConfiguration: filename pattern

26. Size Bounded Input Stream • Creates safe way to upload a file without a problem with its size • Extends InputStream and warps the original InputStream • In case the number reached the maximum allowed, it returns -1 (EOF) and set the limitReached flag to “true” • Configuration: maximum size allowed

27. XML Configuration file • Configure engine, modules and utilities parameters <file-validator-config> <application-name>Application Name</application-name> <archive-recursion-depth>7</archive-recursion-depth> <modules> … </modules> <file-name-generator> … </file-name-generator> <max-file-size>1024</max-file-size> <char-strips> … </char-strips> <types-collections> … </types-collections> </file-validator-config>

28. XML Configuration file <modules> <!-- File name module --> <file-name-module> <max-file-name-length>50</max-file-name-length> <allowedCharStrips>D C O</allowedCharStrips> </file-name-module> <!-- Anti Virus module --> <anti-virus-module scanInnerFiles="false"> <anti-virus-path>bin/av_wrapper.sh</anti-virus-path> <success-rc>0</success-rc> </anti-virus-module> <!-- File type module --> <file-type-module> <allowed-types>word text application/x-gzip </allowed-types> <force-ext-check/> </file-type-module> <!-- File permissions module--> <unix-file-permissions-module scanInnerFiles="false"> <user-max-permissions>rwx</user-max-permissions> <group-max-permissions>r-x</group-max-permissions> <all-max-permissions>r-x</all-max-permissions> </unix-file-permissions-module> </modules>

29. XML Configuration file <types-collections> <types-collection name="word"> <type allowed-exts="doc">application/x-tika-msoffice</type> <type allowed-exts="doc">application/msword</type> <type allowed-exts="dotx,docx">application/x-tika-ooxml</type> <type allowed-exts="docx">application/vnd.openxmlformats-officedocument.wordprocessingml.document</type> <type allowed-exts="dotx">application/vnd.openxmlformats-officedocument.wordprocessingml.template</type> </types-collection> <types-collection name="text"> <type allowed-exts="rtf">application/rtf</type> <type allowed-exts="txt">text/plain</type> </types-collection> </types-collections>

30. Logging 2011-03-04 18:51:01,859 INFO [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:63] Validating file : C:\tmp_rotem\tmp\out.zip 2011-03-04 18:51:01,859 DEBUG [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:68] Validating module com.amdocs.filevalidator.modules.FileNameModule 2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileNameModule [FileNameModule.java:61] File name length (excluding extension) is 3. Maximum length allowed: 50 2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileNameModule [FileNameModule.java:81] Allowed chars: 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz_-)( 2011-03-04 18:51:01,875 DEBUG [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:68] Validating module com.amdocs.filevalidator.modules.FileTypeModule 2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:61] FileTypeModule was called for out.zip 2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:65] AllowedTypes are {application/x-tika-msoffice=[doc], image/jpeg=[jpg, jpeg], text/plain=null, application/x-bzip2=null, application/x-gtar=null, application/vnd.openxmlformats-officedocument.wordprocessingml.document=[docx], application/msword=[doc], application/x-gzip=null, application/x-tika-ooxml=[docx], application/zip=null} 2011-03-04 18:51:02,296 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:90] content type is application/zip 2011-03-04 18:51:02,296 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:93] forcing ext check 2011-03-04 18:51:02,343 INFO [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:110] Found ZIP file 2011-03-04 18:51:02,343 INFO [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:323] Entry: cfvxcbcf.txt …

31. Quality • XML Configuration – using JAXB • Logging - using SLF4J and LogBack • Unit Testing • Code Examples • Building the project – using Maven • Version Control – using SVN • JAR, sources and documents can be found on: http://code.google.com/p/fuv/

32. Demonstration Validate files using FUV package

33. In the Future How to improve the project

34. In the Future • Add support in client side (JavaScript/PHP packages) • Add module for special treatment to images (malicious code inside image) • Create secure upload server using the FUV package • DoS Attack – limit the size and number of files one user can upload in a given period (track the user using cookies or IP)

35. Thank You!