adventures in large scale http header abuse n.
Skip this Video
Loading SlideShow in 5 Seconds..
Adventures in Large Scale HTTP Header Abuse PowerPoint Presentation
Download Presentation
Adventures in Large Scale HTTP Header Abuse

Loading in 2 Seconds...

play fullscreen
1 / 37

Adventures in Large Scale HTTP Header Abuse - PowerPoint PPT Presentation

  • Updated on

Adventures in Large Scale HTTP Header Abuse. Zachary Wolff. About Me. SIEM Deployments, Research Engineer for LogRhythm (Labs) | 2+ years Threat Research Analyst for Webroot Software | 2+ years. Lets Talk About HTTP Headers. Browser. Web Server. HTTP Headers Basics.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Adventures in Large Scale HTTP Header Abuse

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
about me
About Me
  • SIEM Deployments, Research Engineer for LogRhythm(Labs) | 2+ years
  • Threat Research Analyst for Webroot Software | 2+ years

HTTP Headers Basics

  • Standard Fields for Request & Response defined in RFC- 2616 (RTFRFC)
  • GET / HTTP/1.0 is a legitimate request but may not return expected results
  • RFC sets no limits on size of header, field name, value or number of headers
  • Most webservers now impose their own limits :
  • IIS
  • v4 – 2MB
  • v5 – 128K-16KB*
  • V6 – 16 KB*
  • V7 – 16 KB*


v2.3 – 8KB*

*Per Header Field

the premise
The Premise

I want to break some logging applications!

round 1 begin
Round 1: Begin

Original Premise: GET Request returns 302, 200, (valid response) then send a second GET with a malicious User Agent string* to see if we can get 500 response

  • Crawler to collect URL’s
  • Python script to send attack/test UA String
  • Store results in SQLite3 DB
  • Profit!
round 1 results
Round 1: Results

Data set: 400K URL’s

  • Lots of 500’s!
  • Lots of smaller, low traffic site, some bigger high traffic sites
  • Various different errors….
round 1 results1
Round 1: Results

Custom 500’s…

l e yawn
le yawn..

Non verbose IIS Errors…

not as boring generic apache 500
Not as boring, generic apache 500

and the sites….?

round 1 conclusion
Round 1: Conclusion

What did we find?

  • Some SQL injectable 500’s
  • Possible application level DOS
  • Lots of websites are not expecting malicious Header requests…
  • Further exploration is warranted
the question
the Question

How extensive is the problem of improper HTTP header handling?

round 2 begin
Round 2: Begin
  • Need a more effective way to identify vulnerabilities
  • Lets attack/audit more than just User-Agent Header
  • Expand beyond backtick, additional attack strings
  • Larger sample set, 1.6 Million URL’s
  • Must be able to store and access very large set of result data efficiently (Shodan is amazing)
round 2 vulnerability identification
Round 2: Vulnerability Identification

500’s are ok, but much to broad

What is a good indication of a possible SQLi vulnerability?

Run regular Expression against response to match on, “you have an error in your sql syntax”

round 2 vulnerability identification1
Round 2: Vulnerability Identification

Improved error detection, basic SQLi & beyond

*Thanks to @j0emccray for contributing to regEx list

beyond regex based error detection
Beyond RegEx based Error Detection

Byte Anomaly Detection Added (--bad)

Compare content-length of response data from original/clean GET to data from malicious GET.

*Set margin of alert to 150 bytes above and 150 bytes below clean request, log results (including HTML response data) to file

round 2 additional header fields
Round 2: Additional Header Fields
  • Let’s test: Host, From*, X-Forwarded-For, Referer, User-Agent, Non existent Header
  • Smart Mode (-s) : Will look at all Header fields returned by the server and test those (minus whitelist of rarely dynamic Headers)
  • Cookies!
cookie support
Cookie Support

Cookie Support added. Server Sends us this:

PyLobster Responds with this:

And the server says?

round 3 updates
Round 3: Updates

Updated Testing Values: “,;,%00, %00’

round 2 design
Round 2: Design

“I Improved the crawler to harvest 500K+ URL’s a day. You should put my picture in your whitepaper”

Output additions (beyond SQLite):

  • Elasticsearch Indexing support added (fast, efficient, JSON to webinterface)
  • Flat File logging

Mark Vankempen, LogRhythm Labs

more improvments
More Improvments

Added Footprint mode (-g)

  • Generate random(ish) Hash or value
  • Save to key.txt file in same directory as
  • Activate Footprint mode: ./ –g

pyLobsterwill now send your unique string/hash as a request like so:

Then, Wait for it… Days, Weeks, Months

Google/Bing/duckduckgo your hash/string to discover unprotected Log directories ;)

pylobsters maiden voyage
pyLobsters maiden voyage

Ready Begin!

pyLobsteris currently a single threaded tool so I divided my 1.6 Million URL’s into 78 unique lists and spawned 78 instances


nohup python -f a --bad -s -l -g &

nohup python -f b --bad -s -l -g &

nohup python -f c --bad -s -l -g &

nohup python -f d --bad -s -l -g &

And so on……

pylobster s maiden voyage results
PyLobster’s Maiden Voyage Results
  • Sending a null byte in your HTTP Headers will catch a fair bit of IDS attention ;)
  • Grepresponse HTML on regEx error match directory to find patterns & specific components/module/application/CMS vulnerabilities. (highest value finding: one vulnerable component can lead to many others, shared DB’s as well)
  • Various vulnerable components identified
findings breakdown by regex
Findings: Breakdown by RegEx #

Out of 1.6 Million Unique URL’s, 14,500 Error RegEx’sMatched!

  • *

*0,1 & 2 are MySQL errors, 18 & 19 are PHP

findings error breakdown by test string
Findings: Error Breakdown by Test String

Of the 14,500 Error RegEx’s Matched

findings error 0 breakdown by header field
Findings: error #0, breakdown by header field

Error #0: “you have an error in you SQL syntax"

findings footprint mode
Findings: Footprint Mode

Footprint Mode

12/13/2012 

02/25/2013 

foot print mode
Foot Print Mode

3/27/2013 

findings bad
Findings: (--bad)

Byte Anomaly Detection Results

  • Work to be done….
  • grep over dir for [wordpress|joomla|error|pass.*=|cms|.*?|]
  • Sort response files by size for like errors
  • Sort by status code response & file size
defending against http header attacks
Defending Against HTTP Header Attacks
  • Raise developer awareness that any dynamically handled Header values need to be seen as user input and processed accordingly
  • Audit your sites HTTP Header Processing (pyLobster on github, SQLmap now supports custom Header testing too. bingo!)
  • Proactively review/monitor your web logs


Creates this Log trail:

the end
The End

Thank you!