slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
STIF [Security Tools Integration Framework] STIF-WARE EVOLUTION PowerPoint Presentation
Download Presentation
STIF [Security Tools Integration Framework] STIF-WARE EVOLUTION

Loading in 2 Seconds...

play fullscreen
1 / 37

STIF [Security Tools Integration Framework] STIF-WARE EVOLUTION - PowerPoint PPT Presentation

  • Uploaded on

STIF [Security Tools Integration Framework] STIF-WARE EVOLUTION Fyodor Yarochkin Meder Kydyraliev HackInTheBox, Kuala Lumpur - 2005. Agenda (best question gets an “Industry Slave” HITB T-shirt). Introduction to STIF-ware concepts

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'STIF [Security Tools Integration Framework] STIF-WARE EVOLUTION' - kreeli

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript


[Security Tools Integration Framework]



Fyodor Yarochkin Meder Kydyraliev

HackInTheBox, Kuala Lumpur - 2005

agenda best question gets an industry slave hitb t shirt

Agenda(best question gets an “Industry Slave” HITB T-shirt)
  • Introduction to STIF-ware concepts
  • First generation of STIF (automation, integration, unification)
  • Demonstration
  • Problems with the first generation of STIF
  • STIF2 – wider coverage of knowledge representation format, functionality decoupling, distributed multi-agent system, open system architecture
  • STIF2 prototype


Security Tools Integration Framework (STIF) is aimed to provide a unified environment and data exchange platform for automated security assessments in heterogeneous environments.

In simple words it is a platform for “hacking” automation, where STIF emulates the “brain” of a security analyst to perform repetitive tasks.


Why automation?

  • machine-based knowledge processing
  • automate routine tasks, spend more time on tasks that require brain power
  • create intrusion scenarios, and let machine probe them (nIDS testing)
  • ‘human’ error mitigation
  • reduce human labor involvement in modern corporate pen-testing sweatshop

Why integration?

  • Various security tools, written in different languages, are available, but no unified format for data exchange and representation;
  • No machine data analysis, aggregation and correlation possibilities;
  • Handling large-scale assessments w/ disintegrated tools is a nightmare;
  • No possibilities to automate distributed attacks

Typical scenario for security analyst


Want to see what happened to Joe the analyst after one month?


Poor Joe…

Look what repetitive and boring “hacking” has done to him…


Why not let machine do the boring part???


Of course, you can ...

  • script it: `ls –al ~/code/scripts/`
  • (ab)use security scanners (nessus);
  • (ab)use exploit toolkits (e.g metasploit);
  • hire a full room of pen-testing monkeys, that will do the boring part (sweatshop production);

Scanners vs. STIF

  • Problems with scanners:
  • hardcoded sequence of execution;
  • vendor-specific integration (e.g. NASL, plug-in APIs), requires rewrite or code hacking;
  • vendor-specific data representation/storage (hard to integrate into existing solutions, e.g. custom DBs);

STIF solution

STIF is designed to solve the problems outlined earlier, by introducing the common format for data representation and by providing a platform for data exchange among tools.


First generation STIF provides:

  • Highly customizable rule-based inference engine, which enables analyst to script out ANY scenario based on the data that was returned by tools;
  • Unified data exchange and representation format;
  • Generic database publishing module (save data from tools in DB w/ any scheme);
  • IRC BOT interface: data publisher and importer

STIF Features (continued)

  • Distributed architecture
  • ready to use DB schema
  • STIF is written in Java
    • the reason for that decision is simple: quicker development cycle, cross-platform compatibility;

Data representation unification

STIF encapsulates data in a set of XML messages (STIF-Message)

Input data, provided in XML format, converted by Exec module into the form, which could be understood by the tool

The results of tools execution are converted to STIF and are fed back into the Inference Engine.



Sample STIF-Message:

<STIF-Message created="2004-09-02T15:03:01+6">

<Port number="80" state="open" protocol="tcp">

<Address type="ipv4-addr"></Address>




Apache/1.3.27 (Unix) PHP/4.3.1






Inference engine

  • responsible for data interflow between various tools;
  • makes decisions on which tools to be executed, when new data appears
  • provides data aggregation and correlation facilities(including regular expressions based matching to the
  • knowledge base facts);
  • maintains execution flow using rule-based scenarios;

Data Publishing facility

Publishing in STIF environment means providing the Publisher with newly arrived facts (STIF-Messages from tools).

STIF is able to execute several data/fact publishing modules simultaneously (e.g. database publishing, IRC publishing).


SQL Publisher

STIF comes with SQL publishing module, which can publish/store data received from tools in a form of a STIF-Message, in databases of arbitrary scheme.

<message type="Target">


INSERT INTO ip_address VALUES(NULL,'%h');

</query> </message>

<message type="Port">


SELECT id FROM ip_address WHEREip_address='%h';



INSERT INTO port VALUES(NULL, $1, '%n', '%P', '%S', '%p', '%a'); </query>



IRC Importer/Publisher

STIF supports command input over IRC and can publish new facts to an IRC channel or using private messages.

Other software tools can act as STIF “nodes” embedding the IRC importer/publisher functionality


Your favorite tools integration to support STIF?

  • STIF provides several means to import data into STIF inference engine:
  • Generic2STIFConverter, extracts data from output using regular expressions to form STIF-Message;
  • Tool-specific wrappers

Integration using STIF Generic2STIF Converter

Define rules in parser.xml:

<?xml version="1.0"?>


<Tool name="nmap-syn-version">

<Group name="target address">


<Regex name="address" required="true">

.*ports on .*\(([\d\.]+)\):.+


<Group name="port" generate="port">




<Regex name="portNumber" required="true">




  • <Regex name="portProtocol" required="true">
  • ^\d+/(tcp|udp).+
  • </Regex>
  • <Regex name="portState" required="true">
  • ^\d+/(?:tcp|udp)\s+(open|closed|filtered).+
  • </Regex>
  • <Regex name="portService" required="true">
  • ^\d+/(?:tcp|udp)\s+(?:open|closed|filtered)\s+([\w-]+).*
  • </Regex>
  • <Regex name="portApplication" required="false">
  • ^\d+/(?:tcp|udp)\s+(?:open|closed|filtered)\s+[\w-]+\s+(.+)
  • </Regex>
  • </Group>
  • </Group>
  • </Tool>
  • </Config>


How can you help?

  • You can do several things to contribute
  • to our efforts:
  • Try it!!!
  • Ask your favorite tool’s author to become STIF-compliant;
  • Write regular expressions to parse output for Generic2STIFConverter;
  • Patch you favorite tools to be STIF-compliant;
  • or.. wait until STIF2 is out

First generation STIF Demonstation

problems with current stif implementation

Problems with current STIF implementation
  • Complexity: massive coupled piece of code
  • Centralized system: limited support for task distribution
  • Non-dynamic (fixed at startup) inference engine rules
  • Knowledge interchange format needs to be extended
stif2 concepts

STIF2 Concepts

Functionality decoupling

stif2 concepts1

STIF2 Concepts
  • Platform independent
  • Composed of independent agents
  • Agents communicate with each other using messaging protocol
  • Agent capability service exists to provide agent capability lookup and matching facility
stif2 multi agent architecture

STIF2 Multi-Agent Architecture
  • Multi-agent architecture
    • Tool wrapper Agents
      • Scanning, connection forwarding, attack launching
    • Logic Execution Agents
    • User Interface Agents
    • And more
message exchange framework

Message Exchange Framework
  • Provides facilities for agent communication
  • Provides facilities for communication channel selection (covert channels, tunneling, stenography)
goal driven execution

Goal-Driven execution
  • Goal-driven execution flow
    • Each agent describes its functionality with a set of capabilities. Each capability can be executed on certain type of data object (network, host, user, URL)
    • Each agent is given task to execute the capability, which becomes agent goal. Agent may have different plans to execute the same capability. Plans are scored based on execution success rate
goal driven execution1

Goal Driven execution
  • Each also plan may be assigned with qualifiers:
    • Stealth-ness
    • Latency

Which can be matched to current ‘environment’ settings

event driven execution

Event-driven execution
  • Event-driven execution flow
    • Each agent may subscribe to ‘interests’, expressing its interest to certain types of data objects, which agent is interested in (network, host, open port, URL, a valid user)
    • When an agent discovers a new data. The “interests” list is queried for the list of interested agents. The agent is responsible to forward the data to interested partners.
agent data cache beliefs

Agent Data Cache (beliefs)
  • Agent caches data locally (local data Cache, beliefs)
  • Agent may query other agents or KB for missing data
current implementation prototype

Current Implementation prototype
  • Based on Java/JADE framework
  • The communication protocol: in progress
  • The knowledge interchange format: reviewing current standards (KIF, DAML)
  • Once the communication framework is finalized, JADE messaging framework to be replaced with home-brewed implementations (ports for different languages)

Questions (remember we give out T-shirt for best question)?

Suggestions ?