slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Overview of TIO-index implementations PowerPoint Presentation
Download Presentation
Overview of TIO-index implementations

Loading in 2 Seconds...

play fullscreen
1 / 21

Overview of TIO-index implementations - PowerPoint PPT Presentation


  • 140 Views
  • Uploaded on

Overview of TIO-index implementations. The DAG, GIDS and Desire TIO/LDAP index servers. Henny Bekker. Overview of TIO-index implementations. Agenda. General overview of LDAP/TIO-indexes What are TIO indexes The generic model Some specific implementations

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Overview of TIO-index implementations' - johana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide2

Overview of TIO-index implementations

The DAG, GIDS and Desire TIO/LDAP index servers

Henny Bekker

slide3

Overview of TIO-index implementations

Agenda

  • General overview of LDAP/TIO-indexes
    • What are TIO indexes
    • The generic model
  • Some specific implementations
    • The generic Desire TIO index server
    • The Ericsson DAG server
    • The GIDS server
  • Open Issues
    • The scope and communication between LDAP/TIO index servers
      • Exchanging TIO’s
    • Local access policy
      • Access restrictions
      • Security requirements
      • Senario’s

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide4

General overview of LDAP/TIO-indexes

Tagged Index Object’s

A TIO consists of:

  • Meta information such as
    • A mime header defining the object
    • An object type identifier that uniquely identifies the subtree and scope
    • One or more URI's that will form the base of the created referrals
    • The security options and credentials such as a PGP or S/MIME key
    • The update type indicating the type of TIO (e.g. full or incremental)
  • The payload
    • The tokenization types headers (e.g. Full, Token, RFC822 etc)
      • Indicating which information is ‘tokenizated’ and which delimiters to use
    • The TAG list
      • Containing multiple consecutive tags which might be grouped using a dash.

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide5

General overview of LDAP/TIO-indexes

Content-Type: application/index.obj.tagged; dsi="1.3.6.1.4.1.5062.1.99.1.114"; base-uri= "weetmuts.surfnet.nl:389/o=SURFnet, c=NL"

Content-Length:6219

version: x-tagged-index-1

updatetype: total

thisupdate: 950688539

BEGIN IO-Schema

sn: FULL

cn: FULL

.

o: TOKEN

END IO-Schema

BEGIN Index-Info

sn: 22/Arends

-6/Bezemer

-4/Bos

-8/Neggers

.

-2-3,5-9,11,14-15,18-19/+31 302 305 305

-12/030-2305327

o: 1/SURFnet

END Index-Info

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide6

General overview of LDAP/TIO-indexes

Tagged Index Object’s (cont.)

What is it used for:

  • Provide pointers to servers which most likely contains the requested information
    • The number of false hits is depending on the choice of attribute tokenization types
    • Performing phrase searches is depending on the tokenization of the fields
  • Features a full or incremental update (which uses potentially less bandwidth)

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide7

General overview of LDAP/TIO-indexes

The generic model

  • A TIO interface
    • For importing, deleting and in some cases exporting TIO’s from the index
    • Implementing authentication control
  • A TIO searchable index
    • For searching the index on referrals to other information services
    • Accessible through the TIO query interface
  • The LDAP query interface
    • Containing a LDAP gateway to the query interface of the TIO index
    • Can act as an LDAPv2 chaining server or as an LDAPv3 referral server

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide8

Some specific implementations

The generic Desire TIO index server

  • Sponsored by the European Community and build by SURFnet & DFN in cooperation with Dante.
  • The server consists of:
    • The TIO index server
      • Using the MySQL database engine for storing and searching the TIO’s
      • Containing a TIO push/pull interface and a database for storing TIO’s.
      • An HTTP frontend for direct access to the TIO index server by the NPS.
    • A Native Protocol Server (NPS) for access using the LDAP protocol
      • For connecting clients using specific communication protocols such as LDAPv2, LDAPv3 or WhoIS++.
      • For connecting directory servers with a specific communication protocols such as LDAPv2, LDAPv3 or WhoIS++.
    • An LdapCrawler for gathering and converting LDIF files to TIO’s
      • Currently no encryption of TIO’s implemented
      • Currently only support for LDAPv2. (no characterset conversion problem)

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide9

Some specific implementations

The generic Desire TIO index server (cont.)

slide10

Some specific implementations

The Desire LDAP/TIO index server (cont.)

slide11

Some specific implementations

The Desire LDAP/TIO index server (cont.)

  • Unfortunately we don’t have yet any performance figures 
    • The package is on the brink of being completed 
    • Presumably the GIDS index server will be faster
      • The generic MySQL engine is slow compared to a dedicated TIO database.
  • Current implementation
    • Available on Linux and (hopefully) on Digital Unix
    • The source code and executable for Linux of
      • The LdapCrawler with an integrated LDIF2TIO converter
      • The TIO index (using MySQL v3.23.6)
      • The LDAP NPS implemented using the Open-LDAP v1.2.10 with an API to the TIO index

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide12

Some specific implementations

The Ericsson DAG server

  • Offspring of the TISDAG project
    • Aimed to provide a solution for an uniform telephone directory containing numbers without a centralized database
  • The server consists of:
    • The DAG (Directory Access Gateway) index server
      • Implemented using the TimesTen “In-Memory” database engine for storing and searching the TIO’s.
    • One or more CAP (Client Access Point) modules
      • For connecting clients using specific communication protocols such as LDAPv2, LDAPv3 or WhoIS++.
    • One or more SAP (Server Access Point) modules
      • For connecting directory servers with a specific communication protocols such as LDAPv2, LDAPv3 or WhoIS++.

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide13

Some specific implementations

The Ericsson DAG server (cont.)

slide14

Some specific implementations

The Ericsson DAG server (cont.)

  • Unfortunately we aren’t allowed to present exact figures 
    • The next version is said to be much faster 
  • Performance figures
    • Response times
      • Use LDAPv3 referral requests to measure the response time of the referral server without doing chaining or following referrals.
      • The mean response time related to the number of parallel search queries. (measured with a large number of queries)
    • Number of queries/second (or minute??)
    • The number of parallel requests
      • Related to the response time
    • Maximum number of entries in the TIO index
      • Bounded by the memory size and the algorithm used to search the index

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide15

Some specific implementations

The GIDS server

  • Offspring of the TISDAG project
    • Second implementation of the TISDAG TIO index server
  • The server consists of:
    • An index server
      • Using a dedicated database engine for storing and searching the TIO’s.
      • Is using a dedicated communication protocol (analogous with LDAP) to communicate with the CAP and SAP modules.
    • One or more CAP (Client Access Point) modules
      • For connecting clients using specific communication protocols such as LDAPv2, LDAPv3 and HTTP
    • One or more SAP (Server Access Point) modules
      • For connecting directory servers with a specific communication protocols such as LDAPv2 and LDAPv3
    • An LdapCrawler for gathering and converting LDIF files to TIO’s
      • With support for LDAPv2 and LDAPv3 and character-set conversion

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide16

Some specific implementations

The GIDS server (cont.)

  • Performance figures
    • Response times
      • Measured with LDAPv3 requests (an LDAPv3 bind, sending the query, receiving the message, doing an unbind operation)
      • With one sequence of LDAPv3 requests, a mean time of approximately 23 msec. per LDAPv3 request.
      • With 10 simultaneously LDAPv3 requests approximately 150 msec per LDAPv3 request.
    • Maximum number of queries/second
      • Approximately 65 LDAPv3 queries/second
      • With LDAPv2 the number will be lower because the server has to do chaining.
    • Maximum number of entries in the TIO index
      • Bounded by the memory size and the algorithm used to search the index
      • Current demo implementation (CH, DE, NL, NO & SE) 120K tokens of 450 different data sets. (which consumes about 35-Mbyte of memory).

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide17

Open issues

The scope and communication between TIO index servers

  • Scope
    • Centralized versus distributed LDAP/TIO engines
  • Location of the TIO/LDAP-index server
    • Located close by (in network terms) the end users to minimize the RTT
    • Located close by the referred LDAP servers to minimize the RTT related to LDAPv2 chaining
  • Exchanging TIO’s
    • Global TIO collection versus distributed collections on country level
      • Distributed to country level
      • Knowledge base or ‘where to find what’??
    • Encrypted transport via HTTP
    • Push or pull ??

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide18

Open issues

Local access policy

  • Security requirements
    • Personal data is subjected to privacy legislation
    • For public data other security requirements might be imposed
    • No unauthorized access to local directory servers
      • Only accessible by local inhabitants and peer countries
    • All applications able to access the index should be known
    • Only a limited number of referrals might be returned
    • No ‘access denied’ messages
      • Don’t show entries which are not accessible
  • Access restrictions
    • Restrict access to the TIO/LDAP-index server
    • Restrict access to the LDAP servers containing the information
      • Chaining versus LDAPv3 referral
      • HTTP access control versus LDAP access control
      • Access via HTTP proxies versus LDAP proxies

What is a TIO index bla bla bla bla bl abl ablbbabbnsjdsa bla bla

slide19

Open issues

Senario’s

  • Create trusted relation between country level TIO servers
    • Only peers will communicate with each other
      • Besides the local LDAP clients
    • A peer will enforce their own local access rules
      • The TIO index server should only be accessible by known clients
    • The LDAP query will be chained to the remote peers
      • The TIO objects of the peer country should deliver referrals which will point to a known access point e.g. an LDAP proxy or the FLDSA
      • An LDAP search requests from a known LDAP client must be chained to the known access point.
    • The number of known access points should be limited
    • The TIO objects cannot be duplicated between the peers
slide20

Open issues

Senario’s (cont.)