O’Reilly Open Source Conference 2000
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

O’Reilly Open Source Conference 2000 PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on
  • Presentation posted in: General

O’Reilly Open Source Conference 2000. Stephane Barde. 7/19/2000. I - Value Added Information Channels -> Problem related to information flow management -> Creating the value added channel II - Perl: a key enabler of process re-engineering -> Resources use maximized with Perl

Download Presentation

O’Reilly Open Source Conference 2000

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


O reilly open source conference 2000

O’Reilly Open Source Conference 2000

Stephane Barde

7/19/2000


O reilly open source conference 2000

I - Value Added Information Channels

-> Problem related to information flow management

-> Creating the value added channel

II - Perl: a key enabler of process re-engineering

-> Resources use maximized with Perl

-> Key Advantages

III - Flat Files as a Central Data Repository:

-> Dimensioning.

-> Retrieving Data.


O reilly open source conference 2000

I - Value Added Information Channels

What this is all about:

Company standard processes generating information, that are not part of the core business, but generate important loss of productivity.

  • Examples:

  • Travel Request and Booking Information

  • Bug Tracking System

  • Application Access Request

  • New Employee Check Lists: how to get a new employee productive in a minimum amount of time.

Improve Efficiency Globally by Giving Value to the Information


O reilly open source conference 2000

Travel Request

Departure/Return Date, Hotel, Car.

Approval up to a certain amount. Overnight stay, in lieu off approval...

Detailed Travel Info: Schedule, addresses...

PO #, Invoice #, Folder #, booking #…

A Typical Information Stack: the Approval Chain

User

User Need

Manager

Approval

Detailed Information

Tech/Functional Support

Buyer, Finance, Receiver...

References

Different People Providing Different Pieces of Information


O reilly open source conference 2000

A Small Company, one Department

Department

Department

Travel Requester

Travel Requester

Travel Requester

Travel Approver

Travel Approver

Travel Approver

Travel Booker

Travel Booker

Travel Booker

A Typical Information Stack:

A Small Company: Single Vertical Dimension

Manual process -> time consuming, either the booker or the financial analyst spend time to consolidate data.

Multi-dimensional model:

Processes are different within the organization. Consolidation end up being a nightmare as Data don’t mean the same and are not handled the same way across the organization.



*

MS Excel

!?

Financial Consolidation and Analysis

Loss of Productivity on Administrative Tasks


O reilly open source conference 2000

Do we (we = all users involved in the process) all talk the same language through the entire organization? Do we all use the same information pieces to complete a given action?

Am I (I = anyone in the organization) able to report on given metrics globally (what is spent and how it is spend, what is the mean time to resolve a problem)? Can I easily aggregate the information I work on?

Am I able to quickly retrieve the information I need to analyze, compare, and make the right decision at the right time?

Issues with information Flow Management:

How Efficient are You at Managing and Using Information?


O reilly open source conference 2000

Consistency: No Systematic Processing.

Consolidation: Someone Someday will have to put everything together to report on specific metrics.

Accessibility: No searching capabilities, data not stored on a centralized repository.

Capitalization: Any data repository is a potential wealth: Developing Analysis Capacity will enable to Gain Productivity, Sharpen your Strategy, Refine your Goals.

Issues with Information Flow Management:

Take advantage of the Information You Generate


O reilly open source conference 2000

Travel Request Process Map

Travel Request

Raw Information

*

Approval

*

Travel DB

A/P

No

*

Approved

*

*

Travel Info

Travel Booker

Travel Info

Travel Agency

Travel Info

XML Invoice

XML Invoice


O reilly open source conference 2000

Bug Tracking Process Map

Filtering

End User

Tech. Support

Bug

*

*

Resolution

Help for Resolution

Bug DB

User

Test

Not Solved

OK

Incident closed

*


O reilly open source conference 2000

Information Flow Management

Perl

“Process Sticky”

Development

Web Based

Interface

to Data

Efficient

Data Access

II - Perl as a Key Enabler for Process Re-Engineering

Consistency

Consolidation

Accessibility &

Capitalization

Perl sets a favorable context to Manage Data Flow


O reilly open source conference 2000

Management

IT

?

II - Perl as a Key Enabler for Process Re-Engineering

Need for a Web Based Solution

Time / Resource / $

What Management says to the IT guy:

“We need to (re-)engineer this process. You’ve got no resources, no money and no time. Just do it. When do you thing it will be ready?”

You’re the IT guy!

What Do You Say?

Will Do… with Perl


O reilly open source conference 2000

  • IT resources primarily focused on core business applications development and support.

  • No time to waste in experimenting: need for efficient, documented, pre-existing solutions.

Quick developer learning curve makes project staffing easier.

Productivity

Programmer Learning Curve

  • Perl reminds of many other programming language.

  • Quick Learning Curve.

  • Lot of documented free resources available (CPAN).

  • Perl is free and highly portable.

  • Perl is low on Hardware Resources

Perl

Other Programming language

Time

Perl Key Advantages

Short Implementation Timeline with Few Resources


O reilly open source conference 2000

  • Perl provides connectors to most of the database types.

  • Perl has an efficient set of tools and libraries to develop solutions close to the users process: file processing, triggered email alerts, XML data parsing, synchronization with BackOffice…

  • Flat files are a quick way to deploy centralized web based processes.

  • They offer an efficient way to store information at no cost.

  • Flat files present the advantage of being compatible with most software. For example MS Excel allows further analysis of raw data provided by flat file databases.

Dealing with Data

Perl makes Flat File a Good Solution for Web Based Processes


O reilly open source conference 2000

  • Flat file processing works fine up to 50k lines (and more) depending on the hardware and server workload.

  • Well suited for low transactional volume: up to 200 transactions/day.

  • Travel Request Example:

  • 10.000 employees, 500 frequent flyers (2 trips/month).

  • 2000 employees travel twice a year.

  • => 14.000 rows per year (1k/line => 14 Megs)

  • => flat files yearly split

  • To increase search efficiency:

  • archiving by splitting databases (temporal split / by data value).

  • Have index files containing keywords and reference to static generated file.

III - Flat Files as a Central Data Repository:

Keep File’s Size Small


O reilly open source conference 2000

Retrieving information

XML

Looking for ‘ABC’

Search Results

One Data Entry

XML output dynamically generated

data.csv

………

………

………

……...

XSL

XML

template

2 full scans to retrieve the information needed


O reilly open source conference 2000

Reducing Response Times: File Split

travel_2000.csv

travel.csv

File split: as the database size decreases, the search is more efficient.

………

………

………

……...

………

………

………

Example: yearly split of the master data file

Natural Way to Decrease the Size of the File Parsed


O reilly open source conference 2000

Reducing Response Times: Static Display

data.csv

XML

1 database entry is associated with 1 static XML file.

………

………

………

……...

Static XML

with Ref #

Looking for ‘ABC’

Search Results

Ref #

data.csv

………

………

………

……...

XML

XSL

Processing Allocated to Data Search


O reilly open source conference 2000

Travel.csv

ref;user_first_name;user_last_name;request_date;from;to;date;preferred_airline;preferred_time;airline;time;………..

543;John;Doe;3/5/2000;London;Paris;Air France;8:00 PM;Air France;8:20 PM;……….

542;Sam;Tondac;3/6/2000;Los Angeles;Paris;KLM;6:00 PM;Delta Airlines;5:54 PM;……….

541;Jennifer;Alister;3/6/2000;Houston;Chicago;American Airlines;5:30 PM;American Airlines;3:33 PM;………….

….

….

Travel_543.xml

<travel_request>

<user>

<user_first_name>John <user_first_name>

<user_first_name>Doe <user_first_name>

</user>

<requested_date>3/5/2000

<flight>

<from>London</from>

<to>Paris</to>

<preferred_airline>Air France</preferred_airline>

<preferred_time>8:00 PM</preferred_time>

….

….


O reilly open source conference 2000

Reducing Response Times: Reference Key & Static Display

Static XML

with Ref #

Looking for ‘ABC’

Search Results

Ref #

data_key.csv

data.csv

XML

XSL

………

………

………

……...

………

………

………

……...

+ : Size of the file parsed can decrease by a factor 5 to 10. Can be correlated with temporal split.

- : Edit efficiency decreases.

Volume of Data Parsed Significantly Decreases


O reilly open source conference 2000

Travel.csv

ref;user_first_name;user_last_name;request_date;from;to;date;preferred_airline;preferred_time;airline;time;………..

543;John;Doe;3/5/2000;London;Paris;Air France;8:00 PM;Air France;8:20 PM;……….

542;Sam;Tondac;3/6/2000;Los Angeles;Paris;KLM;6:00 PM;Delta Airlines;5:54 PM;……….

541;Jennifer;Alister;3/6/2000;Houston;Chicago;American Airlines;5:30 PM;American Airlines;3:33 PM;………….

….

….

Travel_key.csv

Travel_543.xml

<travel_request>

<user>

<user_first_name>John <user_first_name>

<user_first_name>Doe <user_first_name>

</user>

<requested_date>3/5/2000

<flight>

<from>London</from>

<to>Paris</to>

<preferred_airline>Air France</preferred_airline>

<preferred_time>8:00 PM</preferred_time>

….

….

ref;user_first_name;user_last_name; from;to;airline

543;John;Doe;London;Paris;Air France

542;Sam;Tondac;Los Angeles;Paris;Delta Airlines

541;Jennifer;Alister;Houston;Chicago;American Airlines

….

….


O reilly open source conference 2000

INSERT

UPDATE

Search Efficiency

  • Lot of Data

  • High volume of searches

  • Few Updates

  • > 100 inserts/day

  • Small Volume of Data

  • Lot of Updates

  • < 40 inserts/day

Reference Keys

& Static Display

+ File Split

Ex: Bug Tracking System

Ex: Phone Directory

Reference Keys

& Static Display

Static Display

+ File Split

Ex: Travel Request

Static Display

File Split


O reilly open source conference 2000

Perl makes possible the restructuration of the Information Channels with a minimal investment.

For more information:

[email protected]

www.ezperl.com

www.arthys.com


  • Login