1 / 22

O’Reilly Open Source Conference 2000

O’Reilly Open Source Conference 2000. Stephane Barde. 7/19/2000. I - Value Added Information Channels -> Problem related to information flow management -> Creating the value added channel II - Perl: a key enabler of process re-engineering -> Resources use maximized with Perl

daniel-gentry
Download Presentation

O’Reilly Open Source Conference 2000

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. O’Reilly Open Source Conference 2000 Stephane Barde 7/19/2000

  2. I - Value Added Information Channels -> Problem related to information flow management -> Creating the value added channel II - Perl: a key enabler of process re-engineering -> Resources use maximized with Perl -> Key Advantages III - Flat Files as a Central Data Repository: -> Dimensioning. -> Retrieving Data.

  3. I - Value Added Information Channels What this is all about: Company standard processes generating information, that are not part of the core business, but generate important loss of productivity. • Examples: • Travel Request and Booking Information • Bug Tracking System • Application Access Request • New Employee Check Lists: how to get a new employee productive in a minimum amount of time. Improve Efficiency Globally by Giving Value to the Information

  4. Travel Request Departure/Return Date, Hotel, Car. Approval up to a certain amount. Overnight stay, in lieu off approval... Detailed Travel Info: Schedule, addresses... PO #, Invoice #, Folder #, booking #… A Typical Information Stack: the Approval Chain User User Need Manager Approval Detailed Information Tech/Functional Support Buyer, Finance, Receiver... References Different People Providing Different Pieces of Information

  5. A Small Company, one Department Department Department Travel Requester Travel Requester Travel Requester Travel Approver Travel Approver Travel Approver Travel Booker Travel Booker Travel Booker A Typical Information Stack: A Small Company: Single Vertical Dimension Manual process -> time consuming, either the booker or the financial analyst spend time to consolidate data. Multi-dimensional model: Processes are different within the organization. Consolidation end up being a nightmare as Data don’t mean the same and are not handled the same way across the organization.  * MS Excel !? Financial Consolidation and Analysis Loss of Productivity on Administrative Tasks

  6. Do we (we = all users involved in the process) all talk the same language through the entire organization? Do we all use the same information pieces to complete a given action? Am I (I = anyone in the organization) able to report on given metrics globally (what is spent and how it is spend, what is the mean time to resolve a problem)? Can I easily aggregate the information I work on? Am I able to quickly retrieve the information I need to analyze, compare, and make the right decision at the right time? Issues with information Flow Management: How Efficient are You at Managing and Using Information?

  7. Consistency: No Systematic Processing. Consolidation: Someone Someday will have to put everything together to report on specific metrics. Accessibility: No searching capabilities, data not stored on a centralized repository. Capitalization: Any data repository is a potential wealth: Developing Analysis Capacity will enable to Gain Productivity, Sharpen your Strategy, Refine your Goals. Issues with Information Flow Management: Take advantage of the Information You Generate

  8. Travel Request Process Map Travel Request Raw Information * Approval * Travel DB A/P No * Approved * * Travel Info Travel Booker Travel Info Travel Agency Travel Info XML Invoice XML Invoice

  9. Bug Tracking Process Map Filtering End User Tech. Support Bug * * Resolution Help for Resolution Bug DB User Test Not Solved OK Incident closed *

  10. Information Flow Management Perl “Process Sticky” Development Web Based Interface to Data Efficient Data Access II - Perl as a Key Enabler for Process Re-Engineering Consistency Consolidation Accessibility & Capitalization Perl sets a favorable context to Manage Data Flow

  11. Management IT ? II - Perl as a Key Enabler for Process Re-Engineering Need for a Web Based Solution Time / Resource / $ What Management says to the IT guy: “We need to (re-)engineer this process. You’ve got no resources, no money and no time. Just do it. When do you thing it will be ready?” You’re the IT guy! What Do You Say? Will Do… with Perl

  12. IT resources primarily focused on core business applications development and support. • No time to waste in experimenting: need for efficient, documented, pre-existing solutions. Quick developer learning curve makes project staffing easier. Productivity Programmer Learning Curve • Perl reminds of many other programming language. • Quick Learning Curve. • Lot of documented free resources available (CPAN). • Perl is free and highly portable. • Perl is low on Hardware Resources Perl Other Programming language Time Perl Key Advantages Short Implementation Timeline with Few Resources

  13. Perl provides connectors to most of the database types. • Perl has an efficient set of tools and libraries to develop solutions close to the users process: file processing, triggered email alerts, XML data parsing, synchronization with BackOffice… • Flat files are a quick way to deploy centralized web based processes. • They offer an efficient way to store information at no cost. • Flat files present the advantage of being compatible with most software. For example MS Excel allows further analysis of raw data provided by flat file databases. Dealing with Data Perl makes Flat File a Good Solution for Web Based Processes

  14. Flat file processing works fine up to 50k lines (and more) depending on the hardware and server workload. • Well suited for low transactional volume: up to 200 transactions/day. • Travel Request Example: • 10.000 employees, 500 frequent flyers (2 trips/month). • 2000 employees travel twice a year. • => 14.000 rows per year (1k/line => 14 Megs) • => flat files yearly split • To increase search efficiency: • archiving by splitting databases (temporal split / by data value). • Have index files containing keywords and reference to static generated file. III - Flat Files as a Central Data Repository: Keep File’s Size Small

  15. Retrieving information XML Looking for ‘ABC’ Search Results One Data Entry XML output dynamically generated data.csv ……… ……… ……… ……... XSL XML template 2 full scans to retrieve the information needed

  16. Reducing Response Times: File Split travel_2000.csv travel.csv File split: as the database size decreases, the search is more efficient. ……… ……… ……… ……... ……… ……… ……… Example: yearly split of the master data file Natural Way to Decrease the Size of the File Parsed

  17. Reducing Response Times: Static Display data.csv XML 1 database entry is associated with 1 static XML file. ……… ……… ……… ……... Static XML with Ref # Looking for ‘ABC’ Search Results Ref # data.csv ……… ……… ……… ……... XML XSL Processing Allocated to Data Search

  18. Travel.csv ref;user_first_name;user_last_name;request_date;from;to;date;preferred_airline;preferred_time;airline;time;……….. 543;John;Doe;3/5/2000;London;Paris;Air France;8:00 PM;Air France;8:20 PM;………. 542;Sam;Tondac;3/6/2000;Los Angeles;Paris;KLM;6:00 PM;Delta Airlines;5:54 PM;………. 541;Jennifer;Alister;3/6/2000;Houston;Chicago;American Airlines;5:30 PM;American Airlines;3:33 PM;…………. …. …. Travel_543.xml <travel_request> <user> <user_first_name>John <user_first_name> <user_first_name>Doe <user_first_name> </user> <requested_date>3/5/2000 <flight> <from>London</from> <to>Paris</to> <preferred_airline>Air France</preferred_airline> <preferred_time>8:00 PM</preferred_time> …. ….

  19. Reducing Response Times: Reference Key & Static Display Static XML with Ref # Looking for ‘ABC’ Search Results Ref # data_key.csv data.csv XML XSL ……… ……… ……… ……... ……… ……… ……… ……... + : Size of the file parsed can decrease by a factor 5 to 10. Can be correlated with temporal split. - : Edit efficiency decreases. Volume of Data Parsed Significantly Decreases

  20. Travel.csv ref;user_first_name;user_last_name;request_date;from;to;date;preferred_airline;preferred_time;airline;time;……….. 543;John;Doe;3/5/2000;London;Paris;Air France;8:00 PM;Air France;8:20 PM;………. 542;Sam;Tondac;3/6/2000;Los Angeles;Paris;KLM;6:00 PM;Delta Airlines;5:54 PM;………. 541;Jennifer;Alister;3/6/2000;Houston;Chicago;American Airlines;5:30 PM;American Airlines;3:33 PM;…………. …. …. Travel_key.csv Travel_543.xml <travel_request> <user> <user_first_name>John <user_first_name> <user_first_name>Doe <user_first_name> </user> <requested_date>3/5/2000 <flight> <from>London</from> <to>Paris</to> <preferred_airline>Air France</preferred_airline> <preferred_time>8:00 PM</preferred_time> …. …. ref;user_first_name;user_last_name; from;to;airline 543;John;Doe;London;Paris;Air France 542;Sam;Tondac;Los Angeles;Paris;Delta Airlines 541;Jennifer;Alister;Houston;Chicago;American Airlines …. ….

  21. INSERT UPDATE Search Efficiency • Lot of Data • High volume of searches • Few Updates • > 100 inserts/day • Small Volume of Data • Lot of Updates • < 40 inserts/day Reference Keys & Static Display + File Split Ex: Bug Tracking System Ex: Phone Directory Reference Keys & Static Display Static Display + File Split Ex: Travel Request Static Display File Split

  22. Perl makes possible the restructuration of the Information Channels with a minimal investment. For more information: stephane@ezperl.com www.ezperl.com www.arthys.com

More Related