1 / 23

Project omniglean

Project omniglean. Team Members: Kenny Trytek Derek Woods Abby Birkett Joe Briggie. Advisor: Simanta Mitra Client: Kingland Systems. Problem statement. Large companies have many layers of corporate hierarchy.

tausiq
Download Presentation

Project omniglean

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Projectomniglean Team Members: Kenny Trytek Derek Woods Abby Birkett Joe Briggie Advisor: SimantaMitra Client: Kingland Systems

  2. Problem statement • Large companies have many layers of corporate hierarchy. • Financial and data records sometimes conflict between various layers/entities. • Accurate and comprehensive company records are needed. • There is a need for “Data Mastering”, to take multiple conflicting sources of data and • determine the reality of the matter in conflict.

  3. Analyst User Omniglean Internet Harvest & store Client User Figure 2.1.1: Concept Sketch Concept Sketch WWW

  4. Functional requirements • System shall autonomously traverse publicly available websites • System shall parse information from downloaded file in portable document format (pdf) • System shall store parsed information in a flat file • System shall allow user the ability to create, modify, and delete records • System shall maintain a normalized database • System shall expose functionality through web services

  5. Non-functional requirements • System shall support up to 250 concurrent users • A single run of system shall complete execution in less than six hours • System shall be easily extensible to include more websites than originally specified • System shall be completed by May of 2011

  6. Constraints and technical considerations • System shall interact with a third party library to facilitate database interaction • The database may not be available at all times • Using SVN to manage code • Using the spiral design process

  7. Omniglean provides a unique combination of access to freely available FDIC and FFIEC data through a data mastering suite. • Omniglean provides access to the mastering capability through web services, to enable a rapid delivery of functionality to customers of Kingland Systems, as well as analysts located in different geographical areas. Market Survey

  8. Potential risks and mitigation • External format change • External availability • New technologies • Web services • Not enough time for testing and debugging • Team members

  9. Cost and resource estimate

  10. Project milestones • Complete modules related to harvesting and transforming the data. • Complete web services and user interface modules • Integrate all modules successfully

  11. Functional decomposition • Harvester – Gathers data • ETL – Transforms data • DAL – Database access layer • Web services – Exposes data to external users • User interfaces

  12. DAL Database ETL Tool Normalized External Client UI Kingland Data Analyst UI Web Svcs. No Conflicts? Scraper Tool System Diagram WWW Data HTML Parser PDF Parser Flat File Create Read Update Delete

  13. Scraper World Wide Web Parser Flat File (XML) PDF Parser HTML Parser Harvester startGatheringData() - Returns the XML document populated with data from the site. getLogFile() - Returns the log file that is either being written to or has been written to this session. stopGatheringData() - Stops all current harvester operations and writes an error to the log file.

  14. Flat File (XML) ETL Tool DAL ETLTool • loadFFIEC() - This method will load the data from the XML file into the FFIEC table. • loadFDIC() - This method will load the data from the XML file into the FFDIC table. • createORGANIZATION() - This method will take the information from both the FFIEC table and the FDIC table and put it into the ORGANIZATION table.

  15. Database DAL Add() Find() Update() Delete() ETL Tool User Interface Data Access Layer Organization- A class that creates and maintains a connection to the ORGANIZATION table GetConnection() CloseConnection() OrganizationService- This class will allow CRUD functionality with an Organization object. Find(String organizationId) Add(...) Delete(String organizationId) Update(String organizationId, ...)

  16. Authenticated Unauthenticated Read() Write() Update() LogOut() Delete() Webservices Allows remote users to access the database through the Internet • Create() • Read() • Update() • Delete() • login() • logout()

  17. Technology platform • SQL Server 2008 • Visual Studio 2010 development environment • Microsoft Windows operating system • WSDL and SOAP for web services

  18. Test plan The team will be testing the system in three phases. The first phase is testing the individual modules, the second phase will be testing the integration of the modules, and the final phase will be testing the system as a whole.

  19. prototyping We have begun prototyping. The harvester is able to traverse the necessary websites easily. The ETL can read in XML files. The user interface has been mocked up.

  20. Current project status

  21. Task responsibility Kenny Trytek – Team Leader, responsible for the harvester Derek Woods – Developer, responsible for the ETL and UI Abby Birkett – Developer, responsible for database and DAL Joe Briggie – Developer, responsible for web services

  22. Plan for next semester • Continue prototyping • Continue to meet with client to be sure we are meeting expectations • Develop a more thorough test plan

  23. Questions?

More Related