1 / 31

Information Searching and Retrieval from Distributed Databases using Mediators, CORBA and XML

Information Searching and Retrieval from Distributed Databases using Mediators, CORBA and XML. Presented by: Vincent Cheung Supervised by: Prof Michael Lyu, Prof K. W. Ng Dec 18, 2000. Research objectives. Address the Use of XML in enhancing Data Representation and System Communication

gitel
Download Presentation

Information Searching and Retrieval from Distributed Databases using Mediators, CORBA and XML

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Information Searching and Retrieval from Distributed Databasesusing Mediators, CORBA and XML Presented by: Vincent Cheung Supervised by: Prof Michael Lyu, Prof K. W. Ng Dec 18, 2000

  2. Research objectives • Address the Use of XML in enhancing Data Representation and System Communication • Address the Use of Mediator Architecture for Searching in Distributed Environment • Address the Use of XML and HTTP to simulate CORBA IIOP calls • We have developed a CORBA-based mediator system which can overcome the limitation of firewall problem to achieve a worldwide system

  3. Presentation outline • What is Mediator? • Using XML • Implementation with CORBA • CORBA vs. Firewalls • Overcome the Firewall • Evaluation • Future Work • Conclusion • Demonstration

  4. Searching in distributed sources • To integrate different components in the systems, there are three approaches: • CORBA • Mediators • Agents • They are not orthogonal, e.g.: • Implement mediators by CORBA • Agents may use mediators

  5. What is mediator? • A middle layer for forwarding clients queries to appropriate sources, and integrate the data before returning to users Clients UI Client UI Client UI query mediator result Database Engine Database Engine Database Engine

  6. Using XML • We need a COMMON format of information in order to let distributed heterogeneous components for data exchange. • XML provides a good solution because: • XML is semi-structured and highly flexible in data representation. Highly structured traditional relational and object-oriented data schema can be mapped to XML schema without losing information. • A common XML schema for heterogeneous system can be achieved • Hence, we use XML data schema in our system.

  7. A piece of XML data <news> <source>South China Morning Post</source> <date>April 15, 2000</date> <title>Press warning appropriate, says Beijing </title> <reporter>Kong Lai-fan</reporter> <reporter>Greg Torode</reporter> <content>Beijing yesterday defended remarks made by senior SAR-based official Wang Fengchao that local media should avoid reporting separatist views. </content> </news>

  8. XML-QL • To query semi-structured data, traditional query languages may not be adequate • We use XMLQL, which is designed for semi-structured XML data • An example: where <news> $B <news> in ‘’news_db.xml’’, <date> <year>2000</year> <month> 4 </month> <day> 15 </day> </date> in $B construct <result> $B </result>

  9. Why use CORBA? • Common Object Request Broker Architecture • Designed for application development within distributed heterogeneous environment • We use CORBA to build our mediator system.

  10. 3rd tier 1st tier 2nd tier Data Source Web-base UI Servlet Interface Data Source Mediator (forwarding queries and integrating results) Web-base UI … Web-base UI Nth tier Mediator Architecture of our system UI Queries and Results Queries Results

  11. mediator mediator mediator Handling some special cases • Some special cases: Infinite loops, broken connection and too many layer of traversal. • Infinite loops: qid = 123 qid = 123 qid = 123 qid = 123 Solution: To give each query a unique ID, and mediators will keep track all query IDs which have no replied answers yet. Duplicated IDs would indicate an infinite looping has occurred.

  12. mediator mediator mediator Special cases • Broken connection Solution: Use a timeout parameter to specify the maximum amount of time that we are willing to wait. • Too may layers of traversal Solution: use a maximum layer parameter to specify the maximum number of layers that we want to go. Timeout = 5000 Max_layer = 1 Timeout = 10000 Max_layer = 2 Timeout = 15000 Max_layer = 3

  13. IDL design of our system • IDL (Interface Definition Language) defines export interface of CORBA objects • Our IDL design: • The parameter type for special cases handling Struct SysPara { long qid; long timeout; short maxlayer; }

  14. QueryEngine Implemented by QueryMed QueryDB IDL design of our system • Mediator may make queries to Databases or Mediators • Hence, we want Databases and Mediators can be very similar objects • Now, both of them are implementing QueryEngine Interface Interface QueryEngine { String query(in SysPara para, in string xmlquery); }

  15. QueryDB Object • Directly connects to the data source • Caller calls query() • It takes the query statement parameter and make query to related data source • Returns answer in XML string stream format

  16. QueryMed Object • Same invoking method, query() • Besides QueryEngine, it implements another interface, QueryMediator public interface QueryMediator { public QueryEngine[] qelist(); public void qelist(QueryEngine[] arg); public void append_result(String res); } • qelist holds a list of QueryEngine objects, i.e. QueryMed or QueryDB objects, which will be called by that mediator. • It starts a thread for each target QueryEngine object, and the thread will call append_result() to integrate results from various sources

  17. Problems with CORBA firewall • We cannot achieve a worldwide query system because of the firewall. • CORBA uses Internet InterORB Protocol (IIOP) for communication. • Message body of IIOP is encoded in Common Data Representation (CDR), which translates IDL data types into a byte-ordering independent octet string. • Firewalls cannot decode the message body of IIOP in application level. • But we have application level gateways for telnet, FTP, and HTTP

  18. CORBA firewalls • We do have some firewalls which are dedicated for CORBA communications • E.g. IONA Orbix WonderWall and Visibroker Gatekeeper • They have limitations: • Vendor dependent: Orbix firewalls must use Orbix developed client and server objects • Not all features of CORBA can be used: Callbacks may not be used • Not commonly used • Extra purchase, etc…

  19. HTTP Servlet XML CORBA enclave CORBA enclave IIOP firewall CORBA and firewalls • IIOP cannot pass firewall, but HTTP can • A real worldwide CORBA system can be achieved

  20. QueryEngine Implemented by QueryMed QueryDB HTTPGateway Solution to firewall problem • HTTPGateway is also implementing QueryEngine. • HTTPGateway is a virtual query engine to forward the query to the target systems

  21. firewall CORBA Enclave CORBA Enclave Mediator M DB Object HTTP Gateway H Servlet Mediator SM Simulated Call

  22. Sample XML message in parameter passing <request object="QueryMed" method="query" return="string"> <parameter order="1" name="para">} <SysPara> <qid>398498241824033984092</qid> <maxlayer>4</maxlayer> <timeout>2000</timeout> </SysPara> </Para> <parameter order="2" name="QueryStatement"> <string> <news> \$B </news> in "database.xml" <keyword>satellite</keyword> in \$B construct <result> \$B </result> </string> </Para> </request>

  23. Its DTD <!DOCTYPE parapassing [ <!ELEMENT request (parameter*)> <!ATTLIST request object (#CDATA)> <!ATTLIST request method (#CDATA)> <!ATTLIST request return (#CDATA)> <!ELEMENT parameter (SysPara | string)> <!ATTLIST parameter order (#CDATA)> <!ELEMENT SysPara (qid,maxlayer,timeout)> <!ELEMENT qid (#CDATA)> <!ELEMENT maxlayer (#CDATA)> <!ELEMENT timeout (#CDATA)> <!ELEMENT string (#CDATA)> ]>

  24. Evaluation ADVANTAGES: • It can overcome the CORBA IIOP vs. firewalls problem by using HTTP calls,XML, and servlet. • Security issue can still be maintained. External CORBA objects can call only the objects combined with Servlet • Both primitive data types, like String, or complex classes, SysPara, can be well represented by XML in parameter passing

  25. Evaluation • It is not limited for the clients and servers must be CORBA implementation. • Internal CORBA objects would not notice the difference of calling an external objects when comparing to callings of internal objects • With sharing the same XML and IDL standards, we can easily achieve a worldwide query system.

  26. Evaluation WEAKNESS: • With using HTTP calls to simulate IIOP is relative slower than using firewalls dedicated for CORBA, as we need to do many extra works to initialize the servlet, to convert parameters to XML format, invoke the servlets to work, etc.

  27. Future Work • Generalize the IIOP simulation • For all kinds of objects, parameter types, return types, etc • Direct code generation from IDL • Design the mechanism that supports CALLBACKS • When callback can be simulated, we can enhance the features of our mediator system

  28. Conclusion • Using Mediators in querying distributed sources • How to use CORBA and XML to integrate the system • Cooperation between XML and CORBA in simulation of IIOP calls

  29. Demonstration • Aim to show the mediator system can: • make queries to varies sources in parallel • go beyond firewalls User Interface (SHB 1027) FIREWALL Servlet Mediator (SHB 913, pc90003) Servlet Mediator H111, Kuomao Hall Data Source 2 (SHB 913, pc90003) Data Source 1 H111, Kuomao Hall HTTP Gateway H111, Kuomao Hall

  30. Q & A Session Welcome to Give Questions and Comments

  31. <appreciation>Thank You</appreciation>

More Related