1 / 32

Hyrax Architecture

OLFS. BES. Data. Hyrax Architecture. Two cooperating processes: Front-end provides DAP interface Back-end reads data Both parts can be customized Front-end: different network protocols Back-end: different data formats/systems N-Tier design is flexible, secure. Hyrax Architecture.

caswell
Download Presentation

Hyrax Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OLFS BES Data Hyrax Architecture • Two cooperating processes: • Front-end provides DAP interface • Back-end reads data • Both parts can be customized • Front-end: different network protocols • Back-end: different data formats/systems • N-Tier design is flexible, secure

  2. Hyrax Architecture • Front-end (OLFS): • Authentication & Authorization (via Tomcat) • DAP2 compliant interface. • SOAP interface. • Extensible via pluggable dispatch handlers • Back-end (BES): • Data resource access. • Extensible (see Patrick’s slides) • Single/Multiple Machine Installations • Security

  3. OLFS BES Java Servlet Engine Unix Daemon Hyrax Architecture BES Commands DAP2 THREDDS HTML XML- encapsulated object Data Store Optional THREDDS catalogs File system with data files, SQL Database, …

  4. OLFS and Authorization/Authentication • Authorization & Authentication (Az/Ac) actually provided by Apache or Tomcat • Apache/Tomcat provides a range of Az/Ac features: • Realms: Role-based Az/Ac • Usernames & passwords -> Authorization • Roles -> Authentication • Realm info in RDBs, XML docs, LDAP, … • TLS/SSL: • Server and client Az/Ac • Secure data transmission • Tomcat provides single sign-on capabilities • Clients must support cookies • Suitable for portals

  5. Possible Installations • Both OLFS and BES run on one machine • OLFS on one machine and BES on another • One OLFS and several instances of BES on different machines • OLFS communicating with one or more BESs and other backend processors

  6. Hyrax Architecture Summary • Hyrax can be installed on one machine or several • Installation security merits serious consideration • Authentication & Authorization are handled by the web servers (Tomcat and Apache)

  7. Extending the OLFS • Extension ‘modules’ written in Java • Added to a directory within Tomcat • The new modules have complete access to the request information • Both HTTP GET and POST requests • There is some significant processing done before the handler is called: • Conditional GET Requests • Authorization & Authentication

  8. The OLFS uses an ordered list of Dispatch Handlers. • Each handler on the list is asked if it can handle the incoming request. • The first handler on the list that claims the request gets to handle it. • List order is important, as some handlers (for example THREDDS) may claim to handle requests that could (should) be handled by a different handler. • Handlers are free to do anything they need to handle a request: Contact a remote system/process, read files, spawn threads, et cetera. OLFS Dispatch Handlers

  9. Extending the BES • Extension ‘modules’ written in C or C++ • Download already existing modules • Write new modules • Add these modules to the BES configuration file to be dynamically loaded • Modules interact with the BES and can interact with other modules • Modules add to, or modify functionality in, the BES framework

  10. BES BES Framework Network Protocol and Process start/stop activities PPT* Initialization/ Termination DAP2 Access Data Catalogs BES Commands/ XML Documents Commands** NetCDF3 HDF4 FreeForm Data Store Interfaces … *PPT: point to point transport **Some commands are built in, most load at run-time

  11. BES Extensibility In what ways can you extend the BES? • New request handlers (data handlers like netcdf, hdf4) • New response handlers (Cedar's Flat, Tab, Info) • New commands (ESG’s commit command) • Container and Definition storage • Aggregation engines • Methods of returning your data (return as nc) • Initialization/Termination callbacks • Exception Handlers • Reporters

  12. THREDDS Catalogs

  13. THREDDS responses • Use THREDDS to define a logical hierarchy that’s distinct from the set of directories that actually hold the data. • We can request THREDDS catalog XML files using ‘catalog.xml’ or HTML pages using ‘catalog.html’ after a directory name. • While the directory browser works for any directory, THREDDS catalogs are valid only for the logical hierarchy they define • Files/Directories not included in that hierarchy have no catalogs

  14. THREDDS examples • Switch Hyrax to the THREDDS HTML view: Choose the HTML view

  15. The THREDDS HTML view • The top-level THREDDS catalog on our test server defines a single data root directory (SVN Test Data Archive) • This illustrates how THREDDS can be used to control the view of data presented by the server • Use ‘catalog.xml’ in place of ‘catalog.html’ to get the catalog data in an XML document.

  16. Traverse the links to find data

  17. THREDDS data set page • THREDDS catalogs can list more than one access mechanism - here we see on the DAP, but WCS, WMS, et c., are other possibilities

  18. Choosing DAP access leads to the HTML form

  19. Security: Authentication & Authorization

  20. Authentication & Authorization Hyrax currently relies on the security features implemented by Tomcat for authentication and authorization services. The Tomcat authentication model is based on Realms and roles. • A Realm is a “database” of usernames and passwords that identify valid users of a web application. • A role is similar to a UNIX group because access to to resources is granted to all users possesing a particular role. A particular user can have any number of roles.

  21. Authentication & Authorization Tomcat supports 5 standard plug-ins that support connections to various sources of authentication (Realms)

  22. Authentication & Authorization • JDBCRealm - Accesses authentication information stored in a relational database, accessed via a JDBC driver. • DataSourceRealm - Accesses authentication information stored in a relational database, accessed via a named JNDI JDBC DataSource. • JNDIRealm - Accesses authentication information stored in an LDAP based directory server, accessed via a JNDI provider.

  23. Authentication & Authorization • MemoryRealm - Accesses authentication information stored in an in-memory object collection, which is initialized from an XML document (conf/tomcat-users.xml). • JAASRealm - Accesses authentication information through the Java Authentication & Authorization Service (JAAS) framework.

  24. Authentication & Authorization Realm and Role configuration is achieved by editing XML files in the Tomcat distribution. Passwords may be saved as clear text, or if desired they may be stored in a digested form. The standard Realms support SHA, MD2, and MD5 digest algorithms.

  25. Hyrax Logging

  26. Hyrax Installation: Logging • Access Logging - Many people will want to record access logs for their Hyrax server. We want you to keep access logs for your Hyrax server. The easiest way to get a simple access log for Hyrax is to utilize the Tomcat/Catalina Valve Component • Informational/Debug Logging - In general you shouldn't have to modify the default logging configuration for Hyrax. It may become necessary if you encounter problems, but otherwise I suggest you leave it be. Enabling it can both consume disk space and increase your security vulnerability.

  27. Hyrax: Logging Access Logging Since Hyrax's public facade is provided by the OLFS running inside of the Tomcat servlet container you may utilize Tomcat's handy access logging which relies on the org.apache.catalina.valves.AccessLogValve class. By default Tomcat comes with this turned off. It can be easily enabled by editing an XML file in the Tomcat distribution.

  28. Hyrax: Logging Informational and Debug Logging in the OLFS The OLFS uses the Log4j logging package to provide an easily configurable and flexible logging environment. All "console" output is routed through the Log4j package and can be controlled using the Log4j configuration file. Log4j allows the user to control logging output in a hierarchical manner from the (java) package down to the individual class level.

  29. Hyrax: Logging Informational and Debug Logging in the BES The BES has two forms of logging. • Event logging in a bes.log file. This is configured in the bes.conf file: • BES.LogName=/path/to/bes.log • BES.LogVerbose=no • Debug logging for the BES • besctl start -d “/path/to/bes.debug,nc,bes,ppt”

  30. BES Debugging • Format of the command line option -d • “cerr|<log_file_name>,<context1>,…,<contextn>” • E.g. -d “/tmp/bes.debug,nc,bes” • Specify a context of “all” to turn on debugging for everything • Writing debug statements in your code • Macro BESDEBUG( “<context>”, <output to ostream> ) • E.g. BESDEBUG( “nc”, “reading DAS for “ << file << endl ) • Chunk of debugging If( BESDebug::IsSet( “nc” ) ) { list<string>::const_iterator i = _somelist.begin() ; while( ; i != _somelist.end(); i++ ) BESDEBUG( “nc”, ““ << (*i) ) }

  31. BES Debugging Most BES classes derived from BESObj, and some libdap class derived from DapObj • You can dump BESObj instances and DapObj instances to ostream • virtual void dump( ostream &strm ) ; • E.g. DAS &das ; BESDEBUG( “nc”, “DAS = “ << endl << das ) • Allows for indentation as well

  32. BES Debugging - dump method example

More Related