710 likes | 724 Views
RETS Software Development. New Topic. RETS the standard RETS transactions Identify issues, concerns and recommendations Sample session workflow. RETS Software Development Topic Objectives. Improve understanding of the RETS architecture Build familiarity with the RETS document
E N D
RETS Software Development New Topic • RETS the standard • RETS transactions • Identify issues, concerns and recommendations • Sample session workflow
RETS Software Development Topic Objectives • Improve understanding of the RETS architecture • Build familiarity with the RETS document • Enable more productive RETS software development
RETSPrinciples - 1 • Stateless communications protocol • Carry state using a cookie: • Basically a token that points to a persistent store on a server somewhere • RETS defines an optional cookie for this purpose • Recommendations: • Return all cookies received • Client applications using browserswill need to accept cookies
RETSPrinciples - 2 • Uses Http-based transport • Uses DMQL - an open query language • Supports XML and site-specific output data formats • Supports site-specific metadata to describe the local MLS system • Recommendation: • Effort to support metadata produces interoperability benefits
RETS Document • Organization follows a typical real estate industry workflow: • Initiate single session • Multiple data requests • Terminate session • References standards produced by other standards organizations RETS document can be challenging to read
Login Transaction Making the Initial Connection • Authentication • Response with authorized resources • Types: • Basic • Digest using MD5 or SHA1 • SSL • Recommendations: • Implement Digest using MD5 • SHA1 if you can
Login TransactionAuthentication • Most common authentication type is digest using MD5 • Somewhat tricky to implement algorithm • Easiest to borrow the implementation from one of the open source projects • When in doubt: • Ask the RETS-DEV list • Ask your software vendor
Login TransactionDigest Authentication • Message Digest algorithms are one-way functions • Easy to perform the calculation • Difficult to reverse the calculation • Both sides calculate the same message digest • Transmit only the digest • Compare your calculated digest with the received digest
Login TransactionUserAgent • Application name accessing the MLS server • All client requests MUST include this field • This is a standard HTTP header field as defined in RFC 2616 • MLS server should validate application name presented against list of allowable applications maintained on a table • Application name and userid is a useful key to a table that tracks MLS server usage
Login TransactionGotchas • Using operating system libraries for MD5 can cause authentication failure • Proxies and tunnels can be a problem • Re-authenticate is part of the RFC but few vendors enforce it • Case may matter
Sample LoginChallenge Response Header 1 HTTP/1.1 401 Unauthorized xxx Date: Mon, 02 Aug 2004 05:23:54 GMT Server: WebLogic XMLX Module 8.1 SP1 Fri Jun 20 23:06:40 PDT 2003 271009 with WWW-Authenticate: Digest realm="users@ftc2.com", nonce="313035", opaque="6e"
Sample LoginChallenge Response Header 1 cont. Content-Length: 0 Content-Type: text/html RETS-Version: RETS/1.0 Cache-Control: private
Sample LoginNonce nonce= "313035393731353435373630342058dd631265e3360724c45d15f23aec7a"
Sample LoginOpaque opaque="6e6f742075736564"
Sample LoginChallenge Response Header 2 HTTP/1.1 200 OK Date: Mon, 02 Aug 2004 05:31:03 GMT Server: WebLogic XMLX Module 8.1 SP1 Fri Jun 20 23:06:40 PDT 2003 271009 with Content-Type: text/plain
Sample LoginChallenge Response Header 2 cont. Set-Cookie: RETS-Session-ID=20000439936 RETS-Version: RETS/1.0 Transfer-Encoding: Chunked Cache-Control: private 0230
Sample LoginChallenge Response Body 2 <RETS ReplyCode="0" ReplyText="Success"> MemberName=Mark Crawford User=632344, NULL, NULL, 632344 Broker=National,1 MetadataVersion=1.2.0 MinMetadataVersion=1.1.7 OfficeList=National;1 TimeoutSeconds=900
Sample LoginChallenge Response Body 2 cont. Search=http://rets.ftc2.com:6103/search GetObject=http://rets.ftc2.com:6103/get object Login=http://rets.ftc2.com:6103/login GetMetadata=http://rets.ftc2.com:6103/getmetadata Logout=http://rets.ftc2.com:6103/logout </RETS> 0000
Login Transaction Capability URL List • Response from the Login transaction includes a list of known resources on the server • Each URL points to a RETS transaction type • Capability URL list may be different based on the user and their authorization level • Required that the server return a list containing at least Search • The URL references are not permanent locations • the standard allows the location to change
Action-URL and Get Transaction • Included in the Capability URL List is an Action-URL • Points to a human readable message • Most useful if changed regularly; perhaps daily • Immediately following successful login, may need to do a Get on the Action-URL • Get Transaction is a file retrieval from the server
Metadata • Description of the structure of the data on a specific server • Unique to each site • There will be common elements • Most powerful feature of the RETS specification Perhaps the most challenging part of the RETS specification
GetMetadata Transaction Overview • Transaction to retrieve system metadata • Can select a specific level, the level and descendants or a specific metadata ID • Formats supported are COMPACT and Standard-XML • Metadata id value of 0 • Request is for all types at the same level – breadth • Metadata id value of * • Request is for all types at the same level and all child types - depth
Metadata FormatID • Hierarchy of types of metadata • See Figure 11.1 on page 11-2of the RETS specification • When referencing hierarchy as ID • Parent: Child is the form • Example: • Type: METADATA-UPDATE_TYPE • ID: Property: RES: Add
Metadata FormatVersioning - 1 • Hierarchy of metadata is versioned • Changes to lower level of the hierarchy propagate upward • Example: Changed METADATA-TABLE attribute will propagate up through CLASS, RESOURCE and SYSTEM
Metadata FormatVersioning - 2 • Caching of metadata is permitted • Use the login information to determine metadata changes • Recommendation: • If possible, get all the metadata on a change instead of attempting to get a specific type
Metadata FormatVersion 1.5 changes • Added the Foreign Keys metadata • Provides relationships between the offered resources of the form parent:child • Clarified a number of issues with Metadata Format • Recommendation: • Use 1.5 as a reference even when implementing 1.0.1 compliant software
Application Responses to Metadata Changes • Ignore metadata • Change breaks the application, intervention required to fix • Static metadata • Change is invisible, applications keep working, intervention to required view change • Dynamic metadata • Change is visible to applications without intervention
Search Transaction Overview • Most important RETS transaction • Can use either http GET method or http POST method • Example – GET method requesthttp://rets.server.com:6103/search?Class=RES • Issue: GET method is limited in the character length of the request • Recommendation: POST method
Sample SearchSearch Request Header HTTP/1.1 200 OK Date: Fri, 31 Jul 2004 23:22:43 GMT Server: WebLogic XMLX Module 8.1 SP1 Fri Jun 20 23:06:40 PDT 2003 271009 with Content-Type: text/xml
Sample SearchSearch Request Header cont. Set-Cookie: RETS-Session-ID=20000439934 Set-Cookie: RETS-Request-ID="42" RETS-Version: RETS/1.0 Transfer-Encoding: Chunked Cache-Control: private
Sample SearchSearch Response Body 0fe8 <?xml version="1.0" ?> <!DOCTYPE RETS SYSTEM "http://www.ftc2.com/rets/dtd/RETS-20010812.dtd"> <RETS ReplyCode="0" ReplyText="Success"> <COUNT Records="2507" />
Sample SearchSearch Response Body cont. <REData> <Properties> <ResidentialProperty> <Building> <BuildingDescription> <Stories>1</Stories> <PropertyCondition>Shows Well</PropertyCondition> RETS Software Development
Search TransactionSearchType Required • SearchType: The resource id to search on • Example – SearchType=Property
Search TransactionClass Required • The class of data within the resource to search against • Example – Class=RES • Issues: • Some of the data may be in system specific Class names rather than the Class name that represents an XML Standard Name
Search TransactionQueryType Required • QueryType: The query language • Only two types: DMQL and DMQL2 • Example – QueryType=DMQL2 • Issues: • Mainly stylistic difference, DMQL2 supports quoted text • Not suitable for highly complex searches • Not possible to perform cross Class searchesExample – NOT Residential and Lot Land
Search TransactionQuery Required • Query: • Key – value pairs • Uses field name based on metadata or on standard names • Example • Query=(ListPrice=200000-300000),(Status=|A) • Issues: • Lookup type queries can execute for a long time
Search TransactionOptional Arguments • Optional argument are just that • A vendor MAY choose to implement none, one or more • Software is RETS compliant without any of the optional arguments implemented • Issue: • Mismatch between client & server feature sets • Recommendation: • Do NOT assume, verify the feature set
Search TransactionCount • Default is no count • Behavior outside of given three values not defined
Search TransactionFormat • Return records in XML or column header/row values format • XML is more verbose, but contains structure information • Some issues with site specific XML – clarification in RETS needed • Default is Standard-XML • Issue: • Site specific XML is not defined in the current standard
Search TransactionFormat Sidebar: XML - 1 • XML is a meta-language that defines a markup language • RETS is an example of an XML application • An application consists of a DTD and documents that obey the rules embodied in the DTD • The XML DTD is a document that describes the structure and elements of an XML document
Search TransactionFormat Sidebar: XML - 2 • The XML document takes data and applies the DTD • RETS will eventually move from a DTD based vocabulary to a Schema-based vocabulary
Search TransactionFormat Sidebar: Compact Formats • Provides a delimited column description • Provides delimited data in the same position as the column description • Field delimiter is chosen by the server • Gotcha: • Delimiter tag is a two character representation of the ASCII character • Delimiter value may collide with data values in earlier implementations
Search TransactionLimit • Limits the number of records returned: • The NONE well known name requests a suspension of limit • Behavior is implementation dependent • Requesting a Limit should make the query faster • May still search for all records but only transmit records to a Limit
Search TransactionOffset • Return records starting from the specified point in the result set: • One method to handle very large data pulls • Given that the underlying transport mechanism is not guaranteed, very large results can fail with transport errors • Given that the data volumes can be very large, out of memory errors can occur
Search Transaction StandardNames • Indicates that the query uses the standard names or the system names: • Servers are not required to implement standard names to meet RETS compliance criteria • Standard Names are intended to be common across all RETS systems Refining the list of Standard Names is an ongoing process
Search TransactionRestricted Indicator • Certain fields may have restricted visibility: • Client may request fields that are restricted be replaced by a specific character string • Example - compensation amount field • Client may request that this amount be masked by ### • Server default is to return a null value