I've been involved in scores of MLS contract negotiations over the past dozen years, working alongside attorneys to make sure business issues are properly addressed - but the environment keeps changing and I must be ever vigilant to make sure new issues are taken into account so that my clients are well protected.  Last year I posted an article on the subject – “Negotiating Win-Win Technology Contracts” (http://www.realtown.com/mattcohen/blog/negotiating-technology-contracts) but it’s time for an update.

RETS (www.rets.org) has been a boon to the real estate industry, but it has added another level of complexity to MLS system contract review.

Consider RETS when defining contractual performance and uptime language.  Back when RETS was less critical, one might define system speed, functionality, and availability as part of a performance and uptime guarantees solely in terms of common “front end” functions: this search would take a maximum of this amount of time; these statistics would take a maximum of that amount of time; these functions work substantially as they were documented this percent of the time. Now that RETS is becoming more important for many MLSs, benchmarks for the speed, functionality and availability of RETS must be included in the contract either explicitly or as part of more general performance-oriented language. Also, a process for setting benchmarks and the follow-up step of measurement against those benchmarks must be set.

It’s also important to make sure the RETS feed is available, tested, and validated as early in the conversion process as possible, and that there is a contractual obligation on the part of the vendor to provide this in a timely fashion – ideally at least two months before system cutover. This is especially important if your MLS has been providing delimited data and photographs via FTP but would no longer plan to do so with the new vendor. You don’t want a lot of last minute calls from brokers, IDX vendors, statistics producers and other companies getting a data feed from the MLS – all complaining that there is not enough time for them to learn RETS and transition to it. Even if you provided RETS to these companies in the past, the new vendor’s RETS server may likely have some idiosyncrasies or differences in RETS metadata that will cause problems if brokers and others to whom data is distributed don’t have a chance to properly test the new RETS feed properly.

Those are just two examples of how a single technology becoming more important requires MLS system contract changes. MLSs should be wary of signing extensions without appropriate review on contracts that no longer have the relevance they did during the original term and should carefully review their new contracts to ensure business issues are addressed. MLS systems are continually changing and interacting with other real estate information systems in new ways (i.e. SSO) and MLS system contracts have to reflect that. 

 

I'm on my way back from the RETS meeting in Chicago so I'm writing up a small recap of what I was privy to during the meeting sessions I attended.

First, a bit of background. Last week the COVE group (several of the largest MLSs) met and determined that the need to complete data standardization (improving on standardization of data fields, including field contents) was urgent. They plan to work on this to and then present the work to RESO – hopefully this will accelerate the current pace of the standardization effort.

During the “RETS Issues, Challenges & Perceptions Panel”  Matt Lavallee (MLS Property Information Network) presented a number of his theories, including an addition of or move to a standard more focused on synchronization than the current transactional approach. In his market he sees few use of ‘ad hoc’ or ‘live’ queries and provides a "live" CSV feed or direct SQL access. Other MLSs, vendors, and client writers explained their differing perspective on the feasibility of this approach. Rob Overman (LPS) in particular described the use of ad hoc queries for mobile PDA and voice-based solutions. Dan Woolley responding via Twitter indicated that one still needs transactional model for small data sets and real-time reports - CMA, buyer tours etc. Ray Ewing (SANDICOR) said that they have many vendors that query just for a few listings.

Matt L. then suggested that there are only a few nationwide vendors needing data standardization – that most vendors (local) don't care. This was not in alignment with others understanding, especially the COVE group as previously mentioned. It also didn’t make sense to me, as the standardization is a key component of making it easy for vendors to get data and provide their product into additional markets. Mike Wurzer (FBS) spoke up on the subject of data standardization to say that “Brokers are sick of the expense of disparate data feeds. Transport is irrelevant. The lack of data standards is the pain point. Data standards are possible - CARETS and others did it. RESO needs the [guts] and resources to drive that down the road.” Kristen Carr (Bridge Interactive Group) responded that data standardization is moving forward. Mary Frances Adams (TREND) indicated that “If we all agree on a data standardization and every MLS maps to that, we can all talk to one another - but we don't have to change input sheets.” This may not exactly be true if data standardization gets down to what enumerated values of any given field might be and there is no way to translate from existing fields to the new values. Matt L. made a great point – “It's not the name changes that are hard. How do you back out 12 years of old data? Condo is not a property type. It is an ownership type. How do you back out 12 years of data where condo has been a property type? To our membership the data has value in the form it is in.” Pat Bybee (Metrolist) warned, “We're not fond of mandates from NAR. When they said they would create green fields it had ripples around the industry.Setting up an aspirational standard would be good. As soon as NAR mandates something forces rally against it.” Matt L. responded, “Standard names have been there a long time but it's voluntary - no one uses them.”

Matt L. then expressed that “RETS keeps little vendors out of markets”.  While there was some agreement that RETS has a learning and support curve, there was not general agreement for the conclusion among MLSs and others present. Jeremy Crawford (SANDICOR) specified that “We've seen vendors from all industries come, in our market and I have vendors that have been successful in minutes. No way I'd open up SQL and try to lock them down.”  Jim Smith (NTREIS) said, “When we used SQL I had 2 DBAs … Now that I have RETS, I get by with a clerk. I got one support call last week for RETS. One.”

Moving on to another subject, I brought up the business concern that the standards effort was not properly resourced to move the standard forward at a good clip – COVE group is just one example of the impatience in the market. I suggested roles for leadership, documentation, project management, and communications. A discussion of how to pay for such resources ensued which I ended by saying, “Every other industry that has standards has figured out how to staff and pay for that effort. We can too.” Pat B. agreed and indicated that the RESO board will be putting a budget into NAR to address these needs. David Harris (eNeighborhoods) agreed that the effort needs resources and suggested that everyone send email to bod@rets.org on the subject to indicate their support.

The second day, the RESO board reported that they are working on a budget - workgroup chairs will submit budgets to the board soon.  They listed 2009 Accomplishments - Server compliance, document managment, improved vendor/MLS communication, and a budget process definition. The board committed to put together a RETS roadmap - purpose, mission, vision, short, medium, long term goals - Kristen Carr, Ryan Bonham, Sergio Del Rio, Steve Clarke leading. At a high level the board has short term goals for improvement - accountability, compliance, standards publication. RETS medium term goals include the strategic plan and roadmap, RETS logo and name usage, and data standardization. Long term goals include  data standardization  for additional schema, review RETS to streamline the standard, additional directions for RESO, and expanding compliance. The RETS roadmap will 'connect the dots' for these goals with timeframes, long overdue!

Hitachi showed the technical structure and features of the compliance tool at great length. They also showed their selected tool for documentation – Confluence wiki. This is exciting, though when I posted to Twitter one friend tweeted back that “Confluence is kind of challenging, actually. My school uses it predominately and, for one, formatting is a real problem. Another friend tweeted, “You'll have to tell me what you like about Confluence. We use it at work and we all hate it.” I’m still excited by this work, because improved documentation management is desperately needed and it looks like Hitachi has done a nice job customizing the tool to the RETS community’s needs.

Both RETS change proposals made by Troy (FBS) passed by wide margins:

• #79 Add Preferred flag to GetObject responses. This proposal recommends adding a flag to GetObject responses that indicates if the given object is the preferred or primary object for the requested record...In some MLS software systems, the ability for photos to be uploaded also allows for a particular photo to be marked as the “preferred” or “primary” photo for that record (without it necessarily being the first photo uploaded or moved as first in the list of photos) which indicates that this photo should be used on reports and other displays when only a single photo is shown.
• #80 Optional Query. This proposal recommends changes to RETS to allow the Search transaction Query and QueryType parameters to be optional. By making the Query and QueryType parameters optional, users only need to provide 2 required parameters (SearchType and Class) and are able to instruct the server to return all records available to their account.

Tomorrow the roadmap will be preliminarily discussed (originally this was a workgroup meeting, so I did not plan to attend) though no decisions will be made at this meeting. For some insight into my own thinking on the roadmap, see my previous posts, Completing RETS (http://www.realtown.com/mattcohen/blog/completing-rets) and Completing RETS: The Survey (http://www.realtown.com/mattcohen/blog/completing-rets-survey). I had a great conversations about the roadmap with Paul Stusiak who will be leading that discussion before heading to the airport.

It's difficult to write about opportunities for the future of the RETS standard without some folks thinking I'm bashing the past. To be clear, I honor and respect the past - but also welcome the future. Discussing the possible future of the RETS standard and engaging stakeholders can only make that future better. So, I put up a short survey for MLS executives to let their voice be heard.

In my previous post, "Completing RETS" (http://www.realtown.com/mattcohen/blog/completing-rets) I pointed out that there are some types of data that it would be ideal to represent using RETS to achieve the business objective of making it easier to move from MLS system to MLS system or to move data from MLS systems to other systems where the data could be used. When asked how important each data type was, here's what the MLS executives said:

Documenting and being able to transfer MLS business rules via RETS was the most strongly desired of the data types I queried on.  Every MLS executive has gone through some pain moving these rules from system to system. As I had previously commented on my blog, how to work with or incorporate some type of business rule markup language is an area ripe for discussion. Being able to move prospects and saved searches from system to system was also strongly desired - end-users not being able to move this data from system to system is a part of what makes MLS system transitions more difficult and time consuming for MLS subscribers. Thinking beyond the MLS, being able to move such data from broker and agent websites into the MLS - or from an MLS public website to a broker/agent website once the consumer has selected a professional to work with - would be wonderfully convenient. Customized search screens in the MLS are not that commonly used, so that didn't rate as highly. David Harris from eNeighborhoods had suggested that Open Houses was an area that needed to be better represented in RETS, and while the MLS executive segment didn't see it as that important, that non-alignment may point to the need for even wider discussion and measurement-taking, involving other stakeholders.

A supermajority of survey respondents wanted to see data types they ranked as 'Very important' or 'Important' to be added to RETS over the next year. I'm not sure that's possible given the current resources of the RETS community. To quote Mike DelGaudio, “Finishing out these remaining schemas …  is can of worms that many implementers won't be prepared to execute quickly, affordably, or safely.”  And, as one respondent said “I rated all the elements as very important and want them all this year. Will that happen? Heck no, and I know it. … RESO has [taken] years to approve what we have now. … After years of trying we don’t have common set of data definitions. If RESO can’t get that done, how on earth do you expect them to tackle some of the "people" fields (to use DelGaudio’s term).”  I agree – this is a challenging endeavor! However, if the interest is there and scope and time are truly fixed then MLS executives could discuss the third part of the project equation, additional resources, with NAR. This is all part of creating a project plan for RETS - a roadmap that we can measure progress against.

That leads to an interesting subject – the RETS roadmap. 26% of the MLS executives surveyed said they were aware of the “long term roadmap for RETS”. That’s really fascinating, because there’s no such thing. There was one back before 2007 but once we left the road heading to RETS 2.0, the roadmap was stuffed into the virtual glove box (just try finding the roadmap on rets.org) and the focus of the effort turned to small-scale changes and, based on the immediate needs caused by the NAR policy, compliance. I confirmed the lack of a long term roadmap with some RESO board members.

On the subject of MLS executive stakeholder involvement in the standard, the majority of MLS executives have not been to a RETS meeting and only 34% of those surveyed said they understand the process for making changes to RETS. The good news – of those five executives that said they had tried to get a change to RETS made, four said that they were successful. The process can work if MLS executives and other stakeholders want to get involved!

While surveying MLS executives on the future of RETS, I did ask some questions about the present that might interest the reader:

58% of respondents said that RETS has made accessing data easier for subscribers and only 16% said it made it more difficult.

• I would say that once they understand the process they like it better, but it is getting them to move to the new process that is more difficult.
• Most Brokerages are required to find vendors who understand RETS. Small vendors, in most cases, don’t have a clue.
• The vendors all seem to have different problems, they blame the MLS programmers. It seems that the different RETS clients cause different problems for the vendor which in turn causes problems for the MLS.
• Too many web masters don’t understand it.
• When we switched from one platform to another (through the same vendor) the agents still had to reenter all their saved contacts, etc. The possibility that this could be different would be a huge improvement.

• More online resources to help third parties get started with RETS.
• MLS administrators need more training as the subscribers seem to be learning RETS as well and a difficult time implementing it.
• Develop a standard RETS Client

As one respondent said, “[RETS is] easier for staff to provide MLS® because they only have to set permissions and criteria once. It’s harder for staff because now have they must liaise with 3rd Party Providers educating them as to the different "flavour" and/or implementation of RETS.”

When asked what RESO could do to make it easier, respondents had a number of comments – but most of them seemed more focused on their technology vendors rather than the standards effort itself:

• Get the vendors to understand that RETS is RETS and the MLS does not have to customize for each vendors RETS client.
• Allow more filters; i.e. a way to "throttle" the data.
• Vendors using RETS need to change their model of operation. Instead of open access for each subscriber, vendor should have an authenticate standard and then all queries are done by the vendors account, instead vendors say turn RETS on for each subscriber creating security issues.

Of those surveyed, a 53% majority still serve data primarily using FTP or some other method – hopefully this will improve over time.

There has been a lot of great discussion lately about charging for RETS, reflecting my 2003 paper on the subject (for which I took so much heat). It’s nice to be validated six years later!

One respondent clarified: “The charge is only for third party vendors who sell product to members. The real issue MLS’s need help with is defining the categories of data subscribers. We don’t charge members for RETS feeds, other than a criteria change fee.”

Let’s end this post on a really positive note. I asked MLSs what benefit they’ve gotten from RETS. There were some negatives to start with, but most of the comments were quite positive.

• I think the true test will be what impact it will have on our servers and bandwidth issues. Also updating to more current versions seems to be an issue.
• At this point none.
• Have not seen any benefits to our MLS. The programmers seem to spend more time on RETS than they do programming the MLS System for the members.
• RETS hasn’t added anything except another way to transfer data. We did fine with and ftp feed and framing. It might have made a difference to some IDX vendors, but was no big deal to MLS.
• If you ask an agent how they get data to their site, they couldn’t tell you. They don’t care as long as it gets there.
• We use ftp to send data to most third-party vendors. For the most part RETS access is used to provide members with another means of data/photo access other than our MLS vendor’s standard front end. One reason we use ftp to service third party vendors is that with RETS my staff frequently has to spend time helping the vendors understand our data structures and resolving issues with their RETS client and query structures/syntax.
• To up a RETS data feed is much, much too confusing. IDX … has an extremely straight forward up process - terminology and the layers to things up in RETS makes no sense to non-programmers

The more positive responses:

• For vendors who are familiar with RETS, they are up & running with their application within days, instead of weeks or months.
• CARETS
• More flexibility with the data, more timely updates and easier marketability for our products.
• Got rid of the ftp servers and closed the security loophole that ftp created.
• Created data standards for accommodating data from different marketplaces; created modules for our MLS application which can be unbundled and plugged into other RETS utilities
• Simplified support to vendors, stability in providing data, not needing to provide various versions as in FTP files, easier for image access (we prefer that vendors pull via http (URL supplied by RETS) versus accessing photo via RETS - saves time, bandwidth etc., able to offer real time data
• Expedited, more secure, and manageable means of data transport.
• Better support and access to real time data
• Better, more accurate and standardized feeds available to our members.
• The move to RETS has made the prospect of data sharing with neighboring systems more straight forward. The neighbor can pull down the data on their own with limited use of our programming staff.
• We have been easily able to partner up with third party aggregators.
• We’ve had a few venders that will only do RETS, so this has opened a door of opportunity for them to present their product to our members.
• List Exporter (self hosted) uses RETS to provide filtered IDX files which requires very little policing of IDX sites.
• Depending on the service - brokers might pay extra.
• For the vendors serving our subscribers it is very valuable.
• Much easier to implement and provide from our side. Non-dues revenue - increased choice for members dealing with 3rd Party Suppliers - better controls for access to MLS® data
• We use it internally to calculate some statistics and to feed a MLS Data Checking tool. It feeds our public website and a good portion of our broker IDX feeds. I agree with the responses to the blog, facilitating the transport of saved searches and such is a very big deal.
• It’s important for the future of real estate to have a standard that is going to put everyone one on the same page. RETS has opened a lot of doors as far as data is concerned for us and all MLS’s that are utilizing it but it’s a standard so I believe it will change as needs and demands evolve!

RETS has come a long way over the past ten years on the backs of hard-working volunteers – but there is room for RETS to mature further. Involving stakeholders and looking at business objectives with fresh eyes will be important as we move forward. Putting more resources behind the effort and continuing to formalize the requirements gathering process, the road map, and the project planning and communications side of the RETS effort will be crucial, if one wants to see more than small incremental changes to the standard made over a long period of time. Working to restructure the effort and providing the resources needed to do so will require resources for RESO that MLS and Association executives will need to advocate for, if they have an interest in RETS moving forward in ways outlined in this article.

The Fall RESO/RETS Conference is September 23rd - 25th, 2009 in Chicago, IL - more information is available here: http://www.rets.org/meeting

 

 

The rise of third-party RETS servers and the emergence of a competitive marketplace

Many MLSs are licensing RETS servers in addition to the one provided by their MLS vendor. Why are we seeing this trend? Reasons include the desire for:

• Control over speed and reliability of the RETS server
• More sophisticated controls over the data exposed and formats it is exposed in
• Enhanced monitoring and reporting
• Additional security controls
• Use of RETS beyond the listings – accommodating other data types 
• Customization beyond what the MLS vendors are providing
• Not having to change RETS servers and support data recipients when changing MLS vendors

This is not to imply that the MLS vendor RETS servers are "bad" – they sometimes just serve different needs and have different capabilities than third-party RETS servers that one might have to pay for. To be frank, most MLS organizations are not paying the MLS vendors sufficiently for them to add all of the types of features one sees provided by third parties.

The primary players in this space are Bridge Interactive Group (RETS IQ) and Threewide (ListHub and ListExporter). Recently Northern Ohio Regional MLS (NORMLS) has also entered the market. Following are profiles of each company and their products.

Bridge Interactive Group was established in 2004. Bridge’s core goal is providing RETS management and enhancement tools, customized RETS solutions and facilitating data sharing initiatives.  Bridge employees actively participate in many RETS workgroups and they hold a Chairperson seat as well as a seat on the Board of Directors for the Real Estate Standards Organization. 

The RETS IQ Suite of products is designed to put control of data management into the hands of the customer and includes the following components:

• RETS Contact™ is an MLS level RETS filtering, monitoring and management tool that adds further fine-grained control and usage transparency to your current MLS RETS system.
• RETS IQ Cheque™ is an add-on Contact application built to administer billing and account management services for MLS RETS Data Recipients.
• RETS IQ Accelerator™ is a middleware product intended to add reliability, scalability and performance to RETS servers by providing an elastic cloud network of RETS server caches.
• RETS IQ Pulse™ is a RETS server tool for MLSs and Server Providers to monitor their RETS servers. With Pulse, notifications are immediately sent the minute problems with the RETS server are discovered.
• RETS IQ Compose™ is a rich web application for adding and editing listings via RETS 1.x. Compose is almost completely driven by RETS metadata including field layout and validation.

Bridge Interactive Group provides a lot of services around these products, including hosting, setup, data mapping and support.

Some of the key differentiators of the RETS-IQ suite include the ability to create and use data “views” (I’m not going to get technical in this article – but it’s important), the ease of identifying excessive use and getting paid exactly what one should be for that use, the architecture needed for enhanced speed and reliability, and the tools one needs to monitor and assert control over RETS service levels.

Recently Bridge Interactive Group launched RETS IQ Rosetta - a free cooperative RETS Metadata mapping service that Bridge Interactive Group is offering to the RETS community and in particular to RETS client vendors and developers. Its goal is to lower the technical and domain knowledge entry level for new developers in the RETS world, promoting innovation and development of new RETS applications.

The Bridge Interactive Group’s RETS IQ Suite of products now manages MLS listing content for more than 150,000 real estate professionals nationwide via 11 implementations.

Threewide Corporation was founded in 1999 and their ultimate goal is to make possible a single point of data entry that powers the many diverse software systems used by real estate brokers, agents, and administrators in their daily business.

Threewide's primary product is ListHub, which provides a mechanism for easy listing syndication. A broker/agent interface allows for opt-out from listing syndication to specific sites, which Threewide has arranged with over thirty of the most popular national real estate sites and a number of regional portals. ListHub syndicates not only the listing information, but unlimited photos, open house and virtual tour information. MLSs can select where links from the syndicated listings go – choosing from a default redirection to MLS consumer facing site property pages, ListHub property pages, or other property pages of their choice. ListHub offers advanced reporting on the exposure given to the listings - not on the sites they have syndicated to - but via default landing pages for each listing that ListHub provides, assuming the syndication sites link back to the Threewide site. Threewide also tracks and provides reports on the leads sent from their landing pages. Reports are available at MLS, franchise, brokerage, office and agent levels, as well as a branded report for the consumer for their individual listing.

Threewide’s ListExporter enables the MLS staff to selectively package listing data and images and send them to virtually any location in a variety of custom and industry-standard formats. With ListExporter, an MLS can control where the data is being sent, either on demand or through a set schedule. ListSecure works in tandem with ListExporter to minimize the unauthorized use of listing data and images through industry-proven data tags, image watermarks and secure file pickup. Extensive audit trails show when data and images are retrieved and by whom to easily identify the misuse of data.

Some of the key differentiators of the Threewide products are having the most “out of the box” syndication feeds (31 national, plus 16 regional) and the data tagging or “watermarking”.

Threewide’s ListHub product is in use by 190 MLSs.

NORMLS, Ohio’s largest multiple listing service, formed a strategic partnership with Ronin Technologies (http://www.ronintech.org/) to launch a new product called RETS Genie®. NORMLS serves 750 member real estate companies and their 8000+ agents. Ronin Technologies has been a leading provider of RETS solutions for over ten years, was heavily involved in writing the RETS 2.0 specification, and has chaired the RETS compliance workgroup since its inception in 2003.

RETS Genie® was developed to enable the MLS and its members to create data packages for advertising, applications, and custom requests. RETS Genie®is available in two versions, Standard and Deluxe. The Standard Edition enables MLSs to aggregate, integrate and organize any data and photos available through RETS. An MLS can configure reusable IDX filters and define delimited and ‘zip’ compressed file exports of this data for members. Data exports can be run on demand or scheduled to run, then automatically sent via FTP or HTTP to locations configured by the MLS. Users can choose to send either “full” (all the allowed data) or “incremental” (only new or changed data and images) updates to data recipients.   The Deluxe Edition additionally allows MLSs to configure custom database columns and tables for RETS import. Data can be syndicated to Trulia, Yahoo, Zillow, Google in their preferred format, or to any other site using the RETS Syndication schema. Since most syndicators require a unique URL for listings, syndication support includes a unique landing page for property details. The Deluxe Edition also offers the capability to deliver data via email or to a file system on a network. 

RETS Genie®offers add-ons to the Deluxe Edition. The Data Source Module enables integration of other data sources (i.e. member management systems, public records, or tax data) in addition to RETS. The MultiFormat Module enables exports to Excel, HTML, PDF and RSS formats, as well as custom XML formats. The Rules Module facilitates the creation of business rule filters to support VOWS, reciprocity, and third party agreements.

NORMLS offers hosting for RETS Genie®, and partners with Ronin to provide installation, data mapping, training and support.

Some of the key differentiators of NORMLS’ RETS Genie® product are that it easily  incorporates data from non-RETS sources and supports a wide variety of custom output formats including Excel, PDF, HTML, and HTTP and email delivery options.

I am pleased to see a competitive marketplace emerging for RETS products, and has written this short article to recognize a few of the many creative and talented industry colleagues that are blazing this trail for our industry.

The RETS standard has come a long way over the past ten years, but the promise of RETS will only be realized through product development that leverages the standard. In much the same way that concepts of hypertext first demonstrated in the 1960s did not achieve their fruition until the 1990s when the Mosaic web browser application blazed the trail for the World Wide Web, it will take products – killer apps – to really bring RETS into its own golden age and create opportunities to enhance the industry with new kinds of information-hungry applications that are only now being imagined.

 

A standard, like software, is never complete - it just goes on to the next version. But, the RETS standard has been going on for ten years - I think it's time to take stock about how far it has come and talk about where it could or should go.

If you're an MLS executive or an agent, why should YOU care about this RETS stuff moving forward quickly? Doesn't it already facilitate your data feeds, syndications and MLS collaborations? Think about this - recently I was negotiating an MLS contract and wanted to know what data, created by my client and their subscribers could be moved from the old system to the new one:

• Listings? Of course.
• Roster? Of course.
• Contacts? That might be possible with custom work, not using RETS.
• Prospects? No.
• Saved searches? No.
• Custom columnar reports? No.
• Custom search screens? No.

More than half of the items that we wanted to move from system to system using RETS couldn't be done. Each one we couldn't move would mean pain (time/effort) for the MLS subscribers as they had to re-create the data in the new system manually. This isn't just relevant to moving from one system to another - it's relevant to offering a parallel system - or multiple MLS front-ends - or any number of potential MLS add-on products. And I could make similar points about the difficulty of moving between other real estate systems, such as association management and transaction management systems.

This difficulty in moving systems or adding systems (some would call this the "friction  of the transaction") does not encourage the most competitive marketplace for real estate technology - and this is bad for my MLS and Association executive clients and bad for their subscribers and members. Face it - most of the people driving the data standards are more established vendors - and they do not have incentive to improve standards to make it easier to move to another system. So, if you want the standards completed to the point where you can move *all* of your data from system to system, increasing real estate application innovation, increasing competition in the technology marketplace and making transitions between systems easier for your members - YOU have got to work with your peers to drive the standard forward.

If you want to contact the Chair of the RESO Outreach and Education committee, email Kristen Carr (kcarr@big-llc.com).

The Fall RESO/RETS Conference is September 23rd - 25th, 2009 in Chicago, IL - more information is available here: http://www.rets.org/meeting

 

I think there's a lot more information needed out there for the RETS standard. We need something like an O'Reilly book for all the real estate application developers. I think the biggest problem with RETS education today is when an MLS executive says "Sure, let's go all RETS" and every broker tech, consultant and local vendor throws their hands up in frustration after going to rets.org and finding little to support the steep RETS learning curve.

How do we get this book written? I'm not sure - books take a lot of time and money to write - I don't think I'm up to the task myself.

 

 

Check this out! RETS IQ RETS Library™ is a Java API that allows simple access to RETS servers. The API is designed to allow developers to connect to RETS servers and execute searches, photo downloads, metadata requests and updates without having to deal with the nuts and bolts of the RETS protocol.

Mind the license.

 

During the NAR meetings I had the opportunity and privilege to help staff the RETS booth, and one booth visitor in particular piqued my interest. The visitor was a developer for one of the major broker back-office applications and expressed that he had integrated with a number of MLSs and was getting more comfortable with RETS, but he was still perhaps a bit fuzzy on RETS as one of those "MLS things".

It really brought home for me how MLS-centric a lot of the RETS effort has been, since RETS really should be providing a tremendous efficiency benefit to brokers and agents beyond the area of the MLS system, moving data efficiently and error-free from forms packages to broker systems and between all of the various broker and agent systems where real estate professionals are performing duplicate data entry.

When I expressed those thoughts, I think it was a real "Ah hah!" moment for that developer. I hope that he and other non-MLS developers continue to become more actively engaged in the RETS effort, and that RETS brings efficiencies to all of the systems in use by real estate professionals - not just MLS.

The message I think we need to get out there?
RETS is not just an "MLS Thing"
RETS is a "Broker Software Thing"
RETS is an "Agent Software Thing"

Background

Real estate professionals are using more systems and applications than ever, and they don’t want to have to log into each one separately. The inconvenience and inefficiency of multiple logins are exacerbated when users have to go back and forth between one system and another. As a result, system providers such as MLSs, transaction management software, larger brokerages and other real estate application vendors have moved to provide login integration of commonly used systems as a convenience for the users. This is referred to as Single Sign-On (SSO).

In order to improve the security and efficiency of implementing SSO, it was critical that the industry’s major software vendors agree on a standard method for doing so. This is why in August of 2005 Clareity Consulting published a free white paper, “The Convenience and Security of Single Sign-On” and subsequently facilitated two meetings, in December 2005 and March 2006, where many of the major MLS and transaction management vendors met to discuss how to achieve SSO in the real estate industry. MLS vendor attendees included Rob Overman, CTO of Fidelity MLS, Dan Mills, CTO of Interealty (First American MLS), Chip McAvoy, CTO at First American Residential Group (MarketLinx), Mike Wurzer, CEO/CTO of FBS, Carlos Grass, CEO of Stratus Data Systems, Brian de Shepper, CEO of Tarasoft, John Hensley, CTO of Homeseekers, and Brett Weiner, CTO/EVP of Rapattoni MLS.  It was critical to achieve cross MLS vendor support and cooperation, because while MLS may not be the center of the universe, imagine the decreased value of SSO without MLS system integration.  In addition to the key MLS software leaders, Cendant, CREA (Canadian Real Estate Association), OSCRE (the international real estate standards group), and other software companies in the Transaction Management, CRM, neighborhood information and online mapping areas also attended and contributed to these meetings.  Paul Stusiak, technical co-chair of the RETS group, attended and represented the RETS working group.

This diverse and talented group of technical leaders in the real estate industry came to a consensus that:

·         Using an open standard such as SAML 2.0 will allow the vendors to cooperate– making it more efficient to implement SSO securely.

·         There was no agreement on, or even serious consideration to, using a proprietary product to accomplish SSO.

·         No single entity or group should try to monetize SSO or control it by making it proprietary because this will create conflict and control issues and be a non-starter for many organizations and likely to slow adoption of free, open standards based SSO.  (Microsoft Passport was discussed as an example of why a controlling or proprietary approach is likely to fail.  Today, even Microsoft is backing open identity management and SSO standards as Bill Gates mentioned during his keynote address at the RSA Conference in February 2007)

·         Clareity and the vendor group should work with the NATIONAL ASSOCIATION OF REALTORS® (NAR) to incorporate support for SAML into the existing standard for real estate software vendor cooperation – the Real Estate Transaction Standard (RETS).

SSO via RETS

In November of 2006, NAR provided a grant to Clareity Security to build an open source reference implementation for SAML. An initial implementation has been demonstrated to NAR staff, and Clareity Security will be providing a complete, documented SAML standards toolkit to the industry for free inclusion into real estate software products in May 2007. Though Clareity Security is creating this software, Clareity will not own it,  but rather it will be owned by the real estate technology community as part of the RETS standard.  Again, if anyone were to own or control a method of Single Sign-On interaction, most vendors will not agree to use it – and vendor cooperation is at the heart of the success of Single Sign-On in our industry.

Unfortunately, there are two or three organizations in our industry that are attempting to monetize or establish control over Single Sign-On via use of proprietary products and portal technologies.

Those offering proprietary solutions are troubling: they may actually claim to be using open standards, but that is not the same as providing an open platform using those same standards. This is equivalent to Microsoft’s use of the XML standard in their Word product – Word is still a proprietary document format that gets in the way of collaboration with those not using the Microsoft product. Again, providing a common, open, and royalty free, vendor-independent platform via RETS is what is going to allow vendors to cooperate and allow for Single Sign-On adoption.

Those trying to control Single Sign-On using a portal are even more troubling and the issues additional to those described above are myriad:

First, the portal concept assumes that the portal manager is going to maintain identity (and possibly role) for everyone using all of the systems behind it. Imagine a state or local association, or its vendor, managing identity not just for association members, but for all of the non-members including MLS affiliates, unlicensed assistants and virtual assistants, brokerage staff, and potentially even for people in entirely separate industries that service ours – such as title and mortgage. This is a nightmare and the reason that the “federation” of identity outside of the enterprise/corporate environment is extremely difficult and is not available today in similar industries such as insurance, financial services, mortgage banking, or health care.  In fact, several identity portal attempts in those industries have failed miserably. The real estate industry will not be any simpler and presents a huge challenge because it is about as entrepreneurial, independent, and highly fragmented as you can get.  

Second, changing human behavior to go to a different site, other than the web application where people normally go to start their daily work, is difficult and inefficient, if not impossible. Agents (along with brokerage staff, mortgage, title, appraisers, attorneys and others) need and want to start their day on the system of their choice – their starting point can not be dictated as it can be in a corporate environment.  For example, one simply can’t tell a broker that if they want their multi-million dollar Intranet or back-office system to have Single Sign-On functionality, their agents, managers, and staff would need to log in first through an identity portal they don’t control.  This is a non-starter and larger brokers and franchises react violently to this type of intrusion in their business.  Likewise, most MLS executives or CRM desktop application vendors rightfully believe their system should be the starting point for their customers to originate SSO.  Clareity believes the broker, MLS executive, and software vendor are all correct and that as an industry we need to make SSO work for all these stakeholders if we want them to adopt SSO.

Third, it has been suggested that portals could be supported by advertisements (or user fees) and provide profits or revenue sharing to the portal operators. This would be another deal breaker for brokers, because channel conflict with advertisers is inevitable. For example, can Lending Tree, Bank of America, and HomeGain advertise on the “Realtor” portal or can we exclude these kind of advertisers?  But if there are no ads, then where does the revenue come from to support the portal and deliver profits to the owner/operators? User fees?  SSO that costs money can be compared to a toll road.  How could anyone seek to build a toll road if there were many free roads that take users to the same place? 

 Fourth, a centralized portal is vulnerable to a Distributed Denial of Service security attack, or may become unavailable for other technical reasons. A large portal is a tempting target, even Microsoft could not keep their Passport centralized service up enough of the time to serve its function acceptably - and the last thing anyone would want is for all the systems to be inaccessible because one system – the portal - was unavailable. The expense to create portals that “will never go down” and the associated risks are unacceptable and needless. The SAML standard will leverage the less vulnerable “web of trust” which underlies the way SSO is achieved in our industry already today.

 Computer Associates (CA) published a White Paper in January 2007 that explains the business value of SSO.  This paper made it clear that trust cannot be mandated, but rather is established by companies that want to do business together or provide a convenience for their employees or customers.  Following is an excerpt from that paper explaining why SSO and federation of a user’s identity are most likely to succeed in non-centralized fashion:

 “Given the intense focus on personal privacy and control of digital identities, the existing identity infrastructures that can be found in today’s organizations and the high-value of customer information that is often housed within them, it is virtually impossible to expect organizations to collaborate on creating and maintaining a universal, shared point of identity information.  Requiring organizations to first merge and centrally manage their user’s digital identities as a prerequisite to federating their applications for use by those users, is a non-starter.  This is one of the basic requirements driving federation standards and why the term “federation” (implying collaboration between loosely coupled sovereign organizations) is used in the first place.”

 For a free PDF copy of the White Paper from Computer Associates, visit www.ca.com (http://www3.ca.com/solutions/Collateral.aspx?CID=66744&ID=271)

 Single Sign-On done properly, through RETS with SAML extensions, may enable the creation of portals, but does not require there to be a portal. There can and should be hundreds of SSO portals, or as we prefer to call them, “entry points” or “authentication points” nationwide – as many as appropriate to the types of users participating in the Single Sign-On initiative in any market.  MLS systems already serve as de facto portals (or entry points) and many MLSs already serve as the authentication point for Association systems, public records, TMS, CRM, and broker systems, and provide SSO today.  Now, it’s simply time for the industry to do it better by adopting and standardizing SSO via RETS.

Conclusion

Clareity Consulting initiated open discussion of Single Sign-On in 2005 because our industry was inefficiently implementing SSO in a variety of ways and not providing it as securely as it should.  The industry vendors needed coordination to make SSO simple, standardized, secure, efficient and scalable.  Today, many of the industry’s leading technology providers have already agreed to cooperate around Single Sign-On, freely, through use of free and open-standards SSO (SAML) incorporated into the industry’s standard, RETS. Clareity Security will be providing a complete, documented SAML standards toolkit to the industry for free inclusion into real estate software products in May 2007. Though Clareity Security is creating this software, this is not a product and Clareity will not own it, but rather it will be owned by the entire real estate technology community. With your support, SAML standards will become a valuable extension to the RETS standard. By using RETS, the use of proprietary, divisive, issue-laden and potentially expensive products can be avoided, and SSO can be achieved as securely and efficiently as possible. Finally, SSO via RETS will help drive the widespread adoption of RETS 2.0 and everyone in the industry will benefit from greater standards adoption. 
 

The technical details about SSO and RETS are described in the Appendix that follows:

Appendix – The Technical Details about SSO and RETS

SSO in its basic form just means that a service provider (such as a RETS server) will trust authentication credentials provided using the SAML standard by an identity provider (such as another MLS, a strong authentication server, or another RETS server). For RETS to leverage the SAML SSO standard, RETS 2.0 must support SAML tokens. Please note that when we use the term ‘token’ in this document, we’re not talking about some sort of physical security token, but something else entirely, a security ticket that is part of the SAML standard. Please note that SSO is a concept within a single application (such as a web browser) - It does not necessarily mean that a user can move from a web browser authenticated to the MLS system to a separate RETS client application without re-authenticating within the context of the RETS client.

 

With that groundwork laid, consider the scenario where the MLS system has two RETS servers. One allows queries only for property data; the other server handles all other classes of data (agent, history, photos). To pull meaningful data, the RETS client must be able to talk to both. So the RETS Client (RC) starts a transaction with server A sending a SAML authentication token containing its login credentials. At a given point in time, the RC needs to get a photo to go with a listing. So it sends a request to server A asking for a SAML SSO token and Server A replies with that token. The RC then sends this SSO token to server B during the login transaction. Server B validates the SSO token by one of the available methods (message integrity / signing / server to server verification). Server B then lets the RC make its queries for photos.

 

So what in this is above and beyond the existing RETS standard? The first is when the RC asks for an SSO token from server A. Server A must be able to understand that request, it's not really a RETS request, it's a SAML request. It might be that we have to add something to the RETS 2.0 standard to spell this out. Additionally, server A must be able to generate those SSO tokens itself, or via a trusted third party. Server B must be able to recognize the SSO token and take the appropriate action to validate it, whether that is verifying the signature, or by checking with the issuing server.

 

One could also have the scenario where there is an independent identity provider (such as an MLS) could generate the SSO tokens once a successful authentication happens. So an MLS could have a RETS 2.0 interface for authentication only. The real RETS servers would just have to be able to accept and validate the SSO tokens.

 

In summary, to achieve SSO via RETS, the RETS servers will have to be able to understand SAML tokens, and the OASIS standards to a certain extent. Your support of the inclusion of SSO standards extensions to RETS, along with the SAML toolkit that NAR's CRT will be freely distributing soon, will help achieve that goal.