Larry Romito spoke to the Michigan and Ohio AEs last week on the topic of quality of service, and how we as association managers can evaluate the service we offer our members.  Well, wait a minute—“offer” isn’t the right word.  There’s a whole lot of difference between what we say we’re going to do, and the actual level of support our members receive from the association office.

That’s Romito’s point, of course.  How do we as AE’s actually KNOW what response our members receive when they call or email us?  And more importantly, what do we do with the responses we do receive: how do we use that input to modify and adjust our service product?

I’ve always been a proponent of measuring results as a means of keeping self-deception out of the management decision.  “Yes,” says the Board of Directors, “that was a good general membership meeting last week. We had 67 people there, the food was good, and everybody is smiling in the photos we took.”

“Of COURSE they’re smiling in the photos,” says the voice of the Devil Advocate.  “You told people to say ‘cheese’ when you snapped the picture.  Can you say ‘cheese’ with downturned lips???”   And then Emily Evaluation says, “Of the 67 attendees, 40 were affiliates who are paid to be there.  Twelve were the members of the Board of Directors, for whom the Association buys lunch.  Four were staff members.  That leaves 11 volunteers, and one of those was the wife of the President.  That leaves 10 members out of 2,711 total Realtor members.

But the food WAS good.”

There’s another evaluation method —the financial analysis, or “Business Plan” method.  It assumes that members will pay with their pocketbooks for voluntary services and products (forget all the offerings for which they have no choice about payment—that’s another story).  My favorite illustration of this is one association’s summer outing.  It was always a fun party—golf, steaks, music, beer, swimming and trail rides.  Everybody wore casual clothes and a lot of smiley pictures testify to the fact that everyone was saying ‘cheese’.   And then the expenses were totaled up and divided by the number of attendees. The whole shebang cost $57 per attendee, and again, only 30% of the attendees were dues-paying members.    The cost per Realtor was over $200. 

Did it matter where the funds came from? That much of it was contributed by affiliate member deep pockets?  Not really.  This was an event which didn’t interest members, even though a $200 value only cost them $10 per ticket—and that disinterest is an important indicator, no matter who pays for the function.

Another evaluation method might be to look at the organizational mission statement and see if the function even qualifies.  If, for instance, the mission of the local association is “To ensure that our members have the support necessary for their professional success”, can one even pretend that the membership meeting or the summer outing described above are ‘necessary for professional successes?  Apparently the members don’t think so.

“Hey, wait!” you say.  “I evaluate our association performance all the time.  Why, we always send out questionnaires.  Just last week we had an education session on Social Media.  Lots of people came, and the evaluation sheets we handed out were very positive.”

OK, let’s look at those evaluations.  Here are some of the questions:

            “Was the facility adequate?  Could you see the screen?  How was the room temperature? How was your check-in procedure?  Were the handouts helpful?  Was the speaker interesting?  Did he/she encourage class participation?  Was the presentation too long?  Too short?  Could you hear?   How was the parking?”

Aha!  You say your education director is also the meeting planner?  How did I guess?  Well, those questions were not written by those serious about education.   In the blog “Acronym”, Lisa Junker says, “Right now most associations and most directors of education see the finish line as the end of the event or the end of the webinar….But all we've really done is get people trained for the race; the real race is back there in the workplace.”

The important questions have to do with this question: how applicable was the information you heard to what you do in your job?  Or (asked at a later date) “what percentage of the information you gained do you remember?  What information from the presentation are you using today?”  Certainly retention of information and practical application of methods are important goals for our training efforts.  And if we are gauging effectiveness based on those values, conveying those value measurements in advance to course writers and presenters will take your program a long way to success.

“Yes”, you can say, “I know our members will be interested in your high-tech demo of flip cameras and cell phone social networking.  And that live broadcast you can do from your iPhone is awesome.  I will, of course, be asking them to evaluate how they will use this shiny stuff selling real estate—perhaps you’ll be able to help them along with some practical applications.”

One of the most interesting jobs I remember in my own career was taking the generic information on non-profit association management and applying it to the specific profile of the Realtor organization.  The subject areas of, say, advocacy or strategic planning for non-profits need considerable modification to be of real use to a membership based trade organization of competitive independent contractors. Perhaps suggesting a member practitioner as a liaison to your speakers might be a good plan to ensure that programs realistically address member needs.

And a final thought about evaluation that Mr. Romito left with me—just as I was patting myself on the back for the tricky little techniques I’d implemented to get member feedback, he said “Don’t always trust immediate response.  That kind of feedback is subject to the ‘halo effect’.  You know, right after you please someone, ask him how he feels about you.  A good way to always get positive feedback.  But it isn’t very credible.”

So, let me try this.  What did you discover in this blog post that might apply to the day-to-day management of your association?  Did you like that idea of a member/expert liaison to your education presenters?  Are you going to try a financial evaluation of cost per benefitting member for your products and services?  Were these helpful ideas?

How do you feel about me?