Tony Nix AASA, MBA Director, Solutions Marketing & Consulting

In a recent national quantitative research project we knew that the targeted respondents were technical people who spent most of their day out in the field and therefore would be difficult to reach.  Previous studies had been by telephone (CATI), and the client wanted a true market perspective so online was not considered. (Note that the next time the study is executed an online survey could be considered because the market metrics are now known). The client had a data base in which he had previously isolated the likely ANZSIC codes that were more likely to use his services. However he was a little concerned that the data base was now 3 years old. Wisely the client agreed to pilot the project because he was unsure of the validity or currency of the data base.

The advantages of a pilot

Now a pilot is where we design the questionnaire but only complete a small proportion of the interviews, perhaps 20 out of a desired total of 600. The pilot allows us to test the efficacy of the questionnaire, the interview length and the hit rate or number of interviews completed per hour. Usually the questionnaire has been well thought through and requires no changes but the other variables often necessitate some project changes.

In this particular case after completing 11 questionnaires, we were surprised to see the hit rate at 1 completed interview in almost 75 calls, or 1 ¾ hours interview time per completed interview. This was clearly unacceptable and unless improvements could be made the cost blow-out would be unmanageable.

The problem was firstly that the data base was somewhat out-of-date with almost a quarter of calls being disconnected, no answer or wrong number. A further 15% were not qualified as they didn’t use the required services. Most of the remainder were out in the field and either not available or agreed to a call-back. Another issue was evident in that the time to make an uncompleted call was extending way beyond the normal 1 minute due to the time taken to identify the target person and arrange the call back.

Based on the results of the pilot the following action was taken:

1. A new more current data base based on specific ANZSIC codes was purchased
2. The data base was cleaned by eliminating company names with words that suggested they would not qualify e.g. electrical, electricians electronic etc
3. The repeat unanswered phone calls were limited to a maximum of 3 calls
4. The CATI interviewers were briefed to limit the time taken to establish the qualified respondent

The hit rate improved dramatically from 1 in 75 calls made to 1 in 33, then gradually to 1 in 25 calls as the research progressed and we limited the call backs. Note that the key here is to continually learn from the data analytics and adjust the processes to improve the hit rate.

So what monetary difference can these types of improvement make?

Click to Enlarge

Note: field costs exclude project management, analysis & reporting costs

The 1st action – tripling the hit rate from 1 in 75 to 1 in 25 telephone calls has in this example sliced 59% off the original field costs.  Lesson to be learned – start with a good data base!

The 2nd action – improving the connect/call rate from 2 minutes to 1 minute has reduced the field costs by a further 33%, to 28% of the starting situation. Now this is usually harder to do but the lesson is …start with the right contact in the data base and continually monitor the field call statistics and you have a good chance of success

The 3rd action – reducing the interview by 2 mins yields only a tiny saving and is probably not worthwhile given critical information may not be collected

The 4th action – the best possible result

By leaving the interview at 12 minutes and further increasing the hit rate to 1 in 10, we produce the best possible result, a field cost of $19,238 only 18% of the starting situation cost. To achieve this result we would have to have the best possible data base - very up-to-date, with the right contact, easily reached and generally willing to respond to the interview.

Why clients often neglect fieldwork costs

Usually market research agencies do not disclose fieldwork statistics as it exposes them to too much scrutiny. Clients neglect to analyse the fieldwork because they have no knowledge of the metrics.

We at Solutions Marketing & Consulting have developed our own model to monitor the fieldwork progress so we can take effective action to minimise client costs. We impose no margin on the field costs and only charge for project management, analysis and reporting.

The savings in fieldwork costs can mean more left over for deeper, better analysis and still yield significant savings for the client

To find out more about how Solutions marketing can manage your research project for maximum effectiveness and still save you money….