A Business Research Lab Tip
When we receive a research project, we review several items before determining the appropriate methodology. These include:
- Information needs and objectives,
- Research objectives,
- Target audience, including any idiosyncrasies which might make them easier to reach by one method or another,
- Budget constraints,
- Time constraints,
- Complexity of data to be gathered,
- Projectability to the target population.
Careful consideration of the above usually will identify one methodology which clearly is superior for the particular research project.
There has been much ballyhoo of late about Web surveys. While we clearly believe that they have their place, and a growing one at that, in the arsenal of research methodologies, we believe it would be folly to launch into a Web survey without first going through the aforementioned steps. That said, let's examine the pluses and minuses of Web surveys, in the context of quality research design.
Disadvantages are fewer in number, but their severity (the first one in particular), is such that we feel they should be listed first.
Projectability
More and more people are connected to and using the Internet. Just a few years ago, Internet surveys
only could be used in rare cases, as a relatively small proportion of the population used the
World Wide Web. This was true in the United States, and even truer outside the US. This
problem was exacerbated by the fact that the population of Web surfers was not representative
of the population at large.
This is much less of a problem than it used to be. However, it is important to recognize that
web surfers tend to be younger, male, better educated than
average, and have higher-than-average incomes. Thus the main challenge in conducting a
Web survey is in obtaining a projectable sample. Stratified sampling methods can adjust for
differences between the population being measured and an Internet Sample in some circumstances.
Target Audience
It is very difficult to reach certain target audiences with Web surveys. For example,
grandmothers and lower-income people, at this time, be difficult to reach via the Web.
Speed
Web surveys can be conducted much more quickly that mail surveys, which take several
weeks for returns to come in. Up-front design time is comparable to that of programming
a CATI (computer aided telephone interviewing) survey to be administered by telephone.
Data entry time (on the part of the research firm) is negligible, as it is with CATI.
Data collection time is faster than CATI.
Data-entry accuracy
Data entry errors (on the part of the research firm) are non-existent.
Security
Using a secure server (the same technology used to process secure credit card transactions),
security is not an issue. Anyone who attempts to intercept a transmission made to a secure server
will not be able to understand the information they see, as it is encoded.
With written surveys, respondents may question whether their handwriting will give a clue to their
identities. This issue does not exist for Internet surveys.
Control
Internet surveys enable much more control than do paper surveys. A few examples of this include:
- It is possible to make certain questions required.
- Algorithms can be written to ensure that in a ranking question, no rank is given more than once.
- Through the use of cookies, people can be prevented from taking a survey multiple times.
Cost
For medium to large size surveys, Web surveys will have a cost advantage over telephone and
mail. It remains to be seen whether the prices of banner ads will increase to the point
where this cost advantage remains.
Ability to gather Complex Data
It is possible to gather relatively large amounts of data (if the survey is well-written),
and to use complex skip patterns (the order of and particular questions changing in
depending upon answers to previous questions.) Telephone also shares the advantage of
allowing complex skip patterns, but it is not advisable for a telephone interview to be
more than 10 or 15 minutes in length.
The answer to this question will change with time. At the present time, here are the
circumstances under which we would recommend or consider a Web Survey.
When part of the target population definition includes "people who surf the Web."
Web surveys would be the preferred methodology of reaching people who surf the
Web. A good example of a research objective which would steer us in the direction of a
Web survey would be "find which of several Web page layouts is preferred, and identify
the strengths and weaknesses of each layout."
When the target population has many characteristics in common with people who
surf the Web.
For instance, if you were trying to reach upscale, young males with high incomes,
you would want to give serious consideration to using a Web survey. This does not
mean that a Web survey is automatically the preferred methodology -- that determination
will depend upon the specific information needs and research objectives. It may still
be necessary to "weight" the results by various demographic criteria to make
the results more representative of the target population.
When there is a high degree of certainty that the answers given by Web surfers would not
differ substantially from those given by non-surfers.
Examples of this might include finding what proportion of people are left-handed.
Great care must be given to examine possible correlations between "being on
the web," and the characteristics being measured. For instance, it might,
at first blush, make sense to say that one could measure the average weight of
people who surf the web, and project it to the general population. However, because
Web surfers tend to be better educated than the general population, and better
educated people tend to be lower in weight than average (controlling for gender),
it would not be a good idea to use a Web survey for this purpose.
In a web survey, the target audience must be drawn to the web survey page. There are two primary means of doing this:
- Banner ads,
- E-Mail.
Banner ads are, in most circumstances, the preferred method. For a general web population sample, an ad can be placed at the top of one or more search engines. For a specific web population, ads in a search engine can be programmed to appear only when certain key words are typed. For example, in a survey of potential computer purchasers, you might want to have the ads appear whenever the name of any major PC company is typed in the search engine, or when any combination of the words "PC prices," "personal computer prices," "PC performance," etc. is typed. Great care must be taken in the selection of keywords.
Banner ads also can be placed in an array of non-search engine sites. In a survey of Web
news viewers, ads could be placed in an array of news sites.
The use of e-mail to attract surfers to a web survey should be considered only when people would not consider the e-mail to be junk mail. This is a rare circumstance. However, there are certain conditions which would render e-mail superior to banner ads in attracting people to the survey. This would include a customer satisfaction survey to be conducted for an ISP (Internet Services Provider), or an employee survey to be conducted among employees of a company who conduct most communications over the internet and give all employees internet access (assuming that the dissemination of information is web-page, as well as e-mail based.)
Speed of page loading should be minimized. This means graphics, if any, should be kept to a minimum.
If possible, limit each person who takes the survey to complete it only once.
It sometimes is desirable to allow people to return to the survey and
complete it later if they stop it before finishing. If this is done, they should be allowed
to pick up at the point where they stopped.
Collect demographic information to enable the data to be weighted, and to determine
the degree to which you are reaching the target audience with the survey.
Do you own or manage a small business, or a department of a large business? Check out our special E-Value2003 Employee Satisfaction Survey, specially designed and priced for businesses with 1000 or fewer employees.
|
