Sustainable Research (2/26)
Background
The world is on the brink of a massive social & economic upheaval in that the western style civilisation that has developed since the Industrial revolution needs to be re-invented. We can no longer assume that we will have limitless energy – we need sustainable energy sources; we can no longer build our way of life on the assumption of limitless resources – we have to develop sustainable resources; we can no longer drive business in the profit driven ways of the past – we need sustainable business; we can no longer assume that consumers will freely give of their time – to researchers, honestly. We need to develop sustainable research.
On the 2nd June 2010 Ms Gunilla Broadbent, President of ESOMAR touched on this subject when she said that ESOMAR seeks to gather insight and promote debate on how market research can lead the way to a more ethical, responsible and creative society’! Her conclusion emphasised that market research must move from the paradigm of ‘The age of Questioning’ to that of ‘The age of Listening’.
The rest of the world has largely discarded traditional survey methods and have moved to using online methods and ESOMAR has provided guidelines on the ethical conduct of online market research in the form of ‘26 Questions to help research buyers of online samples’. These 26 can be viewed at www.panelservicesafrica.co.za
We'll be examining each of these questions from the African context point of view and will discuss the implications of not getting the correct answers.
Question Two of 26
Please describe and explain the types of source(s) for the online sample that you provide (are these databases, actively managed panels, direct marketing lists, web intercept sampling, river sampling or other?)‘The description of the type of source a provider uses for delivering an online sample might provide insight into the quality of the sample. An actively managed panel is one that contains only active panel members – see question 11. Note that not all online samples are based on online access panels’
Any survey can be undertaken as an online survey depending on what one means by the term ‘online survey’. The factors that have to be taken into account are Objectives of the survey, Nature of the universe, Technology and Methodology.
To examine this topic in more detail the opening sentence above needs to be explored and motivated. But first we need a definition. ‘Online survey’ is taken to be a survey whereby the participant is connected online to the host computer during the course of the interview. A generous interpretation would include the involvement of an interviewer. In this case, it would include CAPI and CATI and, because samples for these methodologies follow more traditional market research sampling procedures that are outside the scope of ESOMAR’s intent, we therefore impose a non-human intervention requirement. The definition then becomes ‘a survey whereby the participant is connected online to the host computer during the course of the interview, in the absence of other human intervention’.
Technology
Because the absence of human intervention is a defining condition, so the overriding requirement is the application of technology. Technology that enables the collection of survey data. Technology is developing very fast and there are now monitoring algorithms that collect the equivalent of survey data and seamlessly activate or inform the business. Amazon’s interactive predictive systems are well-cited examples. These model each person’s preferences and behaviour against all shoppers, therefore enabling the business to customise its offering to each individual. It’s a moot point whether these systems are market research surveys. The important point is there is no particular reason why market research surveys cannot be automated to this extent. The issue about how the sample is drawn becomes irrelevant because effectively a census of the universe is surveyed. The question shifts to whether the universe is relevant. In the above example the universe is made up of all people who have bought anything from Amazon.
Objectives
So, a research buyer who is asking these questions of a potential supplier of online research needs to be very clear about the objectives of the study. This is not a trivial issue.
For example, if a business is tracking marketing measures in order to determine change, then it would be more efficient to select a sample, then continue to interview that sample. The reason is that one is eliminating sampling error from each subsequent wave. So, if the objective is to track in order to detect change, the sample should not change from the original and the suppliers’ efforts should be focused on managing compliance of the original sample. This is what is done in clinical trials methodology in medical science. This kind of study is starting to become more popular in the form of research communities, extended focus groups, short term panels and call back interviews. In the case of an objective such as this, the sample could be drawn based on the known experience of compliance, honesty and creativity.
It is hard to conceive of any objective that cannot be achieved using online technology, depending on the availability of a framework that can realistically deliver a meaningful sample.
Methodology
Turning now to the implicit intent of this ESOMAR question, which is – what is the nature of the universe resource?
Databases
Just the word ‘databases’ is not helpful. Every electronic file can be classified as a database – including a record of your wine collection in an Excel spreadsheet. Current legislation worldwide and in particular, in South Africa, has gone a long way to help sort out the confusion. For our purposes we will restrict our discussion to electronic files containing peoples’ names, contact details plus any other information that can be linked to them.
The legislation focuses on source and usage.
Source relates to how the lists where created, collected or generated. So, for example a client list of a company can legitimately be called their proprietary property.
However, legislation intends that there needs to be consent by every person about the fact and content of the information that is stored. It will become illegal to trade people’s contact details without their active agreement. It is quite common practice for a company to have a list of clients and/or suppliers with whom they may have had dealings.
Usage of lists will come under increasing scrutiny as the guiding principle in that lists may only be used for the purposes for which they were created or collected. So, a company can only use its client list in the regular conduct of business, the purpose for which the lists were collected. So, a client list should not be used for market research purposes, unless each person has agreed and has knowledge of what information the company has in its list about them. In principle, the legislation will treat such ‘surveys’ as Spam.
Direct marketing lists and the companies who trade in them are in very real danger if they continue with past practices. These are lists that may have been collected, for example, as part of credit applications. They are then traded to direct marketing operators for selling products that people may, or may not be interested or even capable of buying. These practices will because illegal in some parts of the world. The way that these lists will develop is as voluntarily maintained systems – like Facebook, LinkedIn and all the other social networking systems that are emerging.
There is a further reason to avoid using any kind of list for which the user has no identification verification. Robots and avatars have been developed that can complete hundreds of surveys mechanically. One cannot know whether your competitors have hijacked your survey and are sending disinformation.
An important implication for market research as a consequence of this trend in legislation regarding the privacy of information is that the notion of conducting pure random sampling as a sampling methodology will become increasingly quaint.
So, what kinds of sampling can be undertaken sustainably?
Actively managed panels have been used for years in the marketing, scientific and medical fields. In the marketing field, think of the panels that are set up to provide TV viewership, in the form of Peoplemeters. Think of the panels of retail outlets that companies like Nielsen use to measure trading activity. In medical research think of clinical trials where life-threatening drugs are tested using carefully managed groups of people. It’s only in recent years, with the advent of the Internet and lower costs of sophisticated data warehousing systems, that marketing research has been able to efficiently and inexpensively manage the huge amounts of information generated from large numbers of people. Well run panels, well designed and managed, enable researchers to automatically quota control samples for online research.
Web intercept sampling is the newest form of sampling methodology that technology has ‘blessed’ the information industry with. This involves procedures like ‘web scraping’, which involves matching behaviour and information surreptitiously from web journeys to and amongst websites and creating avatars which have unique, but ‘real’ browsing behaviour. As with all new processes, questions will arise and need to be dealt with.
River sampling, known amongst many researchers as ‘stream’ sampling, where one samples from a stream of activity to reduce the amount of data processing and/or collection that has to be done. So, for example, if there is a queue to enter a stadium, one could sample along the length of the queue. In the case of sampling the transactions in a supermarket, one could sample every nth transaction from different checkouts in different stores.
A final word on sampling. The only reason for sampling is that one cannot conduct or connect to a universe. So, we model reality by drawing a sample. A sample needs to be a reflection of reality of the universe. So, for example,
This chart shows the number of heavy purchasers of three products and the number of heavy purchasers who are also connected to the Internet. This suggests that the marketers of these 3 products would have a good chance of reflecting reality by drawing online research samples from web based research communities.
Mike Broom
Panel Services Africa
July 2010