36 Questions to Ask Before Commissioning Neuroscience Research
How do you make sure you're choosing the correct neuroscience research tools and providers for your research purpose? An article by ESOMAR (the European Society for Opinion and Marketing Research) outlines 36 questions to help buyers commission neuroscience research.
COMPANY PROFILE
1. What experience does your company have with providing market research using neuroscience techniques?
The answer to this question might help you form an opinion about the relevant experience of the provider. How many years has the company been providing market research using neuroscience techniques? How many projects have they carried out? How many participants/respondents have been involved?
2. What experience/education does your team have with using neuroscience techniques in general? What is their experience with using these techniques for market research purposes?
Use this question to establish how many staff with the relevant academic qualifications this company has. What are their academic qualifications? Have they published any relevant papers/articles? What is their experience with neuroscience in general, and with neuroscience for market research in particular? What is their role within the company – are they primarily involved with R&D, or do they participate in a hands-on way with market research? Do they have the support of qualified and experienced market research professionals on projects?
3. What types of industries does your company focus on (e.g. advertising, FMCG, etc.)?
Use this question to find out whether the provider has experience in a sector relevant to you. Ask whether any existing clients are willing to provide references/testimonials. Are there case histories that can be shared, or papers in the public domain?
4. What experience does your company have in providing market research in general?
Again, this question should be answered by number of years and/or number of projects. Note that if this company has been set up solely to provide market research using neuroscience techniques, then the answer will be the same as for Q1. If this is a general market research company which has moved into neuroscience, then the total experience of the company is relevant.
5. What experience/education does your team have with market research in general?
Here you should focus on the market research expertise within the company. What market research qualifications and experience do the key personnel working on projects have? How long have they been involved in market research? Have they presented any relevant papers or published articles? Are they members of ESOMAR or a relevant local market research organisations? What market research codes of conduct do they adhere to?
TECHNIQUE PORTFOLIO
6. Which neuroscience techniques does your company use for market research? (*See annex for a list of techniques with an explanation of each technique.
Advances in bio-/neurometrics provide a wide array of measurement tools which are evolving to focus on varying aspects of consumer response to marketing stimuli. Some of the techniques include EEG, GSR, facial coding, fMRI, eye tracking, etc. Moreover, there are differing definitions, depending on the provider, of what is being measured. For example, two companies may offer EEG, but provide different metrics derived from EEG.
The answer to this question will clarify which techniques, technologies and methodologies the provider intends to use and why. The answer should address why the recommended techniques are best suited to meet a given study’s research objectives.
It should also be noted that neuroscience and neuromarketing research are rapidly evolving. This means that additional techniques and methodologies may emerge over time. The list below is not exhaustive.
7. What other “traditional” market research techniques, if any, does your company provide
Bio-/neurometrics are new tools that expand marketing insights, but do not necessarily replace all learning provided by traditional research methods such as survey, qualitative and behavioural response. Some companies offer only neuromarketing research, some provide a combination of traditional and neuromarketing research either directly, with partners or through third parties. The answer to this question will let you know what types of research are available.
8. If you provide both neuroscience and traditional techniques, when do you recommend using which
The answer to this question should explain when neuroscience would provide the answers a client is looking for, when traditional research would be a better option, and when a combination of neuroscience and traditional research is called for. The response should help clarify the philosophical perspective of the provider.
SPECIFIC TOOLS AND METRIC
9. Which specific neuroscience measures or metrics do you provide (e.g. emotional engagement, cognition, memory encoding/recall, comprehension, etc.)?
The answer should provide a full list of metrics offered by the provider, with a clear definition and explanation of why each metric is important.
Please note: Terms that are common in marketing parlance may have a specific meaning when used in neuromarketing research. In addition, the definitions of various terms may differ among neuromarketing research providers (e.g. engagement is a commonly used but not uniformly defined term, and different researchers measure it differently). The answer to this question will simplify the comparison of neuromarketing research providers in the market, help prevent misunderstandings, and reveal instances of differentiation that may inform the client’s decision making. You should encourage a level of response that meets your interests and needs (e.g. highly technical vs. high level summary).
10. What is the scientific background for the validity of your technology, methods and metrics?
The answer to this question should provide explanation of the scientific basis for the provider’s research technology, methodology and metrics. For example have their methods been peer reviewed and published? In doing so, it should provide the client with confidence for the legitimate application of all three to marketing research.
11. Which, if any, of your techniques deal with the measurement of emotion, and to what degree?
The answer to this question will tell you how a provider defines and measures emotions in their research, and what you can expect in terms of deliverables/insights in this area. For example, does one metric serve as an umbrella for the others or do they all haveequal magnitude? Will the techniques show the intensity of emotional response? Will the metrics show an overall level of emotions or do they distinguish between positive and negative emotions? Will the techniques provide you with detailed emotions (e.g. distinguish between angry and scared, happy and excited)? Will the metrics show which specific element(s) of advertising/product elicited an emotional response? What other insights regarding emotional response will be provided?
12. Which, if any, of your techniques deal with memory, and to what degree?
The answer to this question will tell you if/how a provider measures memory and what type of deliverables/insights should be expected: Will the provider measure how memorable (i.e. likely to be remembered by consumers) your advertising/product is? Will the metrics show which elements are likely to be remembered and which are likely to be forgotten? What other insights regarding generating recall will be provided?
13. Which, if any, of your techniques deal with attention, and to what degree
The answer to this question will show if/how a provider measures attention and what can be expected in terms of deliverables/insights. Will the provider show how much attention consumers pay to an advertisement/product? Which elements drive attention and which are unnoticed? Will the techniques show the reasons why consumers think about the advertisement/product (e.g. are they curious about it or are they confused by it)? What other insights regarding attention will be provided?
14. Are metrics available on a moment-to-moment/element-by-element basis, or as one static measurement, or both?
The answer to this question will clarify whether metrics are available as one number for a complete test, or whether metrics are available per moment and/or per element. Depending on what is being looked at by the client, this is particularly important for TV advertising, where it helps to understand a complete reaction to a whole advertisememnt, but it is also important to measure the reaction to each second of it. Can this technique provide one static measurement for comparison across a range of ads? Does the company have a database of ads in different stages for comparison? Does the company offer moment-to-moment measurement for diagnostics? How does the company know that the responses collected are in line with the audience’s experience of the creative (e.g. is there lag in the output)?
PORTABILITY
15. Can your technique be used in a laboratory setting only? Can it also be used at a respondent’s home, or in cinemas, shops, etc.?
Portability of equipment varies for different methodologies. There can be trade-offs between measuring response in natural environments, in a focus group facility or in a laboratory. In natural environments (such as in-store, at home, etc.), consumers are more realistically exposed to marketing stimuli. In other testing locations, researchers can better control the quality of the response data (controlling for distractions/noise, signal strength, etc.).
There are choices to be made between the realism of the consumer experience and control over measurement conditions. Study objectives should influence this decision, in conjunction with discussions with prospective providers.
16. Can your technique be fully integrated into online research data collection using the same respondents?
This is important to help understand how data that is already available through other market research channels can be integrated with neuroscience techniques. Some techniques require the use of trained technicians and specialised hardware in a dedicated location. Other approaches can be conducted in-home with the use of a webcam, delivered equipment or online programs. Online bio-sensory research techniques (e.g. eye tracking, facial coding and EEG) are relatively new.
17. What sample size do you recommend for clients? What is the rationale behind it? Has it been validated?
The answer to this question will show whether a provider uses the same sample size as traditional market research, or a smaller size (for both qualitative and quantitative research projects). It will demonstrate whether a provider has sufficient evidence that, despite a smaller sample size, results are reliable and, in the case of quantitative projects, representative of a total target.
Please note: Most neuroscience market research agencies recommend much smaller sample sizes for quantitative research than normally used in traditional market research (+/- 15-30 vs. 100+ in traditional research). Various rationales are provided by different agencies for this, and the extent of smaller sample size validation differs. It is important to understand whether the sample size is chosen arbitrarily, based on general knowledge, or through a rigorous validation process (e.g. multiple test/re-tests for the same research project, comparison of results obtained among smaller and bigger samples for the same research project).
18. Does the sample size depend on a target consumer definition?
The answer to this question will determine whether you should expect the same sample size for each research project or whether/how it will vary depending on how homogenous/diverse, broad/narrow, etc. your target consumer definition is (e.g. will it be the same for a research project that targets consumers aged 18-25 vs. consumers aged 18-65? One gender or both genders?, etc.).
19. What do you recommend as a minimum sample size for sub-group analysis?
The answer to this question will explain whether there is a need to use a bigger sample size for projects that require separate analysis of sub-groups (e.g. analysing results among total teenagers, but also looking at only male teenagers vs. only female teenagers, etc.).
SAMPLE SOURCE AND COMPOSITION
20. Can you confirm the source of the sample that you use for neuroscience market research. What is the primary advantage of your sample?
The answer to this question will explain whether a provider uses their own dedicated panel, third party panels (including their size, composition, etc.) or another approach for sourcing respondents. In the case of online 3rd party suppliers the ESOMAR 26 Questions can also be a guide to sampling. Understanding this answer will help in evaluating sample and determining how easy it will be to reach specific or narrow consumer targets and various geographies. This should make comparing agencies easier.
21. Are there any specific health, demographic or other related restrictions on your sample due to the technique or technology used?
By listing all restrictions, this answer will clarify (i) if target consumers can be interviewed using a given technique (ii) how representative of the general population the sample used by a given technique is.
Please note: Certain neuroscience techniques cannot be used on specific groups of respondents due to ethical concerns, health restrictions and/or technical limitations. For example, certain techniques are not used on pregnant women, left-handed people etc. It is crucial to understand upfront which respondent groups will be excluded from the sample to determine if/how it might impact a particular research project’s quality.
22. What are the drop-out rates and/or co-operation rates?
While a certain level of drop-out is expected when recruiting for any research project, very low co-operation rates will limit how representative of the general population a sample is. The answer to this question will allow you to understand if/how much higher, or lower, drop-out rates are compared to traditional market research.
23. What are people told when they are recruited?
The type of reward offered, level of detail provided on what the measurement process looks like (including any health related questions), as well as the wording of any confidentiality agreements, etc. can influence the type of people who agree to participate in a research project. Understanding this will help evaluate sample quality. The response should also explain whether amendments can be made to suit any specific client requirements.
24. How do you validate the sample (e.g. back checking)?
The answer to this question will show if a provider conducts any back checks of respondents’ identities, demographic profiles, whether the interview actually took place, etc. This will help in the evaluation of sample quality, and can boost confidence in the reliability of answers.
STATISTICAL ANALYSIS AND DATABASE COMPARISONS
25. What significance testing, if any, do you use?
The answer to this question will clarify a provider’s philosophy on significance testing: whether, when, how, at what confidence level, and why it should be used in results analysis. It is important to understand whether a provider’s point of view matches your company’s approach to using significance testing in quantitative research.
26. Do you use database norms? What are the sizes of the databases? Which geographies, products and consumer targets do they include? How do you ensure comparability of data between studies for various clients?
Is a provider able to illustrate the magnitude of obtained scores (are they weak, average, strong) by comparing them to previously tested cases? Can this be done in any location or product category, or for any target consumers?
Please note: Raw scores are useful when comparing between two alternative solutions, as they allow you to identify which is stronger. However, without comparison to previous projects, it is difficult to understand how strong the results really are. To address this, some agencies maintain robust databases of previously obtained scores and have processes for ensuring their comparability between projects. Such databases will increase their diagnostic capability. However, since typical scores can differ dramatically between different geographies, target consumers or product categories, it is important to understand not only the size, but also the composition of available databases in order to judge whether they will be relevant for a specific research project.
27. Do you provide additional modelling analytics?
The answer to this question will show if a provider can model/predict any consumer behaviour/business metrics (e.g. purchase intent, equity strength, ROI, etc.) using neuroscience metrics. The answer will also help clarify whether a provider will be able to assist with more complex modelling or statistical analysis, if necessary.
DATA QUALITY AND VALIDITY
28. What is your data quality control process?
The answer to this question will clarify whether a provider has data control processes in place, and what these controls are. For example, what procedures are in place to remove corrupt data or unsuitable respondents? Does the company have processes in place to identify sources of error? If providers offer more than one technique, are there different procedures in place for different techniques, etc.?
29. What processes are in place to ensure that scientific standards are adhered to during the measurement process? Who oversees these processes?
In answering this question, providers should describe standards validation and policies, and identify the key science staff.
30. What are your processes for interpretation of results, and who oversees these processes?
Do providers have qualified and experienced staff to interpret the results? Are they using standard academic practices and, if so, can they reference or point you to peer- reviewed papers that describe the appropriate method for interpretation?
31. Are you considering new approaches? Ask about parallel testing versus control validating between online and offline methodologies.
32. If you provide any business measures (e.g. predicting purchase intent, viewership rates, etc.), are they validated with in-market results?
Provider metrics predicting market behaviour should be validated using market data. Is such validation available for scrutiny or has it been tested against sales uplift? If the latter, providers should supply the client with testimonials.
POLICIES AND COMPLIANCES
33. How do you ensure your techniques are safe for respondents?
Neuromarketing research providers should be able to describe their safety standards, protocols, consent procedures and any other certifications, and explain how these safety measures are reflected in their testing process.
34. Is there a privacy policy in place? Is it compliant with all national and regional laws, and market research industry standards?
The answer to this question ensures that neuromarketing research providers are transparent in holding themselves to high standards on participant privacy. This may make it easier to compare between neuromarketing research providers in market.
35. What, if any, other ethical considerations do you take into account?
Neuromarketing research providers should be able to describe the origins and basis of any other ethical standards, protocols and consent procedures they use beyond those outlined in the questions above.
36. What is the age range for conducting research and what are your procedures for gaining permission to research minors?
The answer to this question may simplify the process of comparing providers across the market and clarify aspects of their standard sampling process and point of view on conducting research with children
Read the full ESOMAR publication here.
Neuromarketing Providers
Now have a look at NMSBA's global neuromarketing company directory