You can find stuff about me here..
First involved in setting up our market research company Inzicht Marktonderzoek ('Insight Market Research') and a few years later in building our separate online fieldwork organisation Panel Inzicht ('Panel Insight').
Panel Inzicht ('Insight') is a young fieldwork organization that has gained a lot of experience in a short period of time. We are the biggest panel of the Netherlands. On our website you will find information about us, our panel and our methods. As a way of welcoming you as a new customer, we offer you an introduction discount of 15 percent! We hope you will contact us when you're looking for solutions to your needs.
Not everyone can have enough experience in all fields; sometimes thorough research is needed before you can make an important decision. This is why our company is called Inzicht Marktonderzoek ('Insight Market Research'): we provide you with the necessary insight you need. We do this by listening attentively when we talk about what you really need to know.
Online sampling is a rather straightforward process which a few distinctive steps, which will eventually lead to reaching the requested target. The steps are identical for all online quantitative projects with the exception of trackers, IHUT’s and re-contacts. These have proven to be a bit more tricky, but that’s not the subject of interest here. I want to point out common pitfalls in setting up and running and online quantitative sampling project. This first post dives deeper into the set-up process and an upcoming post will focus on the actual sampling process.
We, at Panel Inzicht, have a structured approach on setting up a project. First, we want our panel members to end up at specifically designed pages. For exmaple, when someone completes a survey when want this person to end up at our page stating they indeed have succesfully completed the survey and reward points will be added to their account. The same goes for persons who do not qualify for a particular project and persons who cannot participate (anymore) because the survey is now closed. So we have three links available for all projects and they are project-specific. After panel members have completed the survey (or did not qualify, or could not participate since the survey is already closed), they should be instantly redirected to the corresponding pages (also called endpages or redirects).
In order to reward our panel members, we need to know who completed which survey. Since our three endpages are project-specific, we know to what project we need to attribute a completed survey. But in order to reward the correct person, we need to know who (s)he is. That’s why all our panel members have a unique ID (sort of a social service number . When someone enters an online survey, this ID is also sent along in the link. When people are sent back to our endpages, we need this ID again to verify who this person is. It also enables us to keep our promise of (instantly) adding reward points when someone completes a survey. Without this ID we cannot know who actually completed the particular survey.
As I anticipated, the time between blogposts is increasing with every blog I post. Cannot imagine it will bother someone as it does not bother me. I just add something when I’ve come up with something AND I find the time to put it on the web. The occurance of both is rare I might say. So you are actually witnessing an extremely unlikely event as your are reading this. Having said all that, let’s get to the point (it make take me a while though).
Panellists, also called respondents and a bunch of other fancy names, are people (= fact). However, the market research industry is losing sight of this I believe and I fully agree with Ben Leet (twitter) in his blogpost on this topic (link). But let’s be honest, it is not easy to be constantly aware of this when you are looking at a screen which only spits out numbers. And you know all numbers represent people but it does not allways feel like that. Just an ordinary respons update:
These are all people who filled in your questionnaire. They opened their e-mail, clicked on the link and answered your questions (if they got past your screener that is). It took them time and effort to do so. After wrapping up a ‘project’ you might have:
This means that 1.400 people, I repeat 1.400 people, tried to participate in your research project but did not qualify and ended up at that (probably quite frustrating) screenout page. How about the overquota? These 129 people were invited and started the survey only to find out the survey has already been shut down (or: their ‘cell’ is full). Frustrating, to say the least. We can incentivise to compensate for this but on the long-term…?
Anyway, I realize that, from a research perspective, it is nearly impossible to prevent all this from happening. Panels don’t have everything pre-profiled so screeners are not going anywhere. And quota-sampling requires…. well yes: quotas. But I does frustrate me when I read e-mails from clients simply stating they ‘forgot to raise the quotas’ or ‘they noticed some downtime due to technicall errors’ and so on. Everyone can make mistakes but please be aware of the consequences for (our) respondents. Double-check everthing with care as you would be very carefull in sending your first newsletter to all your business contacts from the last five years. And don’t forget, our members have lives (it separates us from the cyborgs).
On behalf of our panel members: I thank you.
Cheers,
Renco
Today I would like to share some thoughts with you about a situation we have to deal with every day. As an online sample provider we help market research firms (and other sample providers) with the online fieldwork of their market research projects. I will be addressing several project management related topics in future posts, but for now I would like to focus on a very important factor: the price/quality ratio.
Recently, I joined an ESOMAR webinar on fieldwork/data collection vendors and in the same period Ron Sellers released his “More Dirty Little Secrets of Online Panel Research” report (link) which was followed a bit later by Gregg Peterson’s post called: “The end of “don’t ask, don’t tell” in online survey research?” (link). The latter two caused quite a stir which surprised me a bit since I believe every methodology has it’s flaws as explained here as well. All emotions put aside, sample buyers must indeed be very thorough in their selection process to ensure high quality standards. Several sample vendors have adopted the ISO 26362 standard for this reason. Focus on data quality is one of the fundamentals of market research and sample buyers should be demanding (1/2) with regards to this.
Having said that, let’s talk about price now since it’s the other part of the price/quality ratio. There is something to be said about that as well. Demanding (2/2) is the word that comes to mind here again. Up to a certain level you can be demanding for high quality online samples at competitive low prices (CPI’s, Cost Per Interview) but it’s really a slippery slope. We are very keen on delivering high quality samples but everything has it’s price, even so after the commoditization of the online samples. Clients are “on a tight budget” and send their requests to numerous fielwork companies, resulting in heavy competion. The market is sometimes described as being over-transparant, which works both in our advantage and disadvantage.
And yet the same clients frown over survey routers and alternative sample sources (also check this SSI presentation called: (River)sampling with Confidence). Not that I am arguing that the increased use of such technologies is solely driven by the rather unbalanced price/quality ratio, but I believe it is a major factor. Nor would I say that there are no potential risks in addopting them, but when used with care and blended with other technologies, we all will manage to live with the (very limited) effects. Cint wrote an interesting paper about it (link).
We, as research vendors, have to live up to high quality standards and we should emphasize more what this exactly entails (given the ‘commodity status’). We must defend our pricing and not give in to suggested price cuts. We stand for our ‘product’ and believe our prices are fair. At the same time, we should be transparant about our pricing and technology. A few tips for research buyers in selecting a vendor can be found in this blogpost by RMS Research. In the end we should be partners – not only vendors and buyers – since we complement each other. So let’s leave out the hypocrisy (a word I have merely chosen to draw your attention) and be friends. Shall we dance now?
See you next time!
Some useful links:
Lots of interest in the gamification topic in the past half year. You surely heard about the buzzword, no doubt. Thought I would also contribute a bit to the ongoing discussion dialogue.
The exact definition of gamification is subject to discussion for instance (also see this short paper). An (in)famous blogpost is this one – Gamification is bullshit, please read it (especially the comments) if you haven not done so already. Not that I agree with Ian’s view per se but I always like it when someone takes a stand at the commercialization of an already known term. For us, researchers, it might be relatively new though. At a MOA meeting last year, called “Next Generation Market Research – What type of researchers will survive?”, there was a discussion on whether researchers have, or will have, the required skills to move from ‘just’ reseacher towards a consultant. And, perhaps more important, if this is a desirable scenario itself. Kristof De Wulf wrote a comprehensive blogpost about this meeting so I will not address any of that here. What I do want to address is whether we, researchers, will be able to gamify research: do we have the required skills for this?
When I think about gamification, the name Jon Puleston comes to my mind. At the BaQMaR conference he presented a both visually appealing and very interesting Prezi on survey gamification. Last month he also presented his approach at (yet another) MOA meeting “Infotainment and Gamification in Market Research”. Quite interesting stuff I would say. Then again, that is all very nice but are we really capable of gamifying surveys? From my experience at my company Panel Inzicht I know that the gamification level in questionnaires is currently at the “funny slider”-level. Cannot speak for what happens in MROC’s as I hear some pretty neat stuff is being applied there. If we want to increase the survey experience something has to change. And I believe we are all convinced that there is room for improvement to be found out there..
As market research agencies and suppliers we tend to look at clients: they need to pay extra money for these gamified surveys but might no be willing to do so. That is what we (might) assume and this reasoning comes in very handy since it relieves us from responsiblity to take the initiative alltogether. But let’s assume clients are willing to pay extra, tomorrow. Uh oh, can we fullfill their request? Can we actually gamify surveys ourselves? I highly doubt it. Although gamifying questionnaires might seem rahter straightforward when reading all the material at our disposal (see below), I believe it certainly is not. Some companies might be ready for this today, but I believe the majority is not.
So why not offer it as a service? “Let me gamify your questionnaire”. Just send to questionnaires to company X, wait 5 days and get your questionnaires returned to you: gamified and all. Anyone knows a company who does exactly this?
See you next time, hopefully Will talk a bit on project management.
A few interesting blogposts on this topic (old & new):
Thought I would give my blog another shot I created this WordPress blog a while back but I never posted a thing. This first post should change that (I hope).
A while back I attended the BaQMaR Conference 2011 in Ghent (“Creative Ways“), Belgium. Since the ‘Marketing Research Track’ seemed the most interesting one to me, I had the pleasure of listening to four quite interesting speakers. Although I like analyzing numbers/data, hence the “Data Mining Track” track would have been an obvious choice, I picked the other track since it allowed me to hear more about behavioural economics and mobile market research. The first one had already proven to be very funny at the ESOMAR Congress in Amsterdam. This post is actually not about behavioural economics but about mobile market research. Also very interesting albeit less funny I would say.
Kristin Luck (Decipher) shared her thoughts about mobile research and we were all given the task to re-design an existing survey in such a way that we would want to take it ourself on our mobile device (whether a phone of tablet). As usual, much attention was given to the obvious, well-known, issues of many questionnaires: grids, length, lay-out, question length etc. All these things were quickly banned from the survey by us, but there were many subtle improvements to be made so we learned after the assignment (combining, skipping, rephrasing questions for example). But in every aspect, the mobile version of the questionnaire was way more enjoyable (more about enjoyable surveys later), way shorter and also more thought-out.
Apparently, we are motivated to make a mobile survey a succes (which is a good thing). But to me it also felt that we were (implicitly) stating that ‘regular’ surveys are beyond rescuing. We sort of accepted that the surveys contain awful grids and are way to long. Our company Panel Inzicht delivers online samples (also called: human beings) to market research firms and thus we deal with online questionnaires every day. And I can confirm that it often happens that my (‘genuine’) interest in a particular surveys drains after only a few questions. “Ah, bummer! I should have ticked only one brand instead of four. Now I have to rate them all by answering 20 questions per brand.” Know the feeling?
Should we accept this? I mean, we all keep saying that we should ban grids, make surveys more enjoyable and shorter etc. New techniques are being applied to market researh online communites (MROC) but I haven’t seen them much in ‘regular’ questionnaires. We must ACT on it and it entails more than adding a slider every now & then. We must fundamentally chance the way we approach questionnaire design. The buzz word #gamification comes to mind… (more on this later on). Mobile gives us a second chance, to start of in the right way!
Hope you enjoyed my shot at some blogging and see you soon!
Renco (@rencosch)