Turbulent times cause changes in consumer behavior, directly affecting and disrupting trackers and normed studies. Today, we’re experiencing the most sweeping changes many of us will encounter in our lifetimes. The COVID-19 pandemic is causing many to consider tracker changes, as the massive shifts in global sentiment and behavior are impacting important metrics and indicators that companies use to evaluate performance over time.
So, what should our new approach to tracking research be?
Changing tracker studies is easier said than done. In more stable times, we’d need concrete, logical reasons for change. We are in a global crisis now, the fallout and duration of which is still unknown. Our previous research note has discussed how data may be affected. If these are driving you to consider change, then read on.
Do we really want to do this?
If you are thinking about changing your tracker, you must first ensure that a cross-functional group of people agree it is necessary and everyone will need to sign up for change. Then, the team should take a methodical approach that clearly and transparently outlines expectations and indicators of success. Below are five stages of tracker study change. Following these can ensure you have the best ongoing data on which to make decisions that will affect reputation, awareness, profitability and more.
Five Steps to Change
- Investigation: Collect as much information as possible about how tracker results are being used and the team’s perspective on the tracker’s strengths and weaknesses. You need to uncover who is using the data, for what business purposes, and how often. Dig into the reports that use the tracked data. Then get a sense of what those using the data see as perceived “problems” or gaps with the existing tracker, and the specific improvements they want. An often-forgotten step at this stage is to examine how reports are created, as this may be a painful or long step in the process.
List all answers to these questions and include all possible changes to the tracker and their impact on KPIs. Be as detailed as possible. Cite who brought up the change in the first place and whom or what it may impact. Then issue a high-level project plan. This must clearly assign task owners, deadlines and how critical each task is to the project. This may take a while, and it shouldn’t be rushed. Your most important deliverable at this point is agreement among team members that the investigation was open, inclusive and thorough and all key players are fully aligned.
- Design: This is when you should ask questions, and a lot of them. Start by seeking a full understanding of whether or not the market and behavior are being effectively captured. If the approach needs to be changed, is the change permanent? Here, I would argue that constant monitoring is needed during these times of upheaval. Almost certainly, things will change once again when the peak of the COVID-19 crisis passes.
The design phase is where the theoretical rubber meets the practical road. What you want to track and how frequently you track it will naturally have an impact on cost, since they affect sample size and survey length. Ensure you connect these dots with reporting requirements and the list of desired changes from the investigation stage. Everyone needs to sign off on the design to move forward. - Testing: Scrutinize your new design from every angle. If you are changing a questionnaire, literally read it out loud as you are testing it, think like a participant, and say what is going through your mind while you are answering the question. Avoid using industry jargon. Have a research methodologist help you carefully create an experimental design to test the new tracker (the “test”) against the old tracker (the “control”). The design needs to encompass the questionnaire, the sampling plan, and the fielding plan. If you are introducing multiple new elements, you should run multiple tests to isolate which new elements are changing the data. Don’t make compromises during this step or your careful preparation will be wasted.
- Analysis: At this stage you analyse the control data alongside the test data, comparing each data point to spot any differences. Imagine you are using the new data as you normally would in your business, producing the same metrics, KPIs and reports. This comparison enables you and your stakeholders to evaluate the new data and decide how to proceed.
The hardest aspect of this stage is socializing the “New Truth.” Change is inevitable, and we should expect this given that it was our intent! People will struggle to understand why data has changed, even though (as researchers) we understand that different methods will yield different results. If everyone agrees that the changes are acceptable, then this stage has been successfully completed. (If the changes between the control data and test data are wildly different, you may need to make some adjustments…that’s another article altogether.) - Implementation: Your careful preparation has led you here, to the final step. Your redesigned tracker – whether it be a new process, questionnaire or sample – is now implemented and is the new “data of record.” If this execution does not match the testing you’ve already done, you must retest the process. Ideally, at this stage, you will succeed with on-time and correct fielding of the new tracker, which will better help you stay abreast of changing consumer behavior and sentiment.
Conclusion
Changing trackers and normed studies is not easy work, yet the set of challenges is clear. When you pay attention to the details, follow a plan and get the right stakeholders involved, you can execute an effective tracker change with confidence. Remember – ongoing communication and transparency are vital for success. Tap into industry experts who have done this before – they can help you navigate and solve the various issues you will inevitably encounter. Their insights can help you accomplish each stage more smoothly. Ultimately, consumer behavior is changing, particularly given the current global pandemic. This means that researchers must consider changing ongoing studies simultaneously in order to continue to measure markets and performance accurately.
