1. Home
  2. Surveys
  3. Qualtrics
  4. Checking Test Data in Qualtrics

Checking Test Data in Qualtrics

This post will help you guide you through one of the best practices for survey project management – checking test data.

Scenario

You’ve created a survey on casual dining chains and launched it. You’ve received all the responses (and more!) you were hoping to receive. You begin to analyze the data and something doesn’t look write…some of the questions have more responses than they should…and some of the questions have fewer responses…what’s wrong?

You probably have something wrong with your skip logic. Skip logic means respondents are conditionally assigned to answer certain questions based on their responses to other questions. Here’s an example survey employing skip logic:

Q1. Which of the following restaurants have you eaten at in the last month? (select all that apply)

  1. Applebee’s
  2. Denny’s
  3. Chili’s
  4. Waffle House
  5. None of the above (skip to Q3)

Q2. Which of the following food items did you order last time you ate out? (select all that apply)

  1. Hamburger
  2. French Fries
  3. Sandwich
  4. Soup
  5. Other (open response)

Q3. How frequently do you eat out?

  1. Once a day
  2. Once a week
  3. Once a month
  4. Once every three months
  5. Less than once every three months

In the survey above, having respondents who selected “None of the above” in Q1 answer  doesn’t make sense for Q2 to be asked to respondents who selected “None of the above” in Q1. So, you implement logic that allows respondents who selected “None of the above” in Q1 to skip to Q3.

Imagine implementing this same sort of process throughout a 40, 50, or 100 question survey. The survey is going to be complicated, and you want to be sure that you’ve correctly implemented all the logic into Qualtrics.

How can you know you’ve got the skip logic right?

There are two steps to being sure your data will turn out:

  1. Use survey test data
  2. Check the frequency tables

Creating Test Data

You should use the “Test Survey” feature in Qualtrics’ Research Suite. By using Test Survey, you can have artificial respondents “take” the survey and follow the possible logic choices. Best practice would be to generate somewhere between 50-100 fake responses in the data table.

Qualtrics Test Survey

Check the Frequency Tables

Then, you should use the frequency tables available in the reporting tool (View results>View Reports>Create Report). You can export the report to a Word document or PDF for easy viewing. After you export the report, you can compare the expected frequencies with the actual frequencies.

To continue building on the casual dining chain survey example, please use the below as your frequency check. For this example, you generated 40 test responses using the Test Survey feature.

Qualtrics Frequency Check 1

Using the survey script from above, you know respondents who select “None of the above” in Q1 should skip to Q3. So, you should expect 26 responses to Q2 (40-14=26).

Qualtrics Frequency Check 2

Looks right so far! Q2 has 26 responses. Now, you should expect Q3 to have 40 total responses because all respondents were directed to this question.

Qualtrics Frequency Check 3

40 total responses – it looks like your survey logic is working. You can now launch your survey. If the frequencies in either of your questions had been incorrect, you could have easily adjusted your skip logic in the Research Suite.

Conclusion

You wouldn’t launch a rocket to the moon without testing it first. You also shouldn’t launch a survey to your valuable respondents without testing it first. Qualtrics’ Research Suite makes it easy for you to do this critical step in project management. By checking your frequency tables with test data, you can rest assured knowing that your survey results won’t be biased by problems with your skip logic.

Updated on June 4, 2020

Was this article helpful?

Related Articles