Survey Says: Null

​Imagine the scene:

 You occupy a leadership role in your org, and you’ve spent the last six months designing a custom survey. Your team struggled through several rounds of revisions and exhaustive rephrasing, finally reaching agreement. The data comes in… and the team watches the response numbers go up with excitement. Finally, you’ve reached your target response number, and the survey closes. It’s time to analyze the evidence you’ve gathered and draw strategic conclusions. So much focus has been put on the outcomes of this survey. It’s almost like your organization couldn’t move forward without this crucial information, right? And then you learn. Quite frankly, the analysis of your data is quite anticlimactic. The associations are weak, and the takeaways are null. Your team spent six months working on this data collection instrument. A lot of incredibly smart people contributed to its design. What on earth happened?
Here are four ways you went wrong:

1. Your Survey is Way Too Long – Effective surveys are brief. Unless you can provide a value-added incentive, you need to make your survey as short as possible. Cut the fat. What do you really want to know? Better yet, what are you actually going to use? Often organizations will spend a lot of time (and money) collecting data that is not useful. 
2. You Sound like a Robot – Survey respondents are most likely to give valid answers to a survey when the survey language is direct, and culturally tailored. For example, if you are surveying youth, use their common vernacular in the survey questions and answers. If you are surveying medical doctors, you’re going to use a different tone. The literacy level should be universal – speak using words that will not need to be defined, and that are acceptable to all of the diverse respondents that may respond to your survey. Speak their language to seek their genuine perspective!
3. You Picked the Wrong Answers – This is the most common downfall I witness among surveys in the wild. Sometimes answer choices are too broad, making it difficult to derive meaning from the data. Sometimes answer choices are too specific, making it hard for respondents to relate to an answer. Sometimes the survey designer collects data in a mutually exclusive format, when answers should be collectively exhaustive. It is also difficult for organizations to determine when qualitative data is more appropriate. 
4. You Reached the Wrong Audience –  When you review respondent demographics, did your survey reach the target community? Is your sample diverse, or homogenous enough to meet the research aims? Consider issues like statistical significance, analyzability, and estimated sample size when you author and administer the survey. You won’t be able to properly use and interpret your findings if you don’t reach the target audience when you administer. 

Following these four guidelines will level up your survey game. If you need additional support in design, administration, or analysis of survey data for organizations or communities, Intersectional Inquiry excels in survey data collection and interpretation, especially mixed methods of quantitative and qualitative approaches. Given the amount of staff time spent in this imagined scenario above (don’t do the math – it’s horrifying!), hiring evaluative consultant support is not so expensive. You’ll save in the long run, and your outcomes will be meaningful and action oriented. Schedule a chat today to see where your organization falls on the spectrum.  

​Share this post: