Using questionnaires in research relies on the expectation that they measure the same things across different groups of individuals. If this is not true, then interpretations of results can be misleading when researchers compare responses across different groups of individuals or use in it a group that differs from that in which the questionnaire was developed. For the questionnaire we investigated, the Social Communication Questionnaire (SCQ), we found that parents of boys and girls responded to questionnaire items in largely the same way but that the SCQ measured traits and behaviors slightly differently depending on whether the children had autism. Based on these results, we concluded that researchers using this questionnaire should carefully consider these differences when deciding how to interpret findings. SCQ scores as a reflection of autism-associated traits in samples that are mostly or entirely made up of individuals without an autism diagnosis may be misleading and we encourage a more precise interpretation of scores as a broader indication of social-communicative and behavioral traits.