• Nancy D Brener, Jo Anne Grunbaum, Laura Kann, Tim McManus, and Jim Ross.
• Division of Adolescent and School Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia, USA. nad1@cdc.gov
• J Adolesc Health. 2004 Aug 1; 35 (2): 91-100.

PurposeTo understand how methodological factors influence prevalence estimates of health-risk behaviors obtained from surveys, we examined the effect of varying question wording and honesty appeals while holding other aspects of the surveys constant.MethodsA convenience sample of students (n = 4140) in grades 9 through 12 was randomly assigned to complete one of six versions of a paper-and-pencil questionnaire in classrooms. Each questionnaire version represented a different combination of honesty appeal (standard vs. strong) and questionnaire type. The questionnaire types varied in wording and in the number of questions assessing particular types of behaviors. The questionnaires were based on those used in three national surveys--the Youth Risk Behavior Survey, Monitoring the Future, and the National Household Survey on Drug Abuse. Logistic regression analyses examined how responses to each survey question assessing behavior were associated with questionnaire type, honesty appeal, and the interaction of those two variables.ResultsAmong 32 behaviors with different question wording across questionnaire types, 12 showed a significant effect of questionnaire type. Among 45 behaviors with identical question wording across questionnaire types, five showed a significant main effect of questionnaire type. Among all 77 behaviors, one showed a significant main effect for honesty appeal and two showed a significant interaction between honesty appeal and questionnaire type.ConclusionsWhen population, setting, questionnaire context, mode of administration, and data-editing protocols are held constant, differences in question wording can create statistically significant differences in some prevalence estimates. Varying honesty appeals does not have an effect on prevalence estimates.

12 keywords...

hide…