ATD Blog
Asking Questions So Respondents Can Answer
Sun Feb 21 2010
Content
Throughout the many workshops on measurement, evaluation, and ROI I have conducted a common question emerges when we discuss data collection -- how do you get people to respond? This question relates particularly to self-administered surveys and questionnaires. Answering this question is sometimes challenging as 'getting people to respond' is quite a broad theme. But to put it into context of the many discussions that have ensued during these workshops, the question relates to:
Throughout the many workshops on measurement, evaluation, and ROI I have conducted a common question emerges when we discuss data collection -- how do you get people to respond? This question relates particularly to self-administered surveys and questionnaires. Answering this question is sometimes challenging as 'getting people to respond' is quite a broad theme. But to put it into context of the many discussions that have ensued during these workshops, the question relates to:
Content
Receiving an acceptable response rate (non-response error)
Receiving an acceptable response rate (non-response error)
Content
Receiving reliable responses (measurement error)
Receiving reliable responses (measurement error)
Content
In an previous post I briefly mentioned non-response error along with another type of error, coverage error. So in today's post, we'll address measurement error.
In an previous post I briefly mentioned non-response error along with another type of error, coverage error. So in today's post, we'll address measurement error.
Content
What is measurement error?
What is measurement error?
Content
Simply put, measurement error reflects the subjectivity or bias inherent in responses to questions. Error always exists in question responses, unless you are answering a math question where 1+1=2. If answered incorrectly, it is not as much error as it is just plain incorrect. To put it in equation form:
Simply put, measurement error reflects the subjectivity or bias inherent in responses to questions. Error always exists in question responses, unless you are answering a math question where 1+1=2. If answered incorrectly, it is not as much error as it is just plain incorrect. To put it in equation form:
Content
Observed Response = True Value + Error
Observed Response = True Value + Error
Content
This means that the response to a survey question to a question (observed response) is the sum of the true answer plus the amount of error due to either chance (e.g. the respondent is in a good or bad mood) or the way in which the measurement is taken. Measurement error has a direct relationship with the reliability of the responses received from a survey instrument. While we cannot remove all random error, there are steps we can take to mitigate that error, improving the probability that what people say on a survey is what they mean.
This means that the response to a survey question to a question (observed response) is the sum of the true answer plus the amount of error due to either chance (e.g. the respondent is in a good or bad mood) or the way in which the measurement is taken. Measurement error has a direct relationship with the reliability of the responses received from a survey instrument. While we cannot remove all random error, there are steps we can take to mitigate that error, improving the probability that what people say on a survey is what they mean.
Content
How do you mitigate measurement error?
How do you mitigate measurement error?
Content
One simple solution to the measurement error issue is to ask people questions that they can actually answer.
One simple solution to the measurement error issue is to ask people questions that they can actually answer.