Doctoral Research: The Importance of a Reliable and Valid Data Collection Instrument

By: Dr. Ron Jones

Doctoral research requires collecting data. Analyzed data allows doctoral scholars to assess, evaluate, or measure the variables included in the study. The data collection instrument is a key component of research because a foundational element of reliability and validity of the study rests with the chosen questionnaire (qualitative method) or survey (quantitative method). A reliable instrument produces consistent results regardless of the setting, yet reliability does not ensure accuracy or validity. Validity refers to an instrument’s capacity to accurately measure what the researcher intended to measure. Reliability must be accompanied with validity regarding an instrument worthy of use in conducting doctoral research.

A few simple examples help explain the meaning of reliability and validity. You step on a bath scale every morning to determine your weight, yet the scales consistently display a weight that is five pounds less than you actually weigh. The scales are reliable, yet reliability wrong. Scales with a high degree of reliability and validity accurately and consistently measure your weight. Using a rifle with a scope, a marksman shoots at a target 10 times. Each bullet hits in a close pattern five inches high and five inches to the right of the center. The reliability of the scope allows the marksman to be consistent, yet consistently missing the mark. A reliable and valid scope allows the marksman to hit the center of the target time after time. A male friend of one of my daughters purchased a truck. His first modification was to install a set of big tires. A few days later he receives a speeding ticket, yet proclaimed his innocence because his speedometer indicated he was not traveling over the posted speed limit. The larger tires caused his actual speed to be about 10% faster than the speedometer indicated. The measurements produced by the speedometer were reliable, yet inaccurate or lacking validity.

The instrument chosen to produce a measurement of a variable(s) must be proven to be reliable and valid; stated in more scholarly terms, hold a high degree of reliability and validity. Qualitative researchers seek high internal validity because of the need to justify statements and conclusions regarding causal inferences and relationships between variables. Quantitative researchers address the potential existence of a statistically significant correlation between two or more variables. Recognizing that a significant correlation does not imply causation, quantitative researchers seek high external validity, meaning the results from testing a sample population apply to the total population. An instrument lacking validity produces skewed or flawed results. The data might be reliable, yet invalid because the instrument did not accurately measure the variable under study. Doctoral researchers typically must defend the study’s reliability and validity. Collecting data with a proven instrument is a first line of defense.