Validity means “Is the test fit for purpose?”
Some different types of validity:
Face Validity (low-level of importance overall)
Asks: “Do the questions appear to measure what the test purports to measure?”
Important for: Respondent buy-in
How assessed: Simply by looking at the questions
Content Validity
Asks: “Do there appear to be enough suitable questions to measure the complete construct we are trying to measure?”
Important for: Ensuring a holistic assessment of the construct
How assessed: Asking subject matter experts to review the questionnaire
Construct Validity (one of the two highest levels of importance overall)
Asks: “Does the test actually assess the construct it purports to assess?”
Important for: Ensuring that the construct is being measured by the test and for use in norm-referenced testing
How assessed: Correlate respondents scores on the test under examination with a well established measure of the same construct. The expectation is that there will be a high correlation given that similar constructs should converge.
Criterion-related validity (one of the two highest levels of importance overall)
Asks: “Can a respondent’s test score predict a real world outcome such as performance at work?”
Important for: Knowing whether the test can predict anything meaningful and for use in criterion-referenced testing
How assessed: Correlate test scores with criteria such as performance appraisal scores.
Concurrent criterion related validity – test scores predict a criterion (e.g., performance) now
Predictive criterion related validity – test scores predict a future criterion (e.g., whether or not respondent will pass training)
Vocab:
Construct = (e.g.) A personality attribute such as extraversion or an aspect of ability such as numerical reasoning
Correlation = Relationship between 2 or more variables/constructs/things
