As noted in the Standard Error of Measurement knowledgebase article, a respondent’s score on any test or scale is not their true score, rather their true score + error. Therefore, if we want to compare two respondents who have taken the same test or one respondent who has sat two different tests, we need to use a formula to assess whether any differences we observe in test scores are real differences or simply differences that occurred due to measurement error.
To elaborate, if Tommy scores 25 on a test and Ada scores 30 on the same test, we cannot immediately conclude that Ada is better than Tommy in the construct being measured (e.g., verbal reasoning). Likewise, if Tommy scores 15 on his verbal reasoning test and 21 on his abstract reasoning test, we cannot immediately conclude that Tommy is better at abstract reasoning than verbal reasoning.
To enable such conclusions, it is first necessary to calculate the Standard Error of Difference between the two test scores. The SEdiff equation is based on Standard Error of Measurement and thus the more reliable a test is (in terms of test-retest reliability), the lower the SEm will be and the lower the SEdiff value. The difference observed between test scores must be the same or greater than the SEdiff value in order to be a true difference.
Where the scores being compared come from different tests, the SEdiff formula is:
SEdiff = The square root of (SEm squared of score one PLUS SEm squared of score two).
Where both scores are from the same test, the SEdiff formula simplifies to:
SEdiff = 1.414 x SEm
