Problems I have observed by reviewing video recordings
of examinations by psychologists and neuropsychologists:
(a) The examiner fails to give instructions in the way required by the publisher. The examiner makes up instructions or uses non-standard instructions.
(b) The examiner fails to properly limit the time in which certain tasks can be accomplished. Some tasks require strict time limits.
(c) The examiner times the tasks as required, but records an inaccurate time.
(d) The examiner fails to follow rules for when certain follow-up tasks are to be administered.
(e) The examiner fails to follow an established pattern for how tests are to be administered and does not record or identify failure to follow the established pattern of test administration.
(f) The examiner fails to read instructions in a verbatim manner as prescribed by the test publisher.
(g) The examiner does not query certain responses in a prescribed manner.
(h) The examiner queries responses that were not to be queried.
(i) The examiner prompts for a response when prompts are explicitly prohibited.
(j) The examiner is given an incorrect response. Instead of scoring it as wrong, the examiner tells the test-taker the answer is wrong and then prompts for a correct response with hints. The resulting response is then given full credit.
(k) The examiner teaches the test-taker how to solve certain problems when this is explicitly prohibited, and then gives the test-taker credit for responses following the non-standard teaching of the task.
(l) The test-taker gives one answer, but the examiner writes down another answer.
(m) Scoring rules for recorded items are followed incorrectly. Responses by the test-taker are assigned incorrect values.
(n) The examiner mis-scores the test. The examiner erroneously assigns scores to items because of calculation errors or by misidentifying incorrect responses as correct or correct responses as incorrect. Some mis-scorings cannot possibly be identified without a video recording. For example, some scores depend on writing down and scoring exactly what is said or by recording the absolutely correct time to complete a task. Some examiners write down a different response or a different time and thereby mis-score the item.
(o) The examiner mis-enters responses into a computer-based scoring program.
(p) The examiner uses bootleg, illegal, or homemade scoring programs that generate incorrect values for test scores used for interpretation.
(q) The examiner derives certain scores when scoring the responses and then reports different values in the written report.