2.2.9. Use of psychodiagnostic instruments on certain groups
The use of psychodiagnostic instruments on special groups, such as people with physical or intellectual disabilities, or people who do not speak Dutch, imposes special demands on the test procedure. If a client belongs to a specific sub-population, this can affect the test results. See the topics in BOXES 11 and 12.
BOX 11: Item bias
Item or question bias can come into play when different groups respond differently to a particular question without these groups’ differing from each other with regard to the characteristic that is being measured. In that case, the relevant question measures something other than is intended with the psychodiagnostic instrument. An example of this is a question in a test on study skills, where information from a colour picture must be used. If the use of colour in that picture is such that people who are colour blind cannot get the relevant information from it, then the answer to that question says nothing about the study skills of people with colour blindness. This is clearly a case of question bias, because the question puts people with colour blindness at a disadvantage. Another example of possible item bias is a narrative computation problem to measure numeracy. Such an item might measure the language skills rather than the numeracy of certain groups of respondents.
BOX 12: Test bias
This phenomenon of bias can also be found in an entire psychodiagnostic instrument: if the response time is measured by the computer and left-handed people must work with a right-hand mouse, then this may yield a distorted, and probably a worse, test result. Another well-known example of test bias is an intelligence test in which many culturally specific questions are included. Respondents with that particular cultural background often score higher than other respondents, and this can lead to the false conclusion that this group is more intelligent. However, such an instrument thus measures cultural knowledge rather than intelligence.
If a client belongs to a specific sub-population, the psychologist should take this into account by mapping out relevant factors such as cultural background, language skills, left-handedness and mental or physical handicaps, prior to conducting a test. The professional responsibility of the psychologist for taking account of the individual characteristics and circumstances of each client is laid down in Article 58, “Respect for individuality and diversity”. The psychologist is also expected to strive to allow for culture and diversity in their professional activities.
It is also important to consider these factors when selecting a psychodiagnostic instrument. For instance, a non-verbal intelligence test can thus be used to reduce the effect of language proficiency or cultural background on the test results. Whether this is possible depends on the purpose of taking the test, because if measuring language skills is part of it, the use of a non-verbal instrument does not make sense. The COTAN has put in place a supplement to the COTAN system for reviewing the quality of tests, whereby the adjudication of the assessment for its fairness is made more transparent. This is done by means of a so-called fairness matrix, in which a structured description of the research that has been done is given, to check for the impartiality of the test. More information on how the COTAN assesses fairness is available on the NIP’s website.
The test conditions must be optimal, and it may be necessary, for the test to be administered soundly, to have separate norm tables for these groups. If there is no instrument available for the specific population to which a client belongs, it is important for the interpretation of the test results to take as full account as possible of the factors that may affect them.
For the benefit of a psychodiagnostic assessment of disabled clients, the testing arrangements are set up in such a way that everything is accessible to and suitable for these clients. This means, then, that the testing location for these clients is adapted ergonomically if necessary, in accordance with the standard instructions for test-taking. The psychologist ensures that clients with disabilities get the information they need so the testing can be reliable – and, of course, the standard instructions for taking the test must be observed.
In some cases, a test publisher prescribes adjustments for cases in which the instrument is used for clients with disabilities. Examples include the use of screen-reading software for people with visual impairments or dyslexia, or, for test-takers with dyslexia or motor problems, prolonging the time allowed for the assessment in cases where there is a time limit.
If there is a deviation from the standard test situation, however, this can, in principle, impact the validity and the reliability of the scores. For example, when the norms of an instrument are based on the test’s being taken without a time extension, giving extra time can distort a test result. A client’s test scores are, after all, comparable to those of a norm group only if the client has taken the test in circumstances as close as possible to those in which the norm group took it. In general, therefore, a justification should be given for a departure from the standard test situation, and the impact on the test results should be discussed (see Article 47, “Exercising due care in obtaining and reporting data”). Thus, it is not a good idea, merely on the basis of a statement by a client that they are dyslexic, to depart from the standard approach called for. The psychologist must have either observed the dyslexia themselves or taken cognisance of a declaration of dyslexia. Such a declaration is a summary of the report of the psychodiagnostic assessment confirming that the client has dyslexia. The statement describes the basis for the diagnosis, what the possible explanations are, and what obstacles the client faces as a result of the dyslexia. It also indicates what treatment, material facilities, guidance and compensation or dispensations are necessary.