

Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
The concept of cultural bias in psychometric assessments, its sources, and its impact on equivalence. It covers item bias, method bias, and construct bias, and provides examples and techniques for identifying and addressing bias in assessments. The document also emphasizes the importance of understanding cultural differences in the context of cross-cultural research.
Typology: Lecture notes
1 / 3
This page cannot be seen from the preview
Don't miss anything!
A lack of equivalence can be due to a host of reasons, which are referred to with the generic term of cultural bias. The effects of a biasing factor can be limited to a single item or a few items (e.g., translation errors). This is called item bias or differential item functioning (DIF). The effects can also influence the responses on most or all items, leading to method bias. Finally, bias can pertain to the op- erationalization or definition of a concept; this is referred to as concept bias or construct bias. An overview of various sources of bias is presented in Additional Topics Table 12.2. More extensive overviews can be found in Van de Vijver and Tanzer (2004) and Van de Vijver and Leung (1997). It is clear that there is a correspondence between levels of equivalence and sources of bias. This is most evident at the level of concept bias: unequal notions about concepts will lead to systematic differences in concept representation across cultures. The consequences for equivalence are least clear in the case of item bias. If only a single item or a few items show evidence of bias, they can be eliminated and this improves the equivalence of scores. However, evidence of item bias can also be taken as an indication that an instrument does not represent precisely identical traits. The notion of item bias is immediately clear if one thinks of an example. An item asking for the name of the capital of a country should be much easier for na- tionals of that country than in other countries. However, item bias can also be the consequence of subtle differences in shades of meaning. An item is identified as biased if test takers with the same score on the instrument but belonging to differ- ent populations do not have the same probability of giving a certain answer (e.g., “yes” answer, correct answer) for the item (Shepard, Camilli and Averill, 1981). Thus, the test score variable on the instrument as a whole is used as the standard to evaluate each separate item. Some simple bias indices have been proposed, such as the correlation between the difficulty levels of ability test items in two cultures, or between the levels of endorsement of questionnaire items. Another index of item bias is the interaction between items and culture in an analysis of variance. Psychometrically more sophisticated techniques are based on item response theory or loglinear models. A popular technique for analyzing bias in dichotomous items (e.g., yes–no; right–wrong) is the Mantel–Haenszel procedure (Holland and Wainer, 1993). For items with a graded response scale better use is made of the
Sources of cultural bias 9
available information if an analysis of variance is carried out with item score as the dependent variable and score level on the instrument and culture as independ- ent variables (for overviews of procedures see Van de Vijver and Tanzer, 2004, and Sireci, 2011). It is important to note that analysis techniques as mentioned examine relation- ships between items within an instrument. With these techniques for internal bias analysis, it is quite possible that method bias (affecting all or most items in an instrument) will remain undetected. For the analysis of method bias, external stan- dards or criteria are needed. In culture-comparative studies where relevant com- mon criterion variables are hard to find, the analysis of method bias requires considerable effort, because usually extensive additional data have to be obtained. In Additional Topics Table 12.2 sources of method bias are the most numerous. This reflects the importance of this category of bias, which is often ignored in cross-cultural research. The main measure against effects of method bias is the standardization of data so that the score distributions in each sample have a mean of 0.0 and a standard deviation of 1.0 (e.g., Leung and Bond, 1989) prior to further
Additional Topics Table 12.2 An overview of three types of bias and their possible sources Kind of bias Source Construct Incomplete overlap of definitions of the construct across cultures Differential appropriateness of item content (e.g., skills do not belong to the repertoire of either cultural group) Poor sampling of all relevant behaviors (e.g., short instruments covering broad constructs) Incomplete coverage of the psychological construct Method Differential social desirability Differential response styles such as extremity scoring and acquiescence Differential stimulus familiarity Lack of comparability of samples (e.g., differences in educational background, age or gender composition) Differences in physical testing conditions Differential familiarity with response procedures Tester effects Communication problems between subject and tester in either cultural group Item Poor item translation Inadequate item formulation (e.g., complex wording) One or a few items may invoke additional traits or abilities Incidental differences in appropriateness of the item content (e.g., topic of item of educational test not in curriculum in one cultural group) From Van de Vijver and Poortinga (1997).