Ethnic Research is Good Science

Stanley Sue & Leslie Sue. Handbook of Racial & Ethnic Minority Psychology. Editor: Guillermo Bernai. Sage Publications. 2003.

Over the years, a number of researchers and scholars have pointed to the paucity of social science research on ethnic minority populations and the lack of attention to cultural variables in the research. These are significant problems that face the social sciences. Without more research, we cannot be in a position to understand the status of these populations and their needs. On the other hand, the lack of ethnic research is instructive. It points to certain processes that plague our scientific practices. In this chapter, we assume that a consensus exists over the importance of accurately understanding the experiences of ethnic minority populations. Given this assumption, we argue that the quality and quantity of ethnic research are uneven and handicapped by the (a) difficulties in conducting such research and (b) current practices in scientific psychology. We conclude by suggesting the kinds of changes that are needed in scientific practices to increase the quality and quantity of ethnic research.

Problems in Conducting Ethnic Research

The fact that research on ethnic minority populations is sparse has been well documented (Graham, 1992; Iwamasa & Smith, 1996; Padilla & Lindholm, 1995). Relatively few journal articles have been published that involved ethnic minority individuals. Although all researchers must grapple with theoretical and methodological issues in research, ethnic research requires additional considerations because theoretical models, assessment instruments, and methodologies have largely been developed on one population—namely, Americans in general and White Americans in particular. Psychology and the social sciences are well developed in the United States, and the subject of the research has been typically White Americans. Therefore, this means that investigators cannot simply assume that existing theories and assessment instruments have validity with ethnic minority populations, and they must expend some efforts in validating instruments or testing the applicability of existing theories before the examination of target research questions. They must take into account cultural considerations into every phase of research (Rogler, 1989). Sue, Kurasaki, and Srinivasan (1999) have outlined the difficulties in conducting ethnic research in critical research phases. They listed seven such phases: planning research, defining variables, selecting valid measures, sampling research participants, gaining cooperation from research participants, designing research, and interpreting findings. In all of these phases, special challenges and difficulties are posed for ethnic minority research, as shown in Table 9.1.

Planning for Research

In planning for research, the research questions must be addressed. The problem in this beginning stage is that there is often a smaller knowledge base on which to guide the research. For example, if a researcher is interested in the responsiveness of African American clients to culturally based psychotherapeutic interventions, the researcher is limited by the fact that there are very few rigorous empirical studies of the efficacy of therapy with African Americans, let alone culturally based therapies. A researcher often does not have the benefit of a direct body of knowledge on which to base the work. Furthermore, because theories and measures used in previous research are largely based on Anglo populations, it is unclear whether they are applicable to African Americans. Although theories and measures may have cross-cultural validity, one does not know a priori. Most researchers use existing theories and measures with the assumption that they are applicable, try to validate them with the ethnic population, or modify measures and theories so that they are pertinent to the ethnic population. In any event, even in this beginning stage of research, complexities confront researchers.

Definition of Variables

Increasingly, social scientists are confronting the fact that certain concepts such as race have been inaccurately defined. Race has been used to refer to a subgroup of people who have certain genetically determined, physical characteristics that are more or less distinct from other subgroups (Jones, 1997). Yet as knowledge has accumulated over the human genome, it is clear that physical characteristics (hair, skin color, facial features, etc.) cannot be used to distinctly separate groups into races—that differences in physical characteristics are largely quantitative rather than qualitative in nature. In fact, race is more of a social, rather than biological, concept. Psychological scientists primarily use self-designation (one’s self-reported race), not biological markers, to divide people. Similarly, ethnicity is also defined by self-report in most studies. Given self-definitions and reports, race and ethnicity are broad concepts that encompass a great deal of heterogeneity so that researchers may find it difficult to characterize members of a race or ethnicity (Trimble, 1988). Furthermore, race and ethnicity are often used as proxy variables that are mediated by, or correlated with, other variables (Walsh, Smith, Morales, & Sechrest, 2000). These other variables may be of greatest interest in explaining findings. For example, a researcher may find that Asian Americans tend to avoid the use of mental health services. Although this is an important finding, Asian Americans may be more susceptible to feelings of shame and stigma, which are the important variables in explaining the avoidance of services. The definition of variables and identification of explanatory variables are important and often difficult tasks.

Table 9.1 Phases in Ethnie Minority Research

  • Planning for research
    • Paucity of baseline research
    • Unknown validity of theories based on other populations
    • Unknown validity of assessment instruments
  • Definition of variables
    • Operational definition of race and ethnicity
    • Race or ethnicity as a distal variable
    • Between- versus within-group emphasis
  • Selecting measures
  • Translation or language equivalence
  • Conceptual equivalence
  • Metric equivalence
  • Selection of subjects and sampling
    • Sampling of small populations
    • Representativeness of samples to the population
  • Gaining cooperation
    • Problems in informed consent
  • Cultural differences in cooperativeness
  • Research designs and strategies
    • Qualitative and quantitative designs
    • Explanatory research
  • Interpretation of data
    • Insider versus outsider perspectives
    • Drawing inferences
    • Generalizations

Selecting Measures and Establishing Cross-Cultural/Language Equivalency

When studying members of ethnic minority groups, some of whom may have limited English proficiency, what assessment instruments or measures should be used? Care must be taken to ensure that the instruments measure meaningful psychological concepts in a valid fashion. Brislin (1993) has noted that in studying ethnic minority populations with instruments primarily constructed for White or mainstream populations, establishing several types of equivalence is critical: (a) translation or language equivalence (when the descriptors and measures of psychological concepts can be translated well across languages), (b) conceptual equivalence (whether the construct being measured exists in the thinking of the target culture and is understood in the same way), and (c) metric equivalence (whether the scale of the measure can be directly compared for different cultural groups, e.g., whether an IQ score of 100 on an English intelligence scale may be truly equivalent to a score of 100 on the translated version of the same intelligence scale).

Thus, researchers interested in the study of ethnic minority populations cannot simply use existing psychological measures and tools without first considering the equivalence of the measures. Sometimes, in working with ethnic minority populations, innovative means must often be found to measure characteristics. For example, in the National Latino and Asian American Survey (NLAAS), a creative strategy was used to survey the mental health of Latinos and Asian Americans in the community. Because interviewers conducting household surveys might find respondents who speak a different language or dialect (e.g., a Chinese-speaking interviewer who encountered a Vietnamese-speaking respondent), NLAAS equipped interviewers with cell phones. If a respondent spoke a language different from that of the interviewer, the interviewer would then dial into a waiting group of interviewers who collectively could speak many different languages so that the respondent could be language-matched with an interviewer over the phone.

Selection and Sampling of the Population

Scientific research principles of selection and sampling of the population are no different for a cross-cultural or ethnic population as for research in the general population. However, complications exist because of the size of ethnic minority populations and possible differences in responding because of culture. Given the relatively small size of ethnic minority populations, it is often difficult to find representative samples and adequate sample sizes. Furthermore, some respondents, because of their cultural background, may find participation in research to be intrusive or taboo in nature, strange because of unfamiliarity with the research process, and anxiety provoking because of fears over how the collected information can be used. Refugees in particular may fear that their responses can somehow be used against them. Because of the problems in finding a representative sample of a relatively small minority group, ethnic minority research can be very costly and difficult to initiate. For example, in one study of Chinese Americans, the project cost $1.5 million to conduct a two-wave study (see Takeuchi et al., 1998). Nearly 20,000 households were approached to find 1,700 Chinese respondents.

The difficulties in finding adequate samples of certain ethnic minority populations have often led researchers to find convenience samples from quite different sources. For example, the samples may come from lists of ethnic organizations, names suggested by other respondents (the snowballing technique), and universities rather than communities at large. Another strategy is to combine groups so that an adequate sample size is reached. Instead of just studying Puerto Rican Americans, a researcher may broaden the base by including all Hispanics. Although these strategies can help to increase sample sizes, they obviously run the risk of subject self-selection, lack of representation of the population, and increased heterogeneity.

Gaining Cooperation from Research Participants

In conducting research requiring self-disclosure, ethnic minority respondents may be reticent in answering questions on their feelings or emotions, making the process of collecting data longer than anticipated. Respondents may be hesitant in discussing issues openly when they fear being overheard by other members of their family and community. Although respondents may be reluctant in participating in research, cultural variables may affect how cooperative respondents are.

The very act of obtaining consent to participate in the research may make prospective participants less likely to cooperate. For example, in the United States, respondents are asked to read a description of the research and then sign an informed consent statement. Although the procedure is common in the United States, respondents from other countries and cultures may be unfamiliar with the notion of informed consent. Respondents may be concerned over providing their signature or alarmed at the possible negative impact that the research may have, as stated in the description given to participants. Many may decline to participate.

Research Design and Strategies

The field of psychology is scientific as well as professional. It is perhaps accurate to characterize the scientific field as quantitative in nature. Knowledge is primarily acquired by the numerical analysis of observations or experiments. As mentioned earlier, mainstream diagnostic concepts, theoretical orientations, and treatment models are laden with values, beliefs, and attitudes, representative of the mainstream U.S. culture from which they were derived.

How, then, do we establish an understanding of relevant concepts and theoretical frameworks across diverse cultures, and how do we ensure that research findings are interpreted within the appropriate cultural context? A number of cultural investigators have emphasized qualitative methodologies that are more holistic in approach to understand the meanings, patterns, rules, and behaviors that exist in ethnic minority communities. Qualitative methodologies are often used when phenomena are difficult to quantify or measure using existing instruments. A number of qualitative strategies can be found, including ethnographic research (the study of the practices and beliefs of cultures and communities), case study, phenomenological research, participative inquiry, and focus groups (Mertens, 1998).

Most of the training in psychology is based on quantitative methodology. Although the learning of such methodology is necessary, training in qualitative approaches has typically been neglected. Thus, many investigators who want to study ethnicity find themselves ill trained.

Interpretation of Findings/Validity

Cultural issues also arise in the interpretation of research findings. For many years, African American researchers have objected to the pervasive view that African Americans are deficient—coming from broken families, low achieving, having poor verbal communication skills, and exhibiting self-hatred (Jones, 1997). Asian Americans have objected to interpretations of research findings on well-being that characterize them as being extraordinarily adjusted, well functioning, and educated compared to other Americans. When comparisons are made between various ethnic minority groups, differences between the groups cannot be assumed to reflect desirable or undesirable characteristics because the value assigned to the characteristics may be simply a reflection of one’s own norms. This does not mean one should adopt absolute relativism, in which there are no standards that cross all groups. Rather, researchers must always consider whether their conclusions are biased in the direction of ethnic stereotypes and misunderstandings. They must also be sensitive to how characterizations are viewed by members of the ethnic group. The interpretation of findings should take into account the perspectives of insiders and outsiders to the ethnic group being investigated.

It should be noted that in the study of ethnicity and race, the research can be highly controversial. Genetic differences in physical features and in intellectual functioning, socioeconomic status, rates of crime, health and mental health disparities, personality characteristics, racism and discrimination, stereotyping, social relations, and hate crimes are issues that have been associated with race and ethnicity. These issues often raise strong emotions, feelings of discomfort, and political controversy. Ethnic researchers are sometimes subjected not only to professional scrutiny and standards of science but also to more personal criticisms about positions taken or conclusions drawn. Thus, researchers who wish to avoid controversy may be reluctant to study race and ethnicity. This is unfortunate because ethnic research is beneficial not only for ethnics but also for all Americans and for our science.

In conclusion, becoming involved in ethnic research has taught us that such research is very difficult to conduct, and many researchers have not been exposed or trained to deal with these difficulties. The unfortunate effect is that investigators, who find the research difficult or expensive to conduct, may prefer not to study ethnicity or prefer to treat ethnicity as a nuisance variable to be controlled. In either case, collection of data is jeopardized, and little knowledge on ethnic minority groups is gained. In each phase of research, efforts must be made to increase the meaningfulness and fidelity of the tasks.

Scientific Practice

In the previous section, we argued that conducting ethnic research requires cultural sensitivity and awareness in all phases of the research endeavor. Researchers may be reluctant to engage in ethnic research because of the difficulties. At a more fundamental level, the lack of more research on ethnic minority groups is attributable to the very practices used in psychological science. In particular, there is a selective enforcement of scientific principles (Sue, 1999). Table 9.2 outlines the arguments in selective enforcement. First, internal validity is emphasized over external validity in scientific practices. Second, because of the de-emphasis on external validity, there is little interest in testing the generalizability of findings, measurement instruments, or theories derived from one (usually the mainstream) population to another. Third, researchers interested in studying an ethnic minority group population must then either accept the generality of findings from the mainstream population (and run the risk of using findings that do not apply to the ethnic population) or reject the generality of findings (and then be forced to conduct baseline studies simply to establish a baseline of knowledge). Fourth, ethnic research has a difficult time being published because reviewers can question the validity of ethnic research because it may be based on findings that cannot be generalized or because a baseline level of knowledge has not been established.

The Selective Enforcement of Scientific Principles

In psychological science, the goal of research is to describe, understand, predict, and control phenomena. Underlying this goal is the need to determine cause-and-effect relationships. If, say, the etiology of schizophrenia is found, then we can describe and understand the disorder and are in a position to predict and intervene to control the disorder. Discovering causal relationships is therefore a basic goal, and this discovery is often accomplished by conducting research with internal and external validity. Internal validity refers to the extent to which conclusions can be drawn about the causal effects of one variable on another (i.e., the effect of the independent variable on the dependent variable). External validity is the extent to which one can generalize the results of the research to the populations and settings of interest. Both kinds of validity are necessary. Researchers need to construct rigorous research designs that will allow one to draw causal inferences and to conduct sufficient research to know the extent to which the findings apply (or do not apply) to different populations and settings. The latter task is particularly important in allowing one to know how universally valid or applicable the research is.

The scientific principles of internal and external validity are differentially enforced and encouraged. In practice, attention is paid to internal validity, and external validity is often assumed but not tested. Far more papers submitted for publication are probably rejected because of design flaws, inability to control for confounding or extraneous variables, and so on, than for possible external validity problems.

Table 9.2 Consequences of Selective Enforcement of Scientific Principles

  • Internal validity elevated over external validity
    • Attention paid to internal rather than external validity
  • Assumed generality of findings
  • Low knowledge base for ethnic minority populations
    • Few empirical findings
    • Need to conduct baseline studies
    • Validity of existing theories and assessment tools unknown
  • Problems in ethnic research productivity
    • Research costly and difficult to conduct
  • Papers submitted for publication considered preliminary
    • Justification for funding more difficult to argue
  • Fewer publications on ethnic minorities

The assumption of the generality of findings and the dominance of internal over external validity are evident in much of the psychological research being conducted. Let us provide three examples. First, much psychological research has been conducted on college students, and major theories of human beings have been developed on college students. Impressive is the rigor and ingenuity of research designs to enhance internal validity. Yet a number of researchers have pointed to limitations in the generality of findings derived from the use of college students who do not represent mainstream Americans in educational attainments, age, ethnicity, social class, attitudes, and values (Kazdin, 1999; Sears, 1986). Second, psychological research is relatively mature in the United States compared to other parts of the world. Theories of human beings, measurement tools, and research methods are developed by this research. Americans are the primary subjects of the research. However, Americans represent only about 5% of the world’s population. This means that theories and findings selectively derived from one country representing about 5% of the world’s population are used to generate universal theories. This is neither good practice nor good science. It points to the lack of interest in external validity.

Finally, the dominance of internal over external validity is apparent in the mental health field, where a distinction is made between efficacy, which has been considered the gold standard for outcome research, and effectiveness research. Efficacy studies are investigations that examine the outcomes of mental health interventions, often by using experimental methods, control over the influence of extraneous variables, and laboratorylike settings. On the other hand, effectiveness research examines outcomes of mental health interventions in a naturalistic setting (such as a mental health clinic) where random assignment of clients, clear-cut experimental manipulations, matched experimental and control groups, and so forth, may not be possible. Whereas efficacy research may employ strict exclusion criteria in the selection of clients to study, this may not be the case in effectiveness research. For example, in an efficacy study of a treatment for schizophrenia, exclusionary criteria might include comorbidity, current use of psychotropic medication, and non-White race. That is, clients who have multiple disorders, are using medication, and are members of an ethnic minority group may be excluded from the study to eliminate variables not being studied. However, such a practice would limit the applicability or generality of the findings. On the other hand, effectiveness research studies treatment outcomes in more real-life situations, where patients may have multiple disorders, may be taking medication, and may be members of various racial groups.

Effects on Ethnic Research

The relative inattention to external validity means that researchers are not particularly motivated to test the generality of their research findings. Generality is simply assumed. In such a situation, researchers may feel little need to devote resources to studying ethnic groups. The assumption of generality is actually antithetical to science because conclusions in science are built on facts and evidence, not assumptions or biases. Scientists should be skeptical and base conclusions on research findings or logical deductions. In the absence of evidence, beliefs or hypotheses should be tested. Because much of the psychological research is not based on ethnic minority populations, it is actually unclear whether a particular theory or principle is applicable, whether an intervention has the same phenomenological meaning for different cultural groups, or whether measures or questionnaires are valid for these populations. The responsibility is placed on those who are interested in ethnic minority groups to show whether research findings that are based on mainstream Americans but assumed to be universal are applicable to ethnic minority populations. When the research is not applicable, ethnic researchers have to conduct basic or descriptive research to establish baseline knowledge or test the cultural adequacy of research measures and designs. This entire process results in fewer publications, more descriptive and pilot studies rather than sophisticated investigations, and lower probability of receiving research funding for ethnic rather than mainstream populations.

Guyll and Madon (2000) believe that the notion of selective enforcement of scientific principles is a serious charge that is leveled against the scientific community. They contend that (a) current practices in science are not biased against ethnic research, (b) ethnic minority research that replicates research on the mainstream population will be judged as being less important, (c) ethnic research that is inconsistent with existing theories is considered only as a weak challenge, and (d) it is parsimonious to assume that theories developed on White populations have generality to other populations. In essence, Guyll and Madon argue that their propositions are based on “theoretical conservatism.” Theoretical conservatism is an approach that is not biased against ethnic minority research. However, ethnic minority research is likely to be considered relatively unimportant because if the research supports the theory, it is considered a replication. Replications are often not highly valued. Guyll and Madon further maintain that if ethnic research findings contradict the theory, then the challenge to the theory is weak because alternative explanations exist for obtaining different findings across theories. Novel theoretical claims based on new research often lack a strong empirical foundation. They are especially vulnerable to subsequent refutation by contradictory evidence or by the discovery of an internal flaw. Finally, they believe that precluding generalizations from one population to another would encourage a Balkanization of psychological science, requiring the testing of all populations before validating the universality of a theory. The position adopted by Guyll and Madon (2000) can be challenged on several fronts (Sue, 2000). It is not clear if their characterization of theoretical conservatism is accurate. However, if one accepts their notion of theoretical conservatism, it seems to run counter to scientific conservatism. As noted earlier, science requires evidence and a skeptical attitude. The assumption that a theory is universally valid, in the absence of cross-cultural validation, seems to violate the scientific principles that require evidence for conclusions and skepticism in thinking. It is not consistent with scientific principles to make generalizations without evidence. Consistent with rigorous science is the careful testing of theories and their generality, especially because there are many instances when theoretical formulations based on one population fail to be validated with other populations (Sue, 1999). It is true that requiring the cross-validation of theories with different populations will entail much effort. However, even if not all populations can be subjected to research, the goal is to construct a nomological net that supports a theory in question. Psychological theories are difficult to “prove.” Our science is largely based on probability. Through research, one can increase the probability of being theoretically sound. The real task is to study different populations, form a nomological net, and increase one’s confidence in the validity of a proposed theory. Inclusion of ethnic populations in research is therefore important and consistent with good scientific practice.

Toward Solutions

As we have tried to indicate, two main problems have hindered the development of ethnic research: (a) the methodological and conceptual difficulties in conducting such research and (b) the practice in psychological science of ignoring the importance of external validity in research. How can these two problems be addressed?

First, as revealed in Table 9.1, cultural considerations are critical in all phases of research. Therefore, the training of researchers with expertise in ethnic minority and cross-cultural research is important. Graduate and postdoctoral training programs should offer opportunities to learn about the methodological and conceptual problems in ethnic research and the means to conduct rigorous ethnic research. Recently, some excellent textbooks have been published that provide a basic introduction into race/ethnic minority research (e.g., see Walsh et al., 2000). Second, the importance of ethnic and cross-cultural research in validating the universality or applicability of theories, methodologies, and measures should be emphasized. In fact, all theories as well as measures should be rated as to their cross-cultural adequacy. For example, if Theory A is grounded in research on only one population, it should be considered as a local (emic) theory until tested and validated with other populations. On the other hand, Theory B, which has been tested and validated with many different populations, should be viewed as being more robust, rigorous, and stringent in meeting research criteria and more applicable to human beings in general than Theory A. It is indeed impressive for a researcher to not only create theories and measures but also demonstrate their meaningfulness for a variety of populations. In reviewing manuscripts for publication, journal editors have a responsibility to require research to demonstrate cross-cultural validity. These actions will do much to increase the importance of ethnic and cross-cultural research. They are good for science.