Follow Us:

FSG Blogs

see all

Strategic Evaluation Blog

See more posts from the Strategic Evaluation blog

Cultural Competence in Evaluation

Posted by: Strategic Evaluation on 9/7/2011

by Katrina L. Bledsoe, Ph.D., Education Development Center, Inc.

Recently, I had a fortuitous discussion with a dean from a reputable medical school in the east coast who asserted that culture accounted for “almost nothing” in suicide prevention programs and that, for instance, issues pertaining to ethnic culture, were “a wash.” To him, there were no differences between cultures since most people have the same physical health attributes (e.g., heart, blood vessels, etc.). While the rest of the conversation was certainly spirited and lively, this particular statement further underscored the fact that despite the year being 2011 of the 21st century, many professionals do not have a good understanding of the effect of culture and cultural contexts, and how these aspects determine the manner in which we evaluate programs. Evaluators are often tasked with understanding and interpreting programs, and we are motivated by the dynamic and changing landscape of organizations and societies to recognize and embrace novel ways of telling stories, and accurately providing feedback on the wide variety of community programs and endeavors.

The American Evaluation Association (AEA) recently passed the organization’s first-ever member supported statement on cultural competence in evaluation, a monumental accomplishment. The Public Statement on Cultural Competence in Evaluation was drafted over a seven-year period, originally commissioned by AEA through the 2005 Building Diversity Initiative (BDI), an initiative designed to build and increase capacity in the field by training and nurturing evaluators of diverse backgrounds. The intended goal of the statement is to serve as a representative of the association’s values and commitment to building diversity, and conducting culturally relevant, reputable, and responsive evaluation. AEA recognizes that it is increasingly important for evaluators not only to understand the basics of evaluation such as methodology, data collection, and dissemination, but also to understand the politics behind programming such as institutional discrimination, which can influence and determine the cultural context, the community, the program, and the evaluation.

In considering these important factors, evaluators must understand that a) evaluation is context and cultural dependent, and b) it is the responsibility of the evaluator to accurately and reliably represent that context. This sometimes requires a shift in role from judge to, in some instances, careful documenter, historian, and collaborative partner.

AEA’s statement acknowledges that cultural competence does not consist of learning a specific set of skills that is generic and cuts across contexts and situations. Instead, cultural competence is a paradigm shift: “Cultural competence is a stance taken toward culture, not a discrete status or simple mastery of particular knowledge and skills.” (pg. 1, Public Statement on Cultural Competence in Evaluation, AEA, April 2011).

The statement stresses that the culturally competent evaluator recognizes four key aspects:

  • That culture imbues all aspect of society from the economic to the political;
  • That culture is fluid rather than static;
  • That the culturally aware evaluator is always self-aware of what she/he brings to the context including but not limited to biases and privileges; and
  • That culture affects all phases of the evaluation and therefore must be considered not only within the methodological design of the evaluation but also in staffing, and ethics.

What would this look like? Well, in keeping with the ethos of the statement, a culturally competent evaluator assessing a drug prevention program designed for adolescents of color might acknowledge the cultural differences within as well as between groups. So too, the evaluator might recognize that these differences might be a result of years of power differentials and stratifications within the community. The evaluator might also recognize her/his own cultural and/or professional privilege and acknowledge that those biases and stereotypes might affect the quality of the evaluation design, data collection methods, and reporting. Finally, in conceding the above mentioned aspects, the evaluator striving for cultural competence would make use of methods and ethical principles that accurately address the cultural context, and employ staff and key informants who can provide a cultural lens by which to view the multitude of effects of the program (while admitting that this lens may be only one of a few). This is but one broad and overarching example of course, but it provides a backdrop for which the statement was written.

The statement’s closing states that cultural competence in evaluation is an on-going conversation with and among one’s colleagues and within one’s self. In striving for cultural competence, evaluators open themselves up to new horizons and broad ways of thinking.  Admittedly the statement is meant to be representative of the times and context during which it was written; thus the expectation is that it is a “living document,” designed to be updated to reflect the cross-sectional times in which it lives.  The statement “went live” on AEA’s website in April 2011 with several web resources added to supplement, and these are available to the public. The resource section is dynamic; evaluators are encouraged to add relevant and helpful readings or materials at any time.

Additional Resources:

Please see the links below for more information concerning AEA’s Public Statement on Cultural Competence in Evaluation.

About Katrina Bledsoe:  Dr. Katrina L. Bledsoe is a research scientist and senior evaluator at the Education Development Center, Inc. in Washington D.C. Dr. Bledsoe has over 16 years of experience leading research and evaluation projects at both local and federal levels. She has worked in community-based settings with cultural communities and has used theory-driven as well as participatory evaluation approaches. Dr. Bledsoe’s current work includes education, suicide prevention, and safety efforts in domestic settings and addresses adolescent physical and mental health issues and social outcomes for communities of color. She was previously project director of the federal Substance Abuse and Mental Health Services Administration’s national evaluation of the Children’s Mental Health Initiative. Dr. Bledsoe is the author of book chapters and articles in leading journals such as the American Journal of Evaluation and the Journal of Cultural Diversity: An Interdisciplinary Journal. Her recent work is forthcoming in the edited volume Qualitative Inquiry in the Practice of Evaluation. Dr. Bledsoe has been active in the American Evaluation Association for over 15 years and currently serves on the Executive Board. She recently served on the task force that developed AEA’s public statement on cultural competence in evaluation and co-chairs the Graduate Education Diversity Internship.


Rate:
Views: 2589
Your Rating: 0.0   Average Rating: 4.8   Ratings: 5
Katelyn Mack
Reply | Report
0 Points
Katrina, thank you for your thoughtful post. It was encouraging to read about the development of a statement on cultural competency that all evaluators can look to when developing evaluations that cross cultural lines. As an evaluator, it is common to desire objectivity and ignore the bias that we might bring to our work. However this document signals to me that there is an increasing recognition that we all have a particular lens through which we view the world and how we judge what we see. When evaluating initiatives that address complex social issues, I am often are confronted with a variety of sometimes competing lenses through which we can the issue at hand. Keeping in mind the four "key aspects" of a culturally competent evaluator can help anyone - myself included - take a good step toward more deeply applying these principles in our practice.
Katrina Bledsoe
Reply | Report
0 Points
Thank you for those comments Katelyn. I often struggle with the idea of objectivity in my thinking and practice. After all, we’d like to think that the methods, designs, and ethical procedures we use are above bias, are cross-cultural, and can be used in any setting. What I often come back to is standards. Whose standards are we using to determine objectivity? Who determined that a particular approach or method is the best way to practice and/or to generate credible evidence?

I usually remind myself of this: we all have values and backgrounds that influence our everyday living as well as the kind of work we choose, the questions we want to answer, and the manner in how we will go about answering them. The hard part is not to assume everyone values the ways in which we go about our work in the same manner. The even harder part is not to judge people as inferior or dismiss their experiences/ways of thinking as having little value. A tall order for a field that values---judgment.
       1       
Post Your Comment  
Comments are moderated and will be displayed after
approval.
 
* Name  
 (Your name will appear with your post.)
* Email  
 (Your email is required, but it will not be posted.)
* Comments:
CAPTCHA image
* Enter the code shown above:

Follow Us:

Facebook Twitter YouTube LinkedIn RSS