EMERGING DIALOGUES IN ASSESSMENT

Higher education assessment reflects values through choices of measures, strategies, samples, and analyses. Conventional practices can (intentionally or unintentionally) perpetuate racist or culturally exclusive values or unjust biases. For example, utilizing assignments where materials are relatable to or assume prior knowledge of dominant students, and/or utilizing assessments that favor Western, Anglo, and European practices may impact non-dominant student success. Using assessment to support diversity, equity, and inclusion presents a grand challenge to assessment professionals today and will require broad and collective cooperation among members of the assessment and higher education community (Singer-Freeman & Robinson, 2020a, 2020b). To meet the daunting challenges, we propose assessment professionals use an ethical reasoning strategy titled the Eight Key Questions (8KQ) (Fulcher, Ames, & Hawk, 2018) when adopting more culturally responsive and socially just assessment practices.

Culturally responsive assessment involves maintaining sensitivity to students’ cultures and backgrounds throughout all parts of the assessment process and acknowledging that some groups are privileged over others (Montenegro & Jankowski, 2017, 2020). Socially just assessment builds upon culturally responsive assessment to advance social justice by considering the ways in which power structures and biases influence assessment processes. The aim is to "challenge the status quo; raise questions [emphasis added] of privilege, power, and oppression; and work to remedy injustices whether purposeful or accidental" (Montenegro & Jankowski, 2020, p. 9).

Using the 8KQ heuristic focuses attention on shared human values necessary for culturally responsive and socially just assessment plans. The following section defines each of the key questions, provides example questions for key components of the assessment cycle, and concludes with a tool for institutions to use in their own practice. We propose that program faculty, student affairs professionals, and assessment committees could apply the 8KQ to each phase of their assessment process. We hope this encourages others to ask questions that help expose biases and nurture cultural responsiveness and social justice in assessment planning and practices.

8KQ Across the Assessment Cycle

Fairness: How can we act justly and equitably, balancing all interests?

• Student Learning Outcomes. Does the curriculum reflect an unfair power structure? Do all groups of students have comparable opportunity to learn?
•    
• Measures, Data Collection, Analysis, and Reporting. When disaggregating and reporting data, are all groups fairly represented without comparison to a privileged group?
•    
• Use of Results for Improvement. Are program changes accessible and equitable to all groups?

Outcomes: What possible actions achieve the best short-term and long-term outcomes?

• Student Learning Outcomes. Do student learning outcomes reflect skills that best serve the least advantaged students?
•   
• Measures, Data Collection, Analysis, and Reporting. How might data reporting privilege one group over another?
•   
• Use of Results for Improvement. Will reported results inform equitable improvement efforts?

Responsibility: What duties and/or obligations apply?

• Student Learning Outcomes. What responsibilities do we have to evaluate and dismantle learning outcomes that perpetuate privilege?
•   
• Measures, Data Collection, Analysis and Reporting. If disaggregated data reveal inequities, who is responsible for reporting and acting?
•   
• Use of Results for Improvement. If results indicate disparate findings across groups, what responsibilities do we have to tailor programming to the needs of each group?

Character: What actions help me become my ideal self?

• Student Learning Outcomes. Are our learning outcomes characteristic of a socially just institution (e.g., free from bias and not narrowly representative of the dominant power structure)?
•   
• Measures, Data Collection, Analysis, and Reporting. Does our reporting reflect our shared values?
•   
• Use of Results for Improvement. Have I honestly reported unfavorable findings?

Liberty: How do we show respect for personal freedom, autonomy, and consent?

• Student Learning Outcomes. Do student learning outcomes reflect the voices, needs, and interests of all groups (e.g., indigenous ways of knowing)?
•    
• Measures, Data Collection, Analysis, and Reporting. Are students offered informed consent prior to data collection, allowing freedom to decline participation?
•   
• Use of Results for Improvement. Have a diverse group of students had a say in programming changes?

Empathy: How would we act if we cared about all involved?

• Student Learning Outcomes. Do we address the challenges faculty members have with writing inclusive learning outcomes?
•   
• Measures, Data Collection, Analysis, and Reporting. Are disaggregation methods used in a manner such that small population learning experiences are considered?
•   
• Use of Results for Improvement. Do we consider the feelings/perspectives of all affected groups when making program changes?

Authority: What do legitimate authorities expect?

• Student Learning Outcomes. Are expectations for student knowledge, skills, and attitudes informed by multiple stakeholders?
•   
• Measures, Data Collection, Analysis, and Reporting. When creating assessment plans, are the perspectives of authorities from a variety of backgrounds represented? Are alternative paradigms considered?
•   
• Use of Results for Improvement. Could we benefit from broadening our view of the current power structure at our institution when making program changes?

Rights: What rights, if any (legal, innate, social), apply?

• Student Learning Outcomes. Whose right is it to develop student learning outcomes?
•   
• Measures, Data Collection, Analysis, and Reporting. When reporting findings, do we maintain all groups’ right to dignity and privacy?
•   
• Use of Results for Improvement. When making program changes, have we considered the rights of all groups?

8KQ Tool for Use with Groups

These example questions are neither comprehensive nor exhaustive when confronting inequities embedded in current educational systems. We encourage others to generate their own questions. Ideally, this process is conducted within groups, encouraging divergent and diverse viewpoints.

Nearly everyone adopts their own default ethical values. One person may tend toward responsibility, another empathy, and yet another to fairness. Generating 8KQ in groups, especially those with divergent perspectives, helps to reveal biases in our own thinking.

The following handout provides a list of the 8KQ with space for participants to generate their own questions. We recommend using a think-pair-share approach in which the 8KQ are first presented. Participants then spend time writing out their own questions, followed by sharing with a partner, and subsequently by sharing with a larger group. Additional materials and a video explaining the 8KQ may be found at https://www.jmu.edu/ethicalreasoning/8-key-questions.shtml (opens in new window) and freely used for 8KQ sessions. Practicing the 8KQ ethical reasoning strategy offers a collaborative and practical method for assessment professionals to encourage all to respond to otherwise unnoticed biases and injustices. We offer it as a pragmatic approach for exploring concerns about diversity, equity, and inclusion in assessment practices.

References