EMERGING DIALOGUES IN ASSESSMENT

April 20, 2022

Recent calls from assessment scholars promoting equity have encouraged work in assessment to be "reflective of the students served and to ultimately use assessment data to address learning, persistence, and attainment gaps" (Montenegro & Jankowski, 2020, p. 4). The inaugural Curious Aggies research project at UC Davis was designed, in part, to heed this call and center student voice and experience in research practices in a study of student perceptions of assessment and learning expectations. Our research team, consisting of undergraduate students, an advanced doctoral student, and the Coordinator for Academic Assessment, implemented an inclusive methodological framework to learn how undergraduate students on our campus understand Student Learning Outcomes (SLOs). Our driving research questions included:

1. How do students know what they are expected to learn?
2. How do students find out what they are expected to learn?

To answer these questions, the Curious Aggies project utilized a system of iterative member checking, a process of affirming or validating qualitative research findings with research participants, to include and represent student voices more holistically, following recommendations by assessment scholars to engage student perspectives more actively throughout assessment research (Levy & Heiser, 2018; McArthur, 2017; Montenegro & Jankowski, 2020). Member checking helps researchers to improve findings by addressing implicit researcher biases (Miles & Huberman, 1994) and ensuring that the dominant voice in the research is not solely that of the researcher (Mason, 2002). There are various models of member checking in qualitative research: in some, participants are given drafts of findings and asked for alternate viewpoints (Stake, 1995); in others, multiple participants are asked to check for the accuracy of the findings (Creswell, 2005); and in other cases, entire manuscript drafts are read by participants to corroborate findings (Yin, 2014). However, our project sought to extend the concept of member checking, creating a more iterative process in which participants are member checked at each stage of the research to include and represent student voices more fully.

In our investigation, student researchers contributed to each aspect of the research cycle, including developing the research questions, leading interviews, and writing the analysis of data. By framing our research as a process-oriented conversation with peers (Burke Reifman et al., 2022), we found that participants were inclined to be more comfortable with sharing their experiences. In this project, our undergraduate researchers were able to leverage their understanding of the student experience to more effectively capture the thought processes and experiences of their fellow students, allowing for diverse student voices to be highlighted in research with the hopes of informing more equitable assessment practices going forward.

From Researched to Researchers

The undergraduate researchers on the Curious Aggies project recognized and understood student confusion around SLOs. Mahalia, the team’s original undergraduate researcher, attended a symposium where a student panel answered faculty questions about expectations and SLOs. Mahalia was struck by how professors responded to panelist insight: "They were intrigued and concerned about certain responses, particularly when students claimed to ‘not know’ where or how to find their SLOs when they were unclear." She became interested in learning about how varying student dispositions impact students’ ability to succeed in a highly competitive academic institution. Following this experience, she was determined to extend the conversation to a larger scope of students, which led to the basis of our inquiry.

Leah, another undergraduate researcher, began her involvement in the project as a study participant. She heard about the interviews and was eager to share her thoughts about her classes and learning. Even though she had taken plenty of courses, she had not reflected on whether her professors were fully articulating what she should learn in a course or whether she reached those expectations. Further, she never felt that there was adequate time to evaluate her assessment experience beyond the grade she received because the next set of classes would begin so quickly. When she entered the interview, Leah was pleasantly surprised that she was greeted by a fellow student who was only a few years older. "We chatted about where we were from, our majors, and our love for UC Davis. As he asked me questions regarding my college learning experiences, I was prompted to be introspective about my interactions with professors, my self-agency and ability to advocate for myself, and the purpose of my education. It was comforting to have a natural flowing conversation with another student and refreshing to be discussing a topic that is rarely talked about between students." In general, Leah felt a strong connection to this topic, which drove her to continue the inquiry as a researcher.

Affirming Student Voices through Iterative Member Checking

As undergraduate students collected both survey and focus group data to answer research questions, Mahalia and Leah noticed that the undergraduate researchers and interviewees felt a balanced power dynamic due to the student-to-student interviews with no faculty presence. Because of this, interviewees were exceptionally forthright in sharing experiences with their peers. This dynamic took immense pressure off the interviewees and allowed these moments to become more informal, conversational, and authentic. This was also reflected in conversations with other participants who expressed that they "had never seen a platform like this where they could express their learning from an honest standpoint." Because of this comfort, Mahalia also found that students were able to discuss "personal factors that influence their perception including ethnicity, race, gender identity, socioeconomic status, ability, and so on. In this case and with inclusivity in mind, our focus group interview style served as a platform for highlighting student voices."

As the project continued, Mahalia and Leah found that their personal understandings of the project were further validated as they began working on the analysis of interview transcripts. Specifically, Leah was able to affirm the experiences of other participants: “Like me, other students felt stressed and without purpose when they were unsure about what they were expected to learn. They were also relieved when professors made their learning goals clear and transparent, and I was not alone in sometimes feeling lost on the ‘treadmill of school.’” In reading and listening to the transcripts, the undergraduate student researchers repeatedly affirmed and added nuance to the team’s initial understandings of the student experience, acting as built-in member checks for the team. Mahalia noted that "hearing differing perspectives on learning gave the research team a holistic understanding of what clear teaching practices can look like for all students, and as undergraduate students, we were able to confirm these ideas." Overall, the involvement of students as both researchers and participants pushed our inquiry and analysis, giving immense power to the student voice in research where students deserve to be at the center of consideration.

Recommendations: Accurately Representing the Student

Our participant-researchers demonstrated how student inclusion in research impacts student identity; moreover, it is evident that both the student researchers and participants felt seen in their positions as students and more engaged in their understanding of learning in higher education. Our methodological approach echoes calls for reimagining member checking in qualitative research (Harvey, 2015; Simpson & Quigley, 2016) and works to address calls for meaningful and purposeful inclusion of diverse student voices in research (Montenegro & Jankowski, 2020). It is important that students are co-creators of knowledge, rather than just subjects of analysis or research assistants. By centralizing student voices in higher education research and including them throughout the research process, we can generate a holistic view of student expectations to help minimize the gap between students and faculty, leading to an inclusive learning environment that is mindful of practices that benefit all students.