Reflective Waves: Creating and Applying Classroom Interventions with the Experiments Working Group

By Ashley Atkinson

Edited by Nita Tarchinski

Continuing with our Reflective Waves series, where we showcase important SEISMIC efforts and initiatives that have taken place over the past five years, Working Group 2 (WG2) takes the spotlight. Also known as the Experiments Working Group, WG2 focuses on classroom interventions that seek to understand disparities and foster equity across multiple disciplines and universities. Throughout the duration of SEISMIC, over 70 members have been involved with WG2, supporting multiple projects and giving workshops, poster presentations, and talks at both SEISMIC and external events.

Co-chairs Vanessa Woods and Mike Wilton both joined WG2 because they were interested in applying interventions to their own classrooms. Woods and Wilton are associate teaching professors at the University of California Santa Barbara, with Woods working within the Department of Psychological & Brain Sciences and Wilton working within the Department of Molecular, Cellular, and Developmental Biology. I recently had the chance to catch up with Woods and Wilton, reflecting on WG2’s progress over the past few years and their perspectives as co-chairs. WG2, like Working Groups 1 and 3, was established at the 2019 SEISMIC Summer Meeting to help organize and encourage collaborative efforts. “We were perceived to be those that were gonna go in and tinker something in their classroom and see the impacts on student outcomes,” Wilton says.

WG 2 Co-Chair Mike Wilton

WG 2 Co-Chair Vanessa Woods

In addition to establishing the Experiments Working Group at the Summer Meeting in 2019, four project ideas were proposed and taken on by participants. One of these was the Access to Practice (AtP) project, originally coordinated by Maggie Safronova and Linda Adler-Kassner. AtP involves faculty designing low-stakes writing assignments that students will peer-review in order to improve their learning in the course. “Instructors, as experts, have ways that they think and talk and they don’t actually explain those ways very well,” Woods says. “Students […] don’t really have those same structures and understanding of all of those norms. That’s why we call it Access to Practice – how do you give students access to how the discipline talks?” As many say, practice makes perfect.

Access to Practice also capitalizes on how the process of writing, reviewing, and revising improves students’ understanding of concepts and conventions. “It allows students to engage with knowledge in a way that helps them to tackle hard concepts,” Wilton says. He gives the example of his students being tasked with explaining a child’s genetic abnormality to their parents: “They have to write their answer in around 400 words, and that gets reviewed by two of their peers.” This communal learning exercise, which has both prompts and rubrics designed by faculty and project leadership together, has had a surprising finding: students seem to learn more by reviewing others than by being reviewed themselves.

In addition to the structured peer review exercises improving student learning and success, preliminary data shows that participation in the written exercises can increase feelings of disciplinary belonging in students. Faculty at UCSB have the opportunity to participate in the intervention, and the AtP leadership team has worked hard to share the impactfulness of their peer review exercises through publications (e.g., Woods et al., 2021) and presentations. “We’ve done lots of workshop presentations for instructors about how to create structured peer reviews,” Woods says. Outside of local UCSB meetings, Woods and Safronova presented at SABER West 2023, where their theme was Supporting Equitable Transitions in STEM Education.

Office Hours Project Leader Lalo Gonzalez

Another one of WG2’s core projects is the Office Hours Project, led by Lalo Gonzalez, also an associate teaching professor at UCSB. Those within the project have worked hard to identify best practices associated with office hours and collect data through faculty surveys, student surveys, and an “office hours tracker” that documents the characteristics of students attending office hours. Characteristics include categories such as race, ethnicity, gender, first-generation status, and academic standing.

STEM instructors can investigate what is happening in their own classroom by collecting data and working with Office Hours leadership to interpret their findings. Wilton recalls that Gonzalez and a few others acted as mentors, helping instructors develop their education research skills by qualitatively coding survey responses and identifying themes. One trend they observed: first-generation and racially minoritized students were not (or rarely) attending office hours.

Wilton shared that to address these results, he and other instructors modified course syllabi to detail how their courses are designed to support students with learning, and how this support includes office hours. The syllabi also include an explanation of office hours norms, which, for Wilton and his colleagues, have shifted from one-on-one conversations in offices to open rooms with tables where students can work in groups. Finally, those who participate in the Office Hours Project normalize struggling and assure students that it’s a part of learning. Because office hours can seem scary and intimidating, these instructors have taken steps to make it a friendlier environment and lower potential barriers to attending.

Perry Samson, an Arthur F. Thurnau Professor of Atmospheric Science at the University of Michigan, has led two different projects within WG2. The Backchannel Project seeks to evaluate if a backchannel – a way for students to ask questions anonymously during class and receive answers in real time – increases students’ sense of belonging. As being called on in a classroom or visiting office hours can be intimidating, oftentimes students will resign to having their questions unvoiced or seek the answers elsewhere. This can lead to lower academic success for students who are unable to receive clarification. To address this, Samson’s backchannel allows students to ask questions through their phone or laptop anonymously during lectures and graduate teaching assistants address the question online. Not only does this allow students to get their questions answered, but students can also see they’re not alone in having questions. Woods reports the project has found that underrepresented students utilize backchannels more than raising their hand, or other non-anonymous forms of communication.

Backchannel and CLUE Project Leader Perry Samson

Samson’s second project, Contextual Linkaging for Undergraduate Education (CLUE), uses Artificial Intelligence to transcribe, annotate, index, and take notes on recorded lectures from instructors. “It’s almost like having bookmarks,” Wilton says. Students who miss class or want to review a lecture’s content can re-watch the recording entirely but also use the added indexes, notes, keywords, and additional resources to enhance their studying process. Both of Samson’s projects received NSF funding (NSF2013316, NSF2016421) and have been featured in several articles (Koenig, 2019 & Priebe, 2022), extending the impact of these projects to wider contexts and ultimately impacting more students.

Both Wilton and Woods are thrilled with the work of WG2. “A strength of Working Group 2 is that you see the direct impacts of the work you’re doing,” Wilton says. “It’s like looking under the microscope.” In addition to the successes of WG2’s projects, the co-chairs have come away with important lessons for managing multi-institutional, collaborative work. First, regular communication is critical and must occur throughout the working group: between participants in the same project, between leadership in different projects, between project leadership and co-chairs, and between co-chairs and SEISMIC Central – SEISMIC’s administrative team. Keeping folks in the loop, alongside having people hold organizational roles, ensures that projects stay on track and deadlines are met. On a similar note, meeting in person (like at writing retreats, summer meetings, or the Weeks of SEISMIC), was incredibly helpful for re-energizing project groups and making large pushes toward goals.

This last year of SEISMIC is bittersweet for WG2: some projects are winding down, and others are growing to become self-sustainable, no longer needing as much support from SEISMIC. However, we are immensely proud of the many important interventions developed and implemented throughout the years. “I really do hope that the working groups stay as a collaborative project that keeps going,” Woods says, thinking about a potential SEISMIC 2.0. Indeed, there is still plenty of work to be done and interventions to be assessed when looking at equity and inclusion in STEM courses. However, regardless of what the collaboration looks like in the future, SEISMIC has given multiple experiments the push they need to hit the ground running.

 

Ashley Atkinson

Ashley Atkinson is a Program Assistant for SEISMIC Central, ensuring that SEISMIC initiatives have the help they need to run smoothly. Her primary responsibilities include maintaining the SEISMIC website, managing the Newsletter, and supporting projects. As an alumnus of Michigan State University, Ashley is passionate about equity and inclusion in STEM alongside science communication. She is currently pursuing an MA in Science Writing and Johns Hopkins University.

 

 

 

 

Reflective Waves: Investigating and Identifying Inequities in STEM Courses with the Measurement Working Group

By Ashley Atkinson

Early on, SEISMIC established four Working Groups to promote collaborative work across member institutions and encourage participants to apply their expertise to topics of interest to them. In this issue of Reflective Waves, we will be highlighting the work and successes of Working Group 1: Measurement (WG1). WG1 was founded with the primary goals of establishing metrics for measuring equity and inclusion in introductory STEM courses, conducting measurements, and identifying actionable data to promote change.

Throughout the past five years, over 70 people have been involved with the Measurement Working Group. This has resulted in the publication of six papers, with four more on the way. In addition, WG1 members have given several presentations on their measurement work at a variety of events such as The Weeks of SEISMIC, our Summer Meetings, and academic conferences such as the American Educational Research Association events.

WG 1 Co-Chair Becky Matz

Co-chairs Becky Matz and Stefano Fiorini have worked hard to organize and support the group’s efforts. “I would say that WG1’s greatest success is the new relationships among staff researchers, faculty, postdocs, and students across SEISMIC’s institutions,” says Matz. “There exists now a bigger and stronger network of folks who can do cross-institutional quantitative work with student records data, and I think each of us in that network has a good sense of the motivations, capabilities, and strengths of the others.” Both co-chairs attribute much of WG1’s success to the involvement of key members and are proud of how WG1 as a whole has contributed to the SEISMIC community. Moreover, Matz and Fiorini are confident WG1 has made important progress toward addressing inequities and structural issues that exist on SEISMIC campuses through the variety of studies conducted.

In 2021, WG1 developed a Fellowship program that allowed graduate students to participate in SEISMIC research during the summer, providing them with funding to support their work. Outside of supporting the Collaboration’s goals of improving diversity, equity, and inclusion in STEM courses, the Fellowship program gave WG1 the opportunity to build and expand the pool of future academics with an understanding of the issues that were at play. “That was extremely exciting because we could bring in students and give them an experience that contributed to their career prospects,” Fiorini recalls.

WG 1 Co-Chair Stefano Fiorini

Working Group 1 has nurtured multiple projects to work towards its mission of identifying and evaluating measurements of inequities in STEM courses. One of the working group’s efforts explores the effects of student demographics in introductory STEM courses. By conducting parallel data analyses across the SEISMIC institutions, participants worked to characterize how the complex identities of students interact with STEM learning environments. The group formalized the Systemic Advantage Index, a scale that indicates the advantages that characterize students within institutions according to race, gender, socioeconomic status, and first-generation status. Using this, Sarah Castle and other members of WG1 explored multi-institutional practices for mapping systemic advantages within STEM courses (Castle et al., 2021). Another paper concerning the impact of systemic advantage on student outcomes is currently in review, and will hopefully be published soon. 

Another WG1 project focused on the impact that Advanced Placement (AP) course credit has on student success in introductory science courses. Christian Fischer and other members of WG1 identified that institutional and departmental policies concerning AP credit varied widely. Additionally, the group found that there was variety in the way students approached using AP credit to skip courses (Fischer et al., 2023).

Working Group 1’s progress was not without challenges, however. Often, the data they were primarily working with was institutional data, meaning information about the courses and their students was collected by the universities themselves. This frequently left many unknown elements when looking at data, such as specific student identities and other unique student experiences, leading WG1 members to wonder, “How can we describe inequities in an effective, meaningful, and actionable way?” 

To address this in part, WG1 consulted with members of Working Group 4: Constructs (WG4), who were familiar with applying new frameworks and perspectives to STEM education research. “They provided us with food for thought [and] and ways of moving this work forward while accounting for the limitations of our data,” Fiorini explains. WG4 assisted WG1 with integrating critical approaches into quantitative STEM equity working, investigating systemic inequities present in higher education and their historical roots. Pearson et al. (2022) is a great example of the two working groups collaborating to inform future analyses concerning systemic inequity in STEM fields.

Matz and Fiorini are hopeful that WG1 will continue to develop the knowledge and skills needed to work towards changing inequitable practices and adverse structural elements, even after the end of SEISMIC 1.0. The two also have future topics they’d like for the group to explore, including analyzing data generated from learning management systems and using WG1’s strong relationships to do research that impacts an even broader audience of researchers, administrators, and faculty.

Because of the efforts of dedicated co-chairs and motivated members, Working Group 1 has advanced the understanding of equity issues in STEM introductory courses, identifying and developing metrics alongside analyzing available data. They are continuing to apply pressure to the higher education system, pursuing new projects and sharing what they’ve learned. The fire they’ve lit under all of us involved in higher education will continue to burn, even beyond the conclusion of SEISMIC.

 

Ashley Atkinson

Ashley Atkinson is a Program Assistant for SEISMIC Central, ensuring that SEISMIC initiatives have the help they need to run smoothly. Her primary responsibilities include maintaining the SEISMIC website, managing the Newsletter, and supporting projects. As an alum from Michigan State University, Ashley is passionate about equity and inclusion in STEM alongside science communication. She is currently pursuing an MA in Science Writing and Johns Hopkins University.

 

 

 

 

Reflective Waves: Behind the Scenes of our Collaboration’s Newsletter

By Ashley Atkinson

With funding for The SEISMIC Collaboration being extended for another year, this is a great opportunity to reflect on past SEISMIC initiatives, synthesizing what we’ve learned and thinking about what we want both SEISMIC 2.0 and our last year of SEISMIC 1.0 to look like. To take advantage of this time, SEISMIC Central is beginning a “Reflective Waves” article series where previous SEISMIC efforts are revisited and explored. The goal is to interview members tied to these efforts to gain additional perspectives on projects. We are hopeful that this series will cover a variety of SEISMIC initiatives including the Working Groups, the Office Hours Podcast, SEISMIC Scholars, the Weeks of SEISMIC, and the Collaboration Council. The first and longest standing SEISMIC initiative we are reflecting on, however, is our Newsletter.

SEISMIC Project Manager Nita Tarchinski

Currently, the SEISMIC Newsletter has 726 subscribers and has released a total of 57 monthly issues. For almost five years, members of SEISMIC and other interested individuals have been receiving information about current Collaboration happenings including upcoming events, featured participants, and project updates. However, the SEISMIC Newsletter had humble beginnings: the first issue published in March of 2019 had just 64 recipients.

SEISMIC Project Manager Nita Tarchinski has been in charge of creating the monthly Newsletters since the beginning. She recalls that it was originally SEISMIC Director Tim McKay’s idea for a Newsletter in order to provide the collaboration with a regular communication mechanism. “No matter what, you were going to hear from us once a month, and that seemed like a good frequency to keep people engaged,” Nita explains. SEISMIC’s Newsletter is also open to those who are not at participating institutions but are interested in the type of work that SEISMIC does. Many of SEISMIC’s virtual talks are open to the public, welcoming those from outside the 10 member institutions.

 

Aside from regular updates related to the Collaboration itself, another regular component of the monthly Newsletters is the SEISMIC Voices series. Since the beginning, SEISMIC Voices has aimed to both help bring the community together and make members feel valued. “We were starting the community from zero. We were trying to make people feel seen,” Nita says. “Every month you’re seeing someone new and seeing another way that people have been involved.” Nita has been intentional about featuring a diverse set of voices, inviting people from different universities and disciplines, people who have held different institutional roles, and people who have been involved in SEISMIC in different ways. She has also tried to ensure that the Voices highlight SEISMIC’s gender and racial diversity.

In addition to showcasing SEISMIC’s members, Nita hopes that participating in SEISMIC Voices can act as a recommitment to the Collaboration: “It’s like this affirmation to them that we see them as a contributing member,” Nita explains. She hopes that by reflecting on their experiences, featured members are reminded of what they’ve enjoyed about the collaboration, and hopefully want to do more.

The SEISMIC Newsletter also has other uses, such as allowing members to share their own events and advertising swag created by SEISMIC Central. Moreover, the Newsletter has been used to gather feedback, both formally and informally. There have been times when members have replied to the Newsletters Nita sends out, cheering SEISMIC on and celebrating accomplishments that have been shared. “I’ve been so touched by that,” Nita says. Additionally, last month’s Newsletter featured a poll allowing members to vote for how SEISMIC should use its remaining funds. While the results didn’t show an immediate “winner”, members’ opinions will be considered when discussing plans for SEISMIC 2.0.

With many of SEISMIC’s projects winding down, Nita believes that next year could be a great opportunity to share lessons learned from SEISMIC 1.0, especially as SEISMIC Central and The Collaboration Council discuss what SEISMIC 2.0 will look like. This would involve a shift in the balance of content, involving less updating and more learning. While the Reflective Waves series supports this, Nita is also interested in sharing lessons directly from Working Groups and other teams.

Moving into our final year, the Newsletter continues to be a vital part of the SEISMIC community. Nita has done a fantastic job of managing and developing the Newsletter, and she has solidified its status as SEISMIC’s most popular way of sharing information. Going forward, SEISMIC Program Assistant Ashley Atkinson will be in charge of developing content for the Newsletter and hopes to continue the momentum that Nita has built. Ashley is welcoming new ideas for the Newsletter and can be reached at atkinash@umich.edu. She hopes that subscribers are excited for the Reflective Waves series!

 

 

Ashley Atkinson

Ashley Atkinson is a Program Assistant for SEISMIC Central, ensuring that SEISMIC initiatives have the help they need to run smoothly. Her primary responsibilities include maintaining the SEISMIC website, managing the Newsletter, and supporting projects. As an alum from Michigan State University, Ashley is passionate about equity and inclusion in STEM alongside science communication. She is currently pursuing an MA in Science Writing and Johns Hopkins University.