The Implementing Change Working Group is led by two co-chairs: Marco Molinaro, Executive Director for Educational Effectiveness and Analytics at the University of Maryland, and Martha Oakley, a professor in the Department of Chemistry at Indiana University. To learn more about the efforts of Implementing Change, I spoke with Molinaro about his experiences with the Working Group. Molinaro recalls conversations with SEISMIC Director Tim McKay early on in the collaboration, regarding uses for equity data and involving administrators in SEISMIC. While the Measurements Working Group is focused on the “nitty-gritty” data work, Implementing Change was designed to be more big picture, raising awareness of findings from data analysis, Molinaro says. He has helped lead the group in highlighting inequities in higher education STEM courses and urging institutional leaders to take action.

Molinaro leads Implementing Change’s “Building Awareness and Empathy” project, which aims to demonstrate the ubiquity of inequitable outcomes for students of different races and socioeconomic statuses in introductory STEM courses. Five of SEISMIC’s member institutions–UC Irvine, Indiana University, University of Michigan, UC Davis, and UC Santa Barbara–contributed data to explore the intersection of identities, experiences, and opportunities and the impacts these factors have on college outcomes. They found that students with these factors, such as being a first-generation college student, a racial minority, and/or being low-income, were more likely to experience inequities and thus disparities in course performance and college success as a whole. Additionally, the disparity was worse for Black and Hispanic students.

“It’s often a shock to people as to how much [this data] true,” says Molinaro, who presented the findings alongside Sehoya Cotner, Michael Dennin, Dennis Groth, Becky Matz, and Tim McKay at the 2020 AAU STEM Network Conference. The group also emphasized that there was more work to be done: educational researchers must investigate if these patterns hold across systems, what learning environments and course structures lead to fewer equity gaps, and how we can craft stories to inspire change. SEISMIC Scholar Michael Johnson III investigated some of these questions with his poster “Exploring and Supporting Equitable Policies from the Campus to the Classroom”, where he looked at general university policies, placement exam policies, and supporting faculty with implementing equitable practices. He worked to develop one-pagers to be distributed to instructors with recommendations for more equitable grading policies such as “What Does Extra Credit Measure?” and “What is Minimum Grading?” All of these efforts are critical to creating change at the institutional level. “We’ve created a system that perpetuates prior opportunity, even while we make the argument that our systems are here to equalize opportunity for people,” says Molinaro.

“Tools that Influence Introductory STEM Structures and Beyond” is another one of the Implementing Change projects, investigating campus tools that support a system of change. Available tools often shape both instruction and the student experience, as they influence methods of evaluation, communication, and student support. One tool project members looked at was the Know Your Students tool at UC Davis, which allows instructors to better understand their student population and foster discussion. Additionally, members analyzed at ECoach, a student-directed, personalized coaching tool and research platform founded at the University of Michigan that allows tailored communication and interventions for students. Many also ended up adopting ECoach for their own classrooms, as research shows that ECoach’s psychosocial profiles and personalized messaging is effective at improving course performance (Matz et al., 2021). SEISMIC Scholar Anna Rickabaugh also gave a poster presentation on the tools project, giving an overview of the tools that the project is looking at.

Despite the group’s progress toward sharing equity data to catalyze change, the journey has not been without challenges. “Higher education is incredibly stable. And with what we’re trying to do, that’s not a good thing,” Molinaro says. While Implementing Change has created a more welcoming environment for equity discussions, identifying who can create change at the departmental or institutional level is a separate challenge. There are curriculum committees, program review systems, and even state legislatures that all must be convinced that change is needed. Still, it’s a fight that Molinaro, Oakley, and many others have taken on. With the important work that Implementing Change has, critical data and effective presentations can accelerate administrative buy-in and amplify insights from the other Working Groups

Co-chairs Vanessa Woods and Mike Wilton both joined WG2 because they were interested in applying interventions to their own classrooms. Woods and Wilton are associate teaching professors at the University of California Santa Barbara, with Woods working within the Department of Psychological & Brain Sciences and Wilton working within the Department of Molecular, Cellular, and Developmental Biology. I recently had the chance to catch up with Woods and Wilton, reflecting on WG2’s progress over the past few years and their perspectives as co-chairs. WG2, like Working Groups 1 and 3, was established at the 2019 SEISMIC Summer Meeting to help organize and encourage collaborative efforts. “We were perceived to be those that were gonna go in and tinker something in their classroom and see the impacts on student outcomes,” Wilton says.

In addition to establishing the Experiments Working Group at the Summer Meeting in 2019, four project ideas were proposed and taken on by participants. One of these was the Access to Practice (AtP) project, originally coordinated by Maggie Safronova and Linda Adler-Kassner. AtP involves faculty designing low-stakes writing assignments that students will peer-review in order to improve their learning in the course. “Instructors, as experts, have ways that they think and talk and they don’t actually explain those ways very well,” Woods says. “Students […] don’t really have those same structures and understanding of all of those norms. That’s why we call it Access to Practice – how do you give students access to how the discipline talks?” As many say, practice makes perfect.

Access to Practice also capitalizes on how the process of writing, reviewing, and revising improves students’ understanding of concepts and conventions. “It allows students to engage with knowledge in a way that helps them to tackle hard concepts,” Wilton says. He gives the example of his students being tasked with explaining a child’s genetic abnormality to their parents: “They have to write their answer in around 400 words, and that gets reviewed by two of their peers.” This communal learning exercise, which has both prompts and rubrics designed by faculty and project leadership together, has had a surprising finding: students seem to learn more by reviewing others than by being reviewed themselves.

In addition to the structured peer review exercises improving student learning and success, preliminary data shows that participation in the written exercises can increase feelings of disciplinary belonging in students. Faculty at UCSB have the opportunity to participate in the intervention, and the AtP leadership team has worked hard to share the impactfulness of their peer review exercises through publications (e.g., Woods et al., 2021) and presentations. “We’ve done lots of workshop presentations for instructors about how to create structured peer reviews,” Woods says. Outside of local UCSB meetings, Woods and Safronova presented at SABER West 2023, where their theme was Supporting Equitable Transitions in STEM Education.

Another one of WG2’s core projects is the Office Hours Project, led by Lalo Gonzalez, also an associate teaching professor at UCSB. Those within the project have worked hard to identify best practices associated with office hours and collect data through faculty surveys, student surveys, and an “office hours tracker” that documents the characteristics of students attending office hours. Characteristics include categories such as race, ethnicity, gender, first-generation status, and academic standing.

STEM instructors can investigate what is happening in their own classroom by collecting data and working with Office Hours leadership to interpret their findings. Wilton recalls that Gonzalez and a few others acted as mentors, helping instructors develop their education research skills by qualitatively coding survey responses and identifying themes. One trend they observed: first-generation and racially minoritized students were not (or rarely) attending office hours.

Wilton shared that to address these results, he and other instructors modified course syllabi to detail how their courses are designed to support students with learning, and how this support includes office hours. The syllabi also include an explanation of office hours norms, which, for Wilton and his colleagues, have shifted from one-on-one conversations in offices to open rooms with tables where students can work in groups. Finally, those who participate in the Office Hours Project normalize struggling and assure students that it’s a part of learning. Because office hours can seem scary and intimidating, these instructors have taken steps to make it a friendlier environment and lower potential barriers to attending.

Co-chairs Becky Matz and Stefano Fiorini have worked hard to organize and support the group’s efforts. “I would say that WG1’s greatest success is the new relationships among staff researchers, faculty, postdocs, and students across SEISMIC’s institutions,” says Matz. “There exists now a bigger and stronger network of folks who can do cross-institutional quantitative work with student records data, and I think each of us in that network has a good sense of the motivations, capabilities, and strengths of the others.” Both co-chairs attribute much of WG1’s success to the involvement of key members and are proud of how WG1 as a whole has contributed to the SEISMIC community. Moreover, Matz and Fiorini are confident WG1 has made important progress toward addressing inequities and structural issues that exist on SEISMIC campuses through the variety of studies conducted.

In 2021, WG1 developed a Fellowship program that allowed graduate students to participate in SEISMIC research during the summer, providing them with funding to support their work. Outside of supporting the Collaboration’s goals of improving diversity, equity, and inclusion in STEM courses, the Fellowship program gave WG1 the opportunity to build and expand the pool of future academics with an understanding of the issues that were at play. “That was extremely exciting because we could bring in students and give them an experience that contributed to their career prospects,” Fiorini recalls.

Working Group 1 has nurtured multiple projects to work towards its mission of identifying and evaluating measurements of inequities in STEM courses. One of the working group’s efforts explores the effects of student demographics in introductory STEM courses. By conducting parallel data analyses across the SEISMIC institutions, participants worked to characterize how the complex identities of students interact with STEM learning environments. The group formalized the Systemic Advantage Index, a scale that indicates the advantages that characterize students within institutions according to race, gender, socioeconomic status, and first-generation status. Using this, Sarah Castle and other members of WG1 explored multi-institutional practices for mapping systemic advantages within STEM courses (Castle et al., 2021). Another paper concerning the impact of systemic advantage on student outcomes is currently in review, and will hopefully be published soon. 

Another WG1 project focused on the impact that Advanced Placement (AP) course credit has on student success in introductory science courses. Christian Fischer and other members of WG1 identified that institutional and departmental policies concerning AP credit varied widely. Additionally, the group found that there was variety in the way students approached using AP credit to skip courses (Fischer et al., 2023).

Working Group 1’s progress was not without challenges, however. Often, the data they were primarily working with was institutional data, meaning information about the courses and their students was collected by the universities themselves. This frequently left many unknown elements when looking at data, such as specific student identities and other unique student experiences, leading WG1 members to wonder, “How can we describe inequities in an effective, meaningful, and actionable way?” 

To address this in part, WG1 consulted with members of Working Group 4: Constructs (WG4), who were familiar with applying new frameworks and perspectives to STEM education research. “They provided us with food for thought [and] and ways of moving this work forward while accounting for the limitations of our data,” Fiorini explains. WG4 assisted WG1 with integrating critical approaches into quantitative STEM equity working, investigating systemic inequities present in higher education and their historical roots. Pearson et al. (2022) is a great example of the two working groups collaborating to inform future analyses concerning systemic inequity in STEM fields.

Matz and Fiorini are hopeful that WG1 will continue to develop the knowledge and skills needed to work towards changing inequitable practices and adverse structural elements, even after the end of SEISMIC 1.0. The two also have future topics they’d like for the group to explore, including analyzing data generated from learning management systems and using WG1’s strong relationships to do research that impacts an even broader audience of researchers, administrators, and faculty.

Currently, the SEISMIC Newsletter has 726 subscribers and has released a total of 57 monthly issues. For almost five years, members of SEISMIC and other interested individuals have been receiving information about current Collaboration happenings including upcoming events, featured participants, and project updates. However, the SEISMIC Newsletter had humble beginnings: the first issue published in March of 2019 had just 64 recipients.

SEISMIC Project Manager Nita Tarchinski has been in charge of creating the monthly Newsletters since the beginning. She recalls that it was originally SEISMIC Director Tim McKay’s idea for a Newsletter in order to provide the collaboration with a regular communication mechanism. “No matter what, you were going to hear from us once a month, and that seemed like a good frequency to keep people engaged,” Nita explains. SEISMIC’s Newsletter is also open to those who are not at participating institutions but are interested in the type of work that SEISMIC does. Many of SEISMIC’s virtual talks are open to the public, welcoming those from outside the 10 member institutions.

Aside from regular updates related to the Collaboration itself, another regular component of the monthly Newsletters is the SEISMIC Voices series. Since the beginning, SEISMIC Voices has aimed to both help bring the community together and make members feel valued. “We were starting the community from zero. We were trying to make people feel seen,” Nita says. “Every month you’re seeing someone new and seeing another way that people have been involved.” Nita has been intentional about featuring a diverse set of voices, inviting people from different universities and disciplines, people who have held different institutional roles, and people who have been involved in SEISMIC in different ways. She has also tried to ensure that the Voices highlight SEISMIC’s gender and racial diversity.

In addition to showcasing SEISMIC’s members, Nita hopes that participating in SEISMIC Voices can act as a recommitment to the Collaboration: “It’s like this affirmation to them that we see them as a contributing member,” Nita explains. She hopes that by reflecting on their experiences, featured members are reminded of what they’ve enjoyed about the collaboration, and hopefully want to do more.

The SEISMIC Newsletter also has other uses, such as allowing members to share their own events and advertising swag created by SEISMIC Central. Moreover, the Newsletter has been used to gather feedback, both formally and informally. There have been times when members have replied to the Newsletters Nita sends out, cheering SEISMIC on and celebrating accomplishments that have been shared. “I’ve been so touched by that,” Nita says. Additionally, last month’s Newsletter featured a poll allowing members to vote for how SEISMIC should use its remaining funds. While the results didn’t show an immediate “winner”, members’ opinions will be considered when discussing plans for SEISMIC 2.0.

With many of SEISMIC’s projects winding down, Nita believes that next year could be a great opportunity to share lessons learned from SEISMIC 1.0, especially as SEISMIC Central and The Collaboration Council discuss what SEISMIC 2.0 will look like. This would involve a shift in the balance of content, involving less updating and more learning. While the Reflective Waves series supports this, Nita is also interested in sharing lessons directly from Working Groups and other teams.