Building Structural Equity: What do our organizations need?

By Nita Kedharnath and Rachel Renbarger

Background and Context

ASCN and SEISMIC began collaborating in spring 2021 to understand how both organizations could synergistically support STEM education transformation. These collaborative meetings included the project coordinators of both organizations and the authors of this piece, Nita Kedharnath (project manager of SEISMIC) and Rachel Renbarger (research director of ASCN). We started meeting monthly to discuss equity challenges within our organizations, to problem-solve together, and to share resources to ensure equity progress continues for both our groups even while we target different priorities.

ASCN: The Accelerating Systemic Change Network (ASCN) is a community of change agents who create events, community, and resources to improve instruction in higher education. The network, sponsored by grant funders, is free for anyone to join.

In March, the murder of Asian American women working at spas in Atlanta amplified calls for anti-racist actions from institutions in the United States, including institutions of higher education. During one of our collaborative meetings, we asked each other what steps our respective organizations have taken toward racial justice. While both ASCN and SEISMIC had been working on diversity, equity, inclusion, and social justice in STEM education since the organizations’ inceptions, this work was outward-facing and dedicated to helping university faculty members change their institutional settings. What were our organizations’ roles in tackling racial justice? What should we be doing as leaders? This lack of a clear path forward led us to ask these questions to experts who have been doing this reflective work within STEM higher education.

Panel Event

We first began to answer this question through examining the latest report by the American Council on Education (ACE) and talking to experts who were openly leading events centered around racial equity. The ACE report, titled Shared Equity Leadership: Making Equity Everyone’s Work, documented how multiple campus leaders promoted equity by convincing more campus stakeholders to be involved in making change on campus. This report encouraged us to examine our own personal reasons for doing this work, but we felt a disconnect since we were part of organizations rather than campuses. This led us to ask organizational leaders to speak on a panel and describe how they helped their organization do this reflective, internal work as well as the outward, external work.

This panel was titled, “Building Structural Equity: National Networks’ Role in Promoting Justice,” and included one of the writers of the ACE report, Dr. Elizabeth Holcombe, as well as Dr. Shirley Malcom, Director of the STEM Equity Achievement (SEA) Change initiative at the American Association for the Advancement of Science; Dr. Tabbye Chavous, Director for the National Center for Institutional Diversity; and Dr. Stanley Lo, President-Elect for the Society for the Advancement of Biology Education Research (SABER). From the panel event, we learned there were consistent challenges to doing this equity work within organizations such as limited mental and emotional bandwidth, difficulty convincing colleagues to join in the work, frustration with changes that are band-aids rather than systemic changes, and limited resources or support for doing justice-oriented work. However, we also learned what we could do to improve equity within organizations: collect data to understand challenges and successes, listen to those impacted by racial injustice, write grants to fund these initiatives, recognize the work already being done, and create groups of dedicated people to do the work together. These challenges and actions led us to work with our members to look internally to see what each organization should focus on for the next year.

After the Panel

Following the panel event, ASCN and SEISMIC co-hosted a working session for ASCN and SEISMIC members to gather and discuss how the lessons from the panel could be applied to our organizations. We asked members to consider what barriers stood in the way of our organizations’ promotion of justice and what actions they’d like us to take to address those barriers. Some barriers that were shared related to interactions between members – questions of power structures, institutional roles, and whether the ideas of STEM scholars or social science scholars were more valued in the organizations. Another key barrier was the lack of a clear vision for the organizations that centered equity and justice. The fact that most of our members are not paid to do the work of our organizations presented another challenge for our organizations’ ability to act. For actions we could take, several members suggested we hold town hall meetings or other open forums for members to develop shared goals and values for our organizations. Members were also interested in offering invitations to self-work, organizing seminar series, applying for external funding, and creating space for members to get what they need from the organizations, instead of assuming all members need the same thing. ASCN and SEISMIC then conducted individual meetings to create organization-specific equity plans.

During the SEISMIC Summer Meeting, SEISMIC leaders took initial steps towards these actions by hosting a discussion on the vision and goals of SEISMIC. The conversation quickly turned inward, with members asking how SEISMIC can achieve its outward-facing goals while internal structures hinder full participation of members. SEISMIC created an anonymous space for members to highlight structural issues with the collaboration, and then facilitated a discussion to hear from members on specific actions SEISMIC needs to stop and specific ones to start to make the collaboration more inclusive. In response to these difficult conversations, SEISMIC leadership sent out a message articulating how SEISMIC would examine its own structures and make changes, set up a facilitated open forum to hear feedback on the plan, and is now in the process of creating a SEISMIC Task Force to examine existing collaboration structures and recommend new ones. In addition, SEISMIC has created spaces for members to come together around shared experiences, including a Black, Indigenous, People of Color (BIPOC) group, a Graduate Students group, and a Lesbian, Gay, Bisexual, Transgender, Queer/Questioning, Intersex, Asexual (LGBTQIA) group.

Since the working session, ASCN has been in discussions with many leadership groups to understand how we can best implement the action items. Leaders included the PIs and the working group leaders along with specific working groups implementing summer programming. Each group discussed how they would want to move equity forward as each working group functions differently. For example, our working group 6 that focuses on aligning incentives with systemic change decided to update the purpose of their working group to weave inclusivity and equity within all of their activities. This differed from our working group focused on change leaders that decided to partner with equity scholars to collaborate on events. The ASCN leadership also created an equity meeting open to all members to receive feedback on the plan and received comments that have thus been incorporated into the fall plan. We created communication and leadership expectation guidelines, reviewed and edited by leadership, to ensure our norms are clearly stated. Lastly, we asked for collaborators in racial equity and wrote a grant this summer regarding a research project to create an institute for institutional teams to create change on their campus using systemic change and racial equity principles. This project will also provide an open access, racial equity framework for systemic change that will be available on ASCN’s website if funded.

Together, ASCN and SEISMIC have proposed strategies for organizational equity within the broader STEM education community and committed to regular accountability meetings. We proposed this conversation as a session at the AAC&U Transforming STEM conference to bring together other networks and share how we can build authentic communities and bridges between communities to focus on common challenges to equity within our organizations. We also plan to continue to meet monthly to keep both organizations accountable toward our racial equity plans.

SEISMIC This Fall

Following summer activities, the SEISMIC Task Force will begin meeting this fall over the course of 12 weeks to examine existing collaboration structures and propose new ones to make SEISMIC more inclusive. SEISMIC will also be co-sponsoring the Society for the Advancement of Biology Education Research (SABER) “Striving towards inclusion in academic biology” seminar series for the second year. SEISMIC leadership will be working to publicize the original vision and goals for SEISMIC and outline paths for members to continue pushing forward its mission. At the same time, leadership has recognized that some members, in part due to the shifting focus brought on by the COVID-19 pandemic and the renewed calls for racial justice, are looking for SEISMIC to have broader impacts in higher education beyond the original vision and goals. This fall will be important for the collaboration to reestablish its scope and priorities while maintaining a community where members can get what they need.

ASCN This Fall

For ASCN, our priorities are to collect data to understand gaps to address, clarify our commitment to equity, increase leadership representation, and commit to recognizing the work of internal and external leaders. This fall we will be collecting data in a member survey, adding questions to previous member surveys to understand demographics (i.e., race, gender, role) of our membership and leadership as well as members’ experiences within ASCN. In clarifying our commitment to equity, our working group leaders suggested equity be a part of ASCN’s values so that equity is part of everything ASCN does, not just as a separate working group. To do this, we will create an ad-hoc committee to draft ASCN values to be ratified at our annual meeting in January. As we continue to make ASCN leadership more representative of our membership, we are seeking nominations for PIs of color to provide a much-need perspective as ASCN moves forward. Finally, we will commit to recognize the leaders of equity. We will highlight the work of ASCN leaders within regular meetings and also continue to advocate for equity work as important within the broader higher education community.

 

 

This is the first in a series of posts called Doing Organizational Equity Work in STEM meant to increase transparency and accountability regarding our organizations’ diversity, equity, inclusion, and social justice goals. The next update will be posted in January 2022 to discuss progress towards the goals mentioned in this post and new directions in our organizations’ internal work to promote equity.

 


 

Nita Kedharnath

Nita Kedharnath is the Project Manager of the SEISMIC Collaboration. She coordinates the implementation and evaluation of SEISMIC activities and co-leads the Student Perspectives project in the Constructs Working Group.

 

Rachel Renbarger, Ph.D.

Dr. Rachel Renbarger is the Research Director of ASCN where she helps support the inclusion and dissemination of research-based educational strategies. With her PhD in Educational Psychology, her research works to address issues of access and equity in postsecondary education. This includes undergraduate as well as graduate education. Follow her on Twitter: @Dr_Rachel_Ren


Introducing SEISMIC Themes

By Linda Adler-Kassner & Nita Kedharnath

Why are we adding Themes to SEISMIC?

At our 2020 SEISMIC Summer Meeting, we heard from participants a desire to more formally connect the work of our Working Groups. In many ways, “measurement,” “experiments,” “implementing change,” and “constructs” are parts of larger processes for institutional change. For instance, to recommend policies, we need to experiment with changes; we need to define “constructs” to assess efforts toward equity and inclusion. With this in mind, while still maintaining the communities and projects of the Working Groups, we decided to add a new organizational layer to our collaboration called Themes.

Themes will tie together the knowledge gained through SEISMIC projects within Working Groups to focus on specific SEISMIC goals and to make SEISMIC-level recommendations. To achieve our ambitions for large-scale improvement in equity and inclusion in introductory STEM courses, Themes will enable us to leverage the expertise across the collaboration, intentionally bringing together experts in DEI scholarship, STEM teaching faculty, campus leaders, and institutional researchers.

What are the Themes?

Each Theme is defined by a short phrase and guiding question. The Theme Leaders (more on that below) will identify specific deliverables for each Theme. Each Theme is intentionally defined to include people and projects from across the four SEISMIC Working Groups (measurement, experiments, implementing change, and constructs); they are not meant to align with a single Working Group or Key Project.

The three Themes that we have identified and the questions at their core are:

Policies, Practices, and Assessments for Change

What are the best data-driven policies, practices, and assessment strategies to promote STEM classroom equity, and how can departments incorporate these strategies across their introductory courses?

Presenting Data at Multiple Levels for Change

What is the best way to operationalize equity and inclusion and present usable and compelling data to different audiences (faculty, department, administration) to motivate change and promote asset-based thinking regarding students at multiple levels?

Capacity-Building for Introductory STEM Change Makers

How can SEISMIC support introductory STEM change makers in leveraging their resources and developing structures so all students who have been historically “othered” can feel they belong in STEM?

Okay, but how are these different from Working Groups?

Good question. SEISMIC’s Working Groups bring together people and/or projects around specific foci under a wider umbrella. All are related to the Working Group topic. For example, our Experiments Working Group brings together many discipline-based education researchers and instructors to run specific experiments in STEM courses. These experiments may or may not be related to each other, but are connected in that they are all experiments in STEM courses. The Working Group provides the community to brainstorm with and try out these many projects.

Our Themes, on the other hand, seek to bring together the knowledge from multiple Working Groups into an aligned set of recommendations or activities. A Theme’s focus is not on running projects, but rather is focused on finding the connections between different SEISMIC activities. The ultimate goal of each Theme is to produce something actionable on behalf of the collaboration. The figure below describes how the different pieces of SEISMIC fit together.

 

What will the Themes actually do?

Based on the core questions (above), Theme Leaders will determine appropriate deliverables for each Theme. This could involve developing a set of recommendations, running workshops for the collaboration, publishing papers on their findings, or other activities. Once this is established, the Themes will connect with Working Groups and project teams as needed to learn what the collaboration has done related to these Themes, and what is missing.

Each Theme is responsible for organizing at least two SEISMIC-wide events in 2021. Events the Themes might hold include facilitated discussions, workshops, talk presentations, and more. These events will be an opportunity for the Themes to share what they are learning, get feedback, and stay connected to the collaboration. Each Theme will also provide a progress report to the SEISMIC Collaboration Council twice a year, including details on their most recent event, how they are making progress on their set of goals and deliverables, and upcoming plans to apply for funding. Finally, each Theme will present an update on their work at the 2021 SEISMIC Summer Meeting.

This sounds like a lot of work. How is this really going to happen?

Each Theme will be guided by two leaders, a Director and a Fellow (6 positions total for three Themes). All leaders will receive a $4,000 stipend for the 2020-2021 year. Applications for these positions close 10/30/2020 (apply here). Our Collaboration Council will review applications and select the leaders by mid-November 2020.

Anyone affiliated with a SEISMIC Institution is encouraged to apply for either the Director or Fellow positions. We especially encourage members with grant-writing experience to apply to be a Theme Director. The Director will take on a bigger picture role around activities of the Theme such as writing grant proposals or planning a SEISMIC-wide Theme event. They will look out for funding opportunities for the Theme. 

We especially encourage graduate students and postdoctoral fellows looking for leadership experience to apply to be a Theme Fellow. The Fellow will be responsible for connecting directly with the SEISMIC projects and staying updated on SEISMIC activities.

Theme Leaders are Expected To:

  • Apply for cross-institutional funding to support the work of the Theme
  • Lead organization of Theme events at least twice per year
  • Pull together Theme progress reports to send to the Collaboration Council twice per year
  • Facilitate collaboration within their Theme and between their Theme and the Working Groups
  • Identify clear goals and deliverables for their Theme and move their team towards them. Deliverables could include developing a set of recommendations, running workshops, publishing, etc.

 

All SEISMIC members are encouraged to join any Theme they are interested in. Participation in these Themes will look different for each one, but will generally involve supporting the Theme to achieve its (to be determined) deliverables. Once our Theme Directors and Fellows are selected, we will provide more information on what it means to be a Theme participant for each Theme. For now, feel free to email Nita (nitaked@umich.edu) if you would like to indicate interest in a Theme or if you have any questions.


 

Linda Adler-Kassner, Ph.D.

Linda Adler-Kassner, Ph.D. is the Faculty Director of the Center for Innovative Teaching, Research, and Learning at the University of California Santa Barbara as well as a Professor of Writing Studies and the Associate Dean of Undergraduate Education. Linda is a member of the SEISMIC Collaboration Council and co-leads the Access to Practice project with Vanessa.

 

Nita Kedharnath

Nita Kedharnath is the Project Manager of the SEISMIC Collaboration. She coordinates the implementation and evaluation of SEISMIC activities and co-leads the Stakeholder Perspectives project in the Constructs Working Group.


Fostering Student Success with Access to Practice

By Vanessa Woods, Maggie Safronova, and Linda Adler-Kassner

 

As we develop courses that introduce students to practices and content within our respective disciplines, we must recognize that the students who enter our classrooms come with different social and academic experiences that inform their learning. In large introductory courses where the majority of participants can be categorized as disciplinary novices, this lack of access can lead to a disconnect that discourages students and frustrates professors.  As educators, we strive to design courses that provide students with access to our discipline regardless of how their prior academic experience shaped students’ process of learning.  Access to Practice (ATP) investigates how the development and implementation of highly structured peer review activities contribute to student academic experience.

Through ATP, UC Santa Barbara instructors work with the Center for Innovative Teaching, Research, and Learning (CITRAL) to design highly structured writing and peer review prompts focusing on challenging course concepts or challenging ways of writing. Over the two+ years CITRAL has sponsored ATP, faculty from Classics to Psychological and Brain Sciences, Political Science to Ecology, Evolutionary, and Marine Biology have developed ATP prompts and peer reviews, often in very large classes. Students have written and provided feedback about subjects as wide-ranging as the calculations involved in stoichiometry and the current-day applications of the Oedipus myth.

Structured peer review activities provide students with an environment in which they can develop ways of practicing within a discipline. As the students practice the application process several times in a guided environment they have the opportunity to develop their own approach that can be transferred to different contexts. The goal of this process helps students figure out the disciplinary norms including what type of questions to ask and how to answer them, writing conventions, and appropriate types of evidence. As the students get the opportunity to actively engage with disciplinary practices we expect to see an increase in student’s metacognitive strategies as well as help them feel like true members of their discipline. Preliminary data from UCSB suggests that by writing and providing feedback aimed at practicing disciplinary norms, students gain a deeper understanding of challenging concepts. This promising data is the basis for further exploration of the mechanism by which peer review activities affect elements that contribute to student success. The possible mechanisms being explored are metacognitive strategies, a sense of disciplinary membership, and disciplinary practices. We are currently looking for collaborators. Contact Vanessa Woods at vewoods@ucsb.edu for more information.


 

Vanessa Woods, Ph.D.

Vanessa Woods, Ph.D. is a Lecturer with Potential Security of Employment (LPSOE) which is a tenure track lecturer position (teaching professor) at the University of California Santa Barbara. She is an expert in effective teaching practices and student success research. Vanessa is active in the SEISMIC collaboration and leads the Access to Practice Project (Experiments Working Group, Project 4).

 

Maggie Safronova, Ph.D.

Maggie Safronova, Ph.D. is the Associate Director at the Center for Innovative Teaching, Research, and Learning at the University of California Santa Barbara. Maggie is involved in projects that explore the role of pedagogical innovations on students’ sense of belonging in large universities. Maggie is also the project director for UCSB’s ECoach project.

 

Linda Adler-Kassner, Ph.D.

Linda Adler-Kassner, Ph.D. is the Faculty Director of the Center for Innovative Teaching, Research, and Learning at the University of California Santa Barbara as well as a Professor of Writing Studies and the Associate Dean of Undergraduate Education. Linda is a member of the SEISMIC Collaboration Council and co-leads the Access to Practice project with Vanessa.


Scaling Up Analysis in a SEISMIC Measurement Project

By Eben B. Witherspoon

 

The AP Project in the Measurement Working Group has been moving quickly through their project milestones. After starting strong last fall and developing shared analyses for the initial three SEISMIC institutions involved, they have started “scaling up” their project to the other seven SEISMIC institutions. While the three initial institutions are ironing out the details of their final models, the scale-up institutions are in various stages of cleaning their data and getting it ready to process. In this scale-up process, our AP Project team has learned much about what it takes to run a project across SEISMIC, and what key challenges can come up. We hope this post will provide some support for our fellow Measurement projects as they start preparing to scale up their efforts!

1. Set Common Variable Names

The first thing to consider when scaling up a Measurement project is setting common definitions and variable names for key concepts in the project. It was surprising to us how much variation there was by institution in seemingly straightforward terms such as “cohort.” Taking the time to clarify these not only facilitates discussions during meetings, but is important for ensuring that variables are being defined and generated in the same way, and that eventually, shared analysis code can be run on each institution’s dataset. 

Variable definitions and names from initial discussions in the AP and Demographic projects were merged in a working document to categorize variables across the Working Group. This could be a good jumping off point for coming up with SEISMIC-wide institutional data variable names.

An example of a method for generating project-wide variable definitions.

 

2. Attend to Dataset Formatting

Second, it is essential to decide up front on the format of the dataset for analysis (i.e. long vs. wide). Unless this is explicitly discussed, it’s easy to assume everyone is doing it the same way (but there are lots of ways people store and think about time-series data!) and this has big implications for doing shared analyses later. For example, in the AP Project, we eventually landed on a combination similar to “panel data” – our data is wide by student (i.e. each row is a single student) and stacked long by each discipline (i.e., all observations who took BIO are “stacked” on top of all observations in PHYSICS), with every student unique within each discipline, but able to be repeated across disciplines (i.e. if a student took both BIO and PHYSICS). This made the most sense for our project, as it allowed us to easily subset our analyses by discipline. It might be overkill, but creating a mock-up dataset could even be helpful to visually represent how the data looks, which variables are time-invariant or not, etc.

3. Share Model Specifications and Basic Descriptive Stats

Last but certainly not least, once you have settled on your Research Questions (RQs), it is very helpful to share clear model specifications including DV, IV and analytic sample. Even when the same model is being run, misinterpretation of patterns across institutions can easily be missed when looking at only regression tables when there are different understandings of what sample is being analyzed. Sometimes something as simple as looking at sample sizes can help catch these discrepancies early on. For example, if two schools of about the same size have vastly different Ns, something may be up. One way our project addressed this issue was by moving the part of the analysis that defines the sample (which was previously done in each individual institution’s data cleaning) to the shared script, so that each institution was literally running the same code to subset the data and generate the sample for each analysis. Of course, in order to do this, there first needed to be common variable names and similarly formatted datasets…hence parts 1 and 2 🙂 As an added “bonus,” these checks and balances worked together; if our shared code couldn’t run or gave us weird results, this led us to uncover previously undiscovered issues in our variable or sample definitions!

Additional Recommendations

We recommend each project save their analysis code in the SEISMIC-wide GitHub repository (email your GitHub username to bkoester@umich.edu to join). This is a great way to share code and track changes, without making overlapping edits. Our project also used R/RStudio and Google Co-Lab with Jupyter notebooks to share, run, and comment on each other’s code as we were developing it. Then, we saved the agreed-upon code in our AP Project (WG1-P4) GitHub repository.

We have also found it helpful to use R-Markdown to create an “Analysis Workflow” file, which acts as a guide for AP project participants in understanding the analysis process overall, including how to create a dataset that will work with our shared analysis code. It captures much of our thinking on streamlining and simplifying the scale-up process.  It also serves as a single location that links to various other relevant documents for running analyses (i.e. variable naming conventions, model specifications). In addition, the document itself is shared and editable, which allows notes to be added by institutions as more specific things pop up that might be useful to others (i.e. Pitt didn’t have a variable for the year AP was taken, so we developed and noted our work-around).

Example of the Analysis Workflow document for AP Project.

 

Calling All Demographics Project Analysts

We would love feedback from Demographics Project (WG 1 P1) analysts about our process and what you have been doing to coordinate. For example, have you found less complicated workarounds for the same issues? Are we missing key parts of scaling up that you’ve experienced? Let us know!

Interested in joining the AP Project?

Overall, the process for onboarding is:

  1. Join the Project GitHub repository (by emailing a GitHub username to bkoester@umich.edu)
  2. Read the “Analysis Workflow” file (available on GitHub)
  3. Preview the SEISMIC variable definitions doc (also linked to in the Analysis Workflow file)
  4. Create an institution-specific folder in the WG1-P4 GitHub that contains the data cleaning files for that institution’s data – these will all be slightly different, but may be useful to see how others have done it as there will be some overlap.
  5. Once the data is in the same format, run the shared analysis file (available on GitHub). 
  6. Join one of our meetings and share your findings!

 

 

 

Eben B. Witherspoon, Ph. D.

Eben Witherspoon is a post-doctoral researcher in the Learning Research and Development Center (LRDC) at the University of Pittsburgh. His main line of research examines attitudinal and environmental factors during the transition to college that influence retention in STEM career pathways for underrepresented students. Currently, he is working on a project looking at the factors influencing gendered attrition in the pre-med course sequence. Eben is an active SEISMIC member and works on the AP Project (Measurement Working Group, Project 4).