Measurement Working Group

Overview

 

Resources

 

This page is all about Working Group 1: Measurement! Keep reading for resources, leadership, projects, and community highlights.

Focus: Establish metrics for measuring equity and inclusion in foundational STEM courses, conduct the measurements, and identify actionable data to promote change.

Leadership

 

 

Co-chair

Stefano Fiorini

sfiorini@indiana.edu

 

Co-chair

Becky Matz

rlmatz@umich.edu

Key Projects

 

 

 

CLICK TO VIEW PROJECT POSTERS

Project Leads

Project Description

This project will conduct parallel data analyses across the 10 SEISMIC institutions in different introductory STEM courses. It seeks to explore the components of student identity - those we can capture, those we cannot, and how they interact with the learning environments typical to intro STEM. Participants will replicate the LARA analyses, exploring equity in lecture versus lab classes and comparing STEM to non-STEM courses at institutions that were not part of the initial LARA project. This project will then extend past the gender analysis done in LARA to also explore how ethnicity, first-generation status, and the interactions of these identities affect student outcomes. To complete this project, the team will need to regularize data across the institutions, determine which courses or course sequences to include in the analyses, clarify the motivations of the project and what they hope to find, and develop and apply analytical techniques designed to elucidate the complex interplay between identity and the learning environment.

Project Leads

Project Description

This project seeks to understand how people fall out of STEM and if that is connected to failing foundational STEM courses. To explore this, this team will work to answer two main questions using first quantitative data analyses and then by studying student motivation. The first question is, what happens to STEM students who fail or earn a D in an intro class? This question gets at where the students go after failing, and what causes them to make those decisions. For example, some students may repeat the class, others may switch majors or even leave the university. The second question asks, what happens to students who repeat the class? To answer these questions, this project will also determine if the paths students take after failing a foundational STEM course differ by demographic groups. After determining the paths that are present for students, this team will then examine why these paths arise in terms of motivation - for example, imposter syndrome may influence minority-group more than majority-group students. To complete the first part of this project, the team will need to establish choose a course and major to start with, determine course equivalencies between universities, regularize their data, receive IRB and FERPA approval, and conduct and interpret the initial results on pathways. Once those paths are understood, the group will work with motivation experts to develop instruments to understand what drives students into the different paths.

Project Leads

Project Description

This team will use a mixed methods approach to analyze the impact of classroom composition with respect to identity groups, and determine how it relates to student outcomes such as course grades and persistence in the major. The primary research question is, does demographic composition of classes at various hierarchical levels affect student success and/or student attitudes? To complete this project, they will need to regularize their institutional data and gather additional data regarding student motivation, self-efficacy, sense of belonging, and other affective factors through surveys or focus groups. This additional data will help the team to understand how these affective elements interact with student outcomes. This team hopes to inform the higher education community on strategies for promoting a sense of inclusion in the classroom and to raise awareness on this topic of classroom composition and its impacts on students. Ultimately this project will inform course-level, college-level, and university-level policies.

Project Leads

Project Description

This project aims to inform university and department policies around accepting AP credit. University policies on AP credit have real consequences for students, as the rejection of this credit could lead to students having to retake courses and lose motivation, particularly if they do poorly in the challenging learning environment of a large enrollment course. Applied across multiple AP courses, students will then take longer to graduate and thus have to pay more for school. The negative consequences may be especially large for students from lower income households and coming from high schools that did less AP exam preparation work. Considering that credit-granting policies vary widely across departments and across institutions, this team seeks to help institutions make evidence-based decisions on when to accept AP credit, if at all. To accomplish this goal, the team will look across the SEISMIC institutions and evaluate the consequences of accepting AP credit, both positive and negative, for students by different demographics.

Project Leads

Project Description

Active Learning has been linked to improved student performance in STEM (Freeman et al., 2014), and has become popular as a shorthand for instruction, activities and classroom environments that move away from lecture-based methods and towards a student-centered approach. However, there is still wide variation in what we mean when we say our introductory STEM courses provide Active Learning experiences for our students. Further, while most research focuses on course performance as an outcome, Active Learning contains many components that could also impact students’ persistence independently of their effects on grades (Ballen, et al., 2017). For example, placing an emphasis on communal values (Jackson, et al., 2016) and improving a sense of belonging (Walton, et al., 2015) have been shown to improve persistence decisions for minoritzed racial groups and women in STEM. Currently, few Active Learning observation protocols include measures of such components. This project aims to 1) understand the Active Learning landscape across SEISMIC institutions 2) identify gaps in current measures of Active Learning, and 3) develop measures and observation protocols that focus on components of Active Learning classrooms that are related to persistence, particularly for minoritized groups in STEM.

Project Leads

Project Description

Despite decades of efforts to increase the diversity of the STEM workforce, women and professionals of diverse ethnicities and backgrounds remain woefully underrepresented. A variety of interventions and instructional changes are currently being implemented in large lower division science courses with the goal of reducing equity performance gaps and thus promote retention and graduation of students traditionally underrepresented in STEM (URMs). In contrast to the abundant literature on disparities in STEM students in introductory courses, much less is known about the disparities that are present in upper division science courses and their causes. While the majority of students who drop out of STEM do so during the first two years of college, STEM attrition occurs throughout the whole major. Furthermore, the existing disparities result in URM students graduating with lower GPAs and taking longer to graduate, factors which reduce their competitiveness in the job and academic market, perpetuating inequities in the workforce. Furthermore, while the DBER field is starting to recognize the importance of intersectional approaches to studying equity gaps, this approach remains largely underutilized. This workgroup will complement the work currently done by WG1P1 and WG1P3 by characterizing equity gaps in large upper division courses in STEM using a variety of quantitative methods.

Project Leads

Project Description

Almost every student experiences at least one large lecture course during their undergraduate experience. This project is focused on the impacts of those large lecture courses – specifically whether they choose to go on in later courses in that department, how well they do in those later courses, and whether they stay at the university. Work this summer will be geared towards analyzing the data sets available and exploring possible predictors and identifying relevant student groups.

Publications

 

  • Whitcomb KM, Cwik S, Singh C. Not All Disadvantages Are Equal: Racial/Ethnic Minority Students Have Largest Disadvantage Among Demographic Groups in Both STEM and Non-STEM GPA. AERA Open. January 2021. doi:10.1177/23328584211059823
  • Castle, S., et al. “Equity in the STEM Landscape: A Multi-institutional Approach to Mapping Systemic Advantages Within STEM Courses” 2021 AERA Conference Proceedings, 2021.
  • Fischer, Christian & Witherspoon, Eben & Nguyen, Ha & Feng, Yanan & Fiorini, Stefano & Vincent-Ruz, Paulette & Mead, Chris & Rodriguez, William & Matz, Rebecca & Schunn, Christian. (2022). Advanced placement course credit and undergraduate student success in gateway science courses. Journal of Research in Science Teaching. 10.1002/tea.21799.
  • Pearson, M. I., Castle, S. D., Matz, R. L., Koester, B. P., & Byrd, W. C. (2022). Integrating critical approaches into quantitative STEM equity work. CBE—Life Sciences Education, 21(1), es1.

Additional Activities

Community Highlights