Research – Education Next https://www.educationnext.org A Journal of Opinion and Research About Education Policy Tue, 02 May 2023 12:24:13 +0000 en-US hourly 1 https://wordpress.org/?v=5.4.2 https://www.educationnext.org/wp-content/uploads/2020/06/e-logo-1.png Research – Education Next https://www.educationnext.org 32 32 181792879 Putting Teachers on the Ballot https://www.educationnext.org/putting-teachers-on-the-ballot-raises-fewer-charters-when-educators-join-school-board/ Tue, 30 May 2023 09:00:31 +0000 https://www.educationnext.org/?p=49716557 Raises for teachers, fewer charters when educators join the school board

The post Putting Teachers on the Ballot appeared first on Education Next.

]]>

Illustration of campaign flyers on a sign

Public K–12 education in the United States is distinctively a local affair: school districts are governed by local boards of education, composed of lay members typically elected in non-partisan elections. These boards have decision-making power over hundreds of billions of public dollars and oversee complex agencies that, in addition to preparing a community’s children for the future, can be the biggest employer in town. Yet we know very little about what factors influence a board’s governance and impact, including the professional backgrounds of elected members.

One profession would seem to have particularly relevant effects: educators. Organizations like the National Education Association and Leadership for Educational Equity, the political arm of Teach for America, are training and supporting their educator members and alumni to run for elected offices. What might be the impacts of such efforts on school board elections, district governance, and student outcomes?

Research focused on boards of directors, which play a similar role in the corporate world, has found that adding members with more industry expertise increases a firm’s value. It stands to reason that electing educators to school boards could have similarly beneficial effects. For example, former classroom teachers or school leaders with firsthand knowledge of common challenges could theoretically make better decisions about teachers’ working conditions and positively influence student performance.

On the other hand, 70 percent of U.S. teachers are members of teachers unions. This raises the possibility that educators serving on school boards could be influenced not only by expertise but also allegiance to union priorities. That could theoretically influence collective bargaining, which is one of the major responsibilities of a school board. Union allegiance could shift bargaining agreements toward union goals, such as increasing teacher salaries or limiting charter-school growth, which may not necessarily benefit students.

We investigate these possibilities in California. State election rules randomize the order of candidates’ names on the ballot, which allows us to estimate the causal effects of an educator serving on a school board. By looking at randomized ballot order, candidate filings, election records, and school district data, we provide the first evidence on how the composition of local school boards affects district resource allocation and student performance.

Our analysis finds no impact on student achievement from an educator serving on a school board; neither average test scores nor high-school graduation rates improve. However, outcomes relevant to union priorities advance. Relative to a district without an educator on the school board, charter-school enrollment declines and the number of charter schools shrinks by about one school on average during an elected educator’s four-year board term.

In addition, each educator elected to a board leads to an increase of approximately 2 percent in teacher pay, while non-instructional salaries remain flat. Benefits spending is stable, while the share of district spending on ancillary services and capital outlays shrinks. We also find that educators are 40 percent more likely than non-educators to report being endorsed by teachers unions.

Despite raising teachers’ salaries, electing an educator to a school board does not translate into improved outcomes for students and has negative impacts on charter schools. We believe this shows that school boards are an important causal channel through which teachers unions can exert influence.

Electing Educators in California

Nationwide, nearly 90,000 members serve on about 14,000 local school boards. These boards have several general responsibilities, which include strategic planning for the district, curricular decisions, community engagement, budgeting, hiring senior administrators, and implementing federal and state programs and court orders. In addition, in nearly all states, school boards determine contracts for instructional staff through collective-bargaining agreements with teachers unions. These negotiations set salary schedules, benefits, work hours, and school calendars. Local school boards also set attendance zone boundaries and, in about three dozen states, authorize and monitor charter schools. In 2020–21, local education agencies accounted for 90 percent of all charter-school authorizers in the U.S. and enrolled 48 percent of the nation’s charter-school students.

While typical in most respects, school district governance in California has several unique characteristics. First, teachers unions are especially influential: 90 percent of California teachers are full voting union members. Second, school boards effectively do not have the power to tax. Under Proposition 13, property-tax collections are capped at 1 percent of assessed value, and assessments are adjusted only when a property is sold. Finally, charter authorization is overwhelmingly a local issue, with about 87 percent of California charters authorized by local school districts. Los Angeles Unified School District is the single biggest local authorizer in the U.S. and enrolls 4 percent of all charter-school students nationwide.

Our analysis is based on records from the California Elections Data Archive for all contested school board elections from 1996 to 2005. The data include each candidate’s vote share, ballot position, electoral outcome, and occupational background. We identify as educators candidates who describe their primary occupation or profession as a teacher, educator, principal, superintendent, or school administrator. Educators account for 16 percent of all 14,150 candidates in contested races and 19 percent of all 7,268 winners during this period.

Almost all school-board members serve four-year terms with staggered contests occurring every two years. The average tenure is seven years, and the average school board has five members. We use candidate-level records to construct yearly measures of school-board composition in each district, including the share of members who are educators. On the average school board, educators account for 18 percent of members. We link school-board rosters with district-level characteristics and charter-school campus and enrollment counts from the federal Common Core of Data, as well as negotiated salary schedules and district finance information from the state Department of Education. To look at impacts on student outcomes, we include average test scores in elementary and middle schools along with high-school graduation rates, also from the state education department.

Investigating Educator Impacts

To estimate the causal effects of an educator being elected to a school board, we need to compare two sets of circumstances: what happens after an elected educator joins the board and what would have happened if the educator had not won. While the effects could appear immediately and persist over time, it is also possible that they only become apparent in the longer run. Our approach therefore must examine the profile of effects over time.

The key challenge we face in making these comparisons is that the school districts that elect educators likely differ from those that do not—and these other differences could be responsible for any policy outcomes that change after an educator’s election. To overcome this challenge, we take advantage of the fact that, under California law, the order in which candidates for elected office appear on the ballot is randomly determined. Our data confirm that candidates who have the good fortune of being listed first on the ballot gain an advantage of 10.3 percentage points of the votes cast in their election. When an educator is listed first, this advantage translates into a 2.3 percentage point increase in the share of the board’s members who are educators. In short, the random assignment of an educator to the top of a ballot will shift a board’s composition.

Armed with this insight, we compare the policy choices of districts where educators are and are not listed first to isolate the causal effects of adding an educator to a school board on student outcomes, district spending, and charter schools. We first look at elementary- and middle-school scores on reading and math tests, as well as high-school graduation rates, and find no impacts.

We then consider teachers’ working conditions and find limited evidence of effects on service days, benefits, or class size. However, when an educator is elected to a school board, teachers’ salaries increase by 2 percent more than they would have otherwise four years after election. These increases apply across the board, for teachers at all levels of education and experience.

Because California school boards cannot raise the tax rate, boards decrease spending on building repairs and services like professional development in order to pay teachers more (see Figure 1). Four years after an educator is elected, a school board has increased the share of spending on certified salaries by 1.3 percentage points and decreased spending on capital outlays and services by 0.6 and 0.7 percentage points, respectively. We do not find evidence for impacts on superintendents’ salaries.

Figure 1: Districts Spend More on Teacher Salaries After an Educator Joins a School Board

In looking at effects on charter schools, the share of district students enrolled in charters declines by three percentage points (see Figure 2). By the end of an elected educator’s four-year term, there are 1.3 fewer charter schools in the district. In a state with an active charter sector serving at least one out of every 10 public-school students, these are sizeable impacts.

Figure 2: Fewer Charters When Educators Serve on Local School Boards

What if a school board includes multiple educators? That could shift the identity of the median board “voter” for a given issue and influence board decisions through deliberations and agenda-setting. To examine these possibilities, we estimate the effects of electing an educator to a school board if it already has a sitting member who is an educator. Our results suggest that this is of limited importance. There are slightly larger negative effects on charter school enrollment, but these are not statistically significant.

We also investigate whether electing an educator to a school board has consequences for subsequent elections and find evidence that it does. In this analysis, we look again at the effect of ballot order. An educator being listed first increases the number of elected educators in that election by 13 percent but decreases the number of elected educators by 9 percent in the next election. Interestingly, educators are no less likely to run in these subsequent elections; those who do run are just less likely to win. The long-term causal effects of electing an additional educator would be even larger in the absence of this electoral dynamic.

The Influence of Teachers Unions

Our findings suggest that educators’ professional expertise on boards does not translate into improvements in student learning. The results are consistent with a rent-seeking framework, in which representation of union interests predicts higher teachers’ salaries and potentially negative effects on student performance. Our own data reveal that educators are 40 percent more likely than non-educators to be endorsed by a teachers union. School board member survey data also indicate a strong positive association between professional experience in education and alignment with union priorities.

We conclude that school boards may be an important causal mechanism for the influence of teachers unions on local education, which points to several avenues for future research. Our ballot-order-based strategy provides a new approach to inferring how the characteristics of candidates causally affect outcomes. A valuable next step would be to analyze candidate-level records of union endorsement. This would facilitate separating out the influence of educators on education production from their possible alignment with teachers unions. Likewise, shifting from aggregate school-level to administrative student records would enable disentangling impacts on student sorting from their effects on education quality. Future work should also focus on broader dimensions of students’ skills and behavior, such as social-emotional attributes and civic engagement.

In summary, the election of an educator to a local school board shifts spending priorities on K–12 public schools, which collectively cost about $800 billion in federal, state, and local tax dollars a year. Yet voter turnout in school-board elections is typically between 5 and 10 percent. While more research is needed, voters don’t need to wait. Our results show just how much these races matter.

Ying Shi is assistant professor at Syracuse University and John G. Singleton is assistant professor at the University of Rochester.

The post Putting Teachers on the Ballot appeared first on Education Next.

]]>
49716557
The Fine Art of School Engagement https://www.educationnext.org/fine-art-of-school-engagement-how-expanding-arts-education-affects-learning-behavior-social-emotional-growth/ Tue, 02 May 2023 09:00:34 +0000 https://www.educationnext.org/?p=49716575 How expanding arts education affects learning, behavior, and social-emotional growth

The post The Fine Art of School Engagement appeared first on Education Next.

]]>
Students at Parker Elementary Music Magnet School in Houston sing at the 22nd annual Hear the Future invitational choral festival presented by the Houston Chamber Choir in January 2022.
Students at Parker Elementary Music Magnet School in Houston sing at the 22nd annual Hear the Future invitational choral festival presented by the Houston Chamber Choir in January 2022.

From their earliest years, children use art for learning and self-expression. Preschoolers draw, paint, and build to understand and depict their surroundings. They learn their letters by singing the alphabet song. And they immerse themselves in stories to learn about their natural and social worlds, from books read by caregivers and during dress-up and imaginative play.

Yet the arts maintain a precarious position in K–12 public education. After a steady increase throughout the middle of the 20th century, arts education has been in decline since the 1980s. In a 2012 national survey, roughly half of public-school teachers reported declines in instructional time and resources for art and music over the previous decade, while only about one in 10 reported similar declines for reading or math. Teachers attributed the declines to test-score pressures, budget cuts, or both.

These trends have been most pronounced for students of color, who are more likely than white students to attend under-resourced schools and about half as likely to experience any arts education, on average. In a survey by the National Endowment for the Arts, the percentage of Black adults reporting any arts education during childhood fell by nearly half in 2008 compared to 1982, to 26 percent from 51 percent. Hispanics experienced similar declines to 28 percent from 47 percent, while the share of white adults who experienced arts education remained relatively flat, at around 58 percent.

How are these changes affecting American students? To begin with, an education without the arts is insufficient and fails to provide what federal education law defines as a “well-rounded education.” The arts have intrinsic value as a foundational form of human expression, providing ways of learning and experiencing different perspectives on the human condition. Moreover, theory and emerging research suggest arts education may have positive effects on student behavior, school engagement, and social-emotional development, all of which contribute to success in school.

We investigate the causal effects of arts education by looking at the Arts Access Initiative in Houston, which brings teaching artists, performances, and workshops to under-resourced public elementary and middle schools from the city’s ballet, symphony, and fine-arts museum, among many others. Our analysis compares schools that were enrolled by a random lottery to schools that applied to participate but were not chosen, in the first large-scale randomized control trial of an arts education program in an authentic school setting.

We find that arts learning has positive effects on empathy, school engagement, student discipline, and writing achievement. Students’ emotional and cognitive empathy increase by 7.2 percent and 3.9 percent of a standard deviation, respectively. At schools with expanded arts education, students are 20.7 percent less likely to have a disciplinary infraction. School engagement increases by 8 percent of a standard deviation. Arts learning improves writing test scores by 13 percent of a standard deviation but does not have significant effects on reading, math, or science test scores. The positive effects are especially pronounced among English language learners, whose writing scores improve by 27 percent of a standard deviation. These results demonstrate that the arts positively affect meaningful educational outcomes and can inform strategies to restore and retain arts education in under-resourced schools.

Mazen Kerbaj, a cornet player, gives a music improvisation workshop for students in Houston. New research reveals the benefits of arts education on overall academic achievement.
Mazen Kerbaj, a cornet player, gives a music improvisation workshop for students in Houston. New research reveals the benefits of arts education on overall academic achievement.

Art for More Than Art’s Sake

The benefits of arts education are rich in theory and testimony, but little rigorous evidence supports most claims. In a recent report co-written by one of us (Brian Kisida), the American Academy of Arts and Sciences took stock of the many theories and claims surrounding arts education and identified several areas of educational benefits that are supported by research. First, there is the primary claim that learning about the arts is good for its own sake, both because the arts are a fundamental mode of human expression and because familiarity with the arts helps students acquire cultural capital. In addition, there are intrinsic benefits to learning about and engaging with the arts. These include broadening students’ understanding of other cultures and history, supporting their social-emotional development and interpersonal skills, and providing opportunities for career exploration and creativity.

In terms of academic outcomes, there has been little causal research to date examining how arts education in school settings affects academic achievement. Some research has found that integrating arts experiences with instruction can boost student interest and content knowledge, such as by pairing a history unit with a live theater performance about the topic, particularly for English language learners and students with low test scores. Other studies focusing on arts education through field trips have found increases in students’ empathy and tolerance of others, as well as improvements in school measures like attendance and behavior—outcomes that contribute substantially to long-term success. For example, students’ attendance and disciplinary records are better predictors of their eventual on-time graduation and college enrollment than their grades (see “The Full Measure of a Teacher,” research, Winter 2019). And students who attend schools that improve social-emotional development have fewer absences and disciplinary infractions and are more likely to graduate and persist in a four-year college (see “Linking Social-Emotional Learning to Long-Term Success,” research, Winter 2021).

In many areas, school districts have formed broad-based coalitions with arts and community partners to restore and expand in-school arts education. According to the U.S. Department of Education, 42 percent of U.S. public schools partner or collaborate with cultural or community organizations, 31 percent with individual artists, 29 percent with museums, and 26 percent with performing arts centers. These arrangements take various forms, such as in-school teaching-artist residencies, workshops for students and teachers, professional artist performances, and after-school programs.

Our analysis focuses on one such program, the Houston Arts Access Initiative. The initiative was created by the Houston Independent School District, city government leaders, and local arts institutions and philanthropists with the goal of equitably advancing student access to the arts. It began in 2013 with a district-wide campus inventory of arts educational offerings, which found that 29 percent of K–8 schools had no full-time arts specialist, and 39 percent had either one or zero arts partnerships with community arts organizations. Meanwhile, 98 percent of surveyed principals and teachers agreed that “students benefit from access to the arts in school.”

The initiative focused on expanding arts education in schools with the fewest resources and raised funds to support expanded partnerships with local arts organizations. School participation was voluntary, but principals had to commit to spending between $1 and $10 per student on the program, with foundation support and in-kind donations from cultural institutions contributing a dollar-for-dollar match. During the first two years of the program, 60 eligible schools applied, and 42 were enrolled through a random lottery. More than 50 local arts organizations provided a diverse array of programs, including theater (54 percent), music (18 percent), visual arts (16 percent), and dance (12 percent). Nearly two-thirds of schools had either teaching artist residencies or on-campus performances during the school day, while about one in four schools went on field trips and one in 10 offered arts education after school.

The mission of the Arts Access Initiative was familiar to the participating organizations, virtually all of which already had well-articulated educational philosophies and had been providing educational services. These organizations also had designed their programs to be culturally representative and meet the needs of underserved students. Arts offerings included classical music and fine art, as well as African dance and drumming, Asian dance, Aztec dance, Brazilian music and dance, Chinese art, Mexican folklórico, hip-hop music and dance, and Hispanic literature.

In addition to touting their programs’ impact on students’ social-emotional development, many organizations also had made deliberate efforts to align their work to state educational standards or content from tested subjects. For example, Writers in the Schools described its workshops as aligned to state tests and core content, while the Mercury Chamber Orchestra offered workshops that integrated science with classical music “to introduce the science of Galileo, Sir Isaac Newton, and Einstein,” or civics by “hearing the favorite tunes of Ben Franklin… while learning about democracy and the people who helped create our nation.”

Assessing the Impact of Art in School

We designed our study to identify the causal impact of community-based arts partnerships and programs in school, including whether a substantial increase in arts education improves student engagement and academic achievement. The study’s central feature is the random assignment of eligible applicant schools to participate (or not) in the initiative. This approach ensures—and our data confirm—that participating and non-participating schools are similar based on their grade levels, student demographics, preexisting arts resources, and percentages of students earning scores of at least “proficient” on statewide math and reading assessments. These schools also had equivalent numbers of school-community partnerships before the initiative began: an average of 2.80 partnerships at schools that did not take part compared to 2.76 at participating schools. After the program, participating schools gained 7.10 more partnerships, and students experienced 5.03 more arts educational experiences over the course of a school year compared to students at non-participating schools.

Our analysis is based on data from 2016–17 and 2017–18 for 15,886 students in grades 3 through 8. These students attended 42 schools, 36 of which were elementary schools. In all, 86 percent of students in the sample qualified for free or reduced-price school lunch, and 33 percent were English language learners. In terms of race and ethnicity, 68 percent of students identified as Hispanic, 25 percent as Black, and 3 percent as white.

We consider individual student attendance and enrollment records, disciplinary records, and test scores on the State of Texas Assessments of Academic Readiness (STAAR), which include reading and math tests in grades 3 through 8, writing tests in grades 4 and 7, and science tests in grades 5 and 8. In addition, we conducted an original survey in 2017–18. We successfully collected and linked outcome survey data to the district’s administrative data for 10,066 eligible 3rd–8th grade students (79 percent), and 7,640 eligible 4th–8th grade students (78 percent of the sample with prior year test scores). We use the latter sample when examining test-score outcomes so that we can control for any minor differences in students’ academic achievement before the start of the intervention.

The survey items are intended to capture levels of college aspirations, opinions about the value of the arts, indicators of social-emotional learning, and school engagement. The baseline survey was administered at the beginning of the fall semester (late September through early October) and the outcome survey at the end of the school year (late April through May).

We group student responses to create measures of school engagement and empathy. Our school engagement measure captures how students rate their agreement with statements like, “School work is interesting” and “This school is a happy place for me to be.” Our emotional empathy measure is based on a single survey item: “I want to help people who are treated badly.” Our cognitive empathy measure assesses the degree to which students can understand and learn from someone else’s perspective, through survey items like, “I can learn about my classmates by listening to them talk about works of art.” and “Works of art… help me understand what life was like in another time or place.” Students’ college aspirations were captured by a single item (“I plan to go to college”) and are indicated by a binary measure of whether students strongly agreed or not.

Figure 1: Benefits of Expanding Arts Education

Results

Increasing students’ arts educational experiences has positive effects on student discipline, writing achievement, school engagement, and empathy. At participating schools, 13.8 percent of students received disciplinary infractions compared to 17.4 percent at non-participating schools—a difference of 20.1 percent. Students’ writing scores are 13 percent of a standard deviation higher than at similar schools with less arts education. School engagement increases by 8 percent of a standard deviation, and students’ emotional and cognitive empathy grow by 7.2 percent and 3.9 percent of a standard deviation, respectively (see Figure 1).

Our analysis does not find effects on students’ math, reading, or science achievement, contrary to popular claims that arts education has a transfer effect on other subjects. However, the positive effects on writing achievement on statewide standardized tests are noteworthy. Many of the arts programs offered opportunities for self-expression and reflection, and some included student writing exercises, either through a specific focus on literary arts or arts-integrated writing activities. The STAAR writing test features open-response expository essays to assess composition skills as well as multiple-choice items on mechanical skills. When we disaggregate student scores on this assessment, we find significant increases on both sections. But the effects are twice as large for the written compositions than for the mechanics sections, at 18 percent and 9 percent of a standard deviation, respectively. This finding aligns with the theory that participation in arts experiences improves students’ ability to express themselves and articulate their own ideas.

The positive effects on students’ writing achievement are especially large for English language learners, whose scores increase by 27.1 percent of a standard deviation overall (see Figure 2). For elementary-school English language learners, the effect is 34.8 percent of a standard deviation. English language learners also experience greater-than-average gains in school engagement, at 14.3 percent of a standard deviation, and emotional empathy, at 15.7 percent of a standard deviation. They are 6.5 percentage points more likely to plan to attend college, despite the fact that the program did not increase college aspirations significantly for students overall.

Figure 2: Larger Effects for English Language Learners

These findings reinforce earlier research showing the benefits of using arts-learning techniques to deliver core content to English language learners, including increases in written and oral language skills and student engagement and decreases in absences. Researchers have suggested that arts learning increases verbal interactions between students and teachers and offers multiple pathways to connect with educational content. Moreover, the arts programs in Houston tended to have a strong emphasis on art from a diverse array of cultures, which may be especially engaging for students whose first language is not English.

Our analysis also finds notable differences in the experiences of elementary and middle-school students. Writing achievement improves by 19.7 percent of a standard deviation for elementary-school students compared to 5 percent of a standard deviation for middle-schoolers. There is no improvement in school discipline at elementary schools, whereas middle-school students are 6.8 percentage points less likely to experience an infraction. We find opposite trends in school engagement: it grows by 21 percent of a standard deviation for elementary-school students but declines by 12.5 percent of a standard deviation among middle-school students. We see a similar split in students’ college aspirations: an increase of 5.2 percent at elementary schools and a decrease of 4.6 percent in middle schools.

One possible explanation is the implementation of the program, which was primarily focused on elementary schools. Programming in middle schools tended to be more piecemeal, one-off experiences, whereas elementary schools were more likely to opt for artist residencies where teaching-artists provided arts instruction to entire grades on a weekly or semi-weekly basis for a semester or full school year. As a result, smaller proportions of middle-school students participated in arts programming, and those that did were exposed to a diluted dosage—limitations that may have compromised students’ enjoyment or engagement with the arts. It could also be the case that younger students are more receptive to arts education experiences, since educational interventions tend to have greater effects in early years.

A Well-Rounded Education

Our investigation, the first large-scale randomized control trial of an arts education program implemented in an authentic school setting, finds significant and policy-relevant benefits for students across a diverse array of elementary and middle schools in the nation’s 7th largest school district. When young people engage with the arts, they gain unique opportunities for self-discovery, social development, and community connections. When arts education is part of the school day, students experience greater school engagement, fewer disciplinary infractions, enhanced social-emotional development, and stronger academic achievement in writing. Arts education is a promising option for policymakers interested in improving social-emotional learning outcomes and student behavior.

Our study is not without limitations. Our results may not be generalizable to schools where leaders are not dedicated to supporting the arts. The results also may not translate easily to communities without sufficient arts resources and institutions. The findings also reflect the severely deficient arts resources that participating schools had at the outset of the program. A similar program in schools with higher initial levels of arts resources may not produce the same effects.

Still, our analysis provides evidence that arts education can support student success above and beyond its intrinsic benefits. We also show that expanding arts education does not harm student achievement on standardized tests—and actually benefits writing performance. As education policymakers seek reforms that improve school engagement, school climate, and other social-emotional and behavioral outcomes to restore student progress and mental health after pandemic-related disruptions, they should weigh the opportunity costs when arts education is decreased or eliminated.

Daniel H. Bowen is associate professor at Texas A&M University. Brian Kisida is associate professor at the University of Missouri. They co-direct the Arts, Humanities & Civic Engagement Lab, supported in part by the National Endowment for the Arts.

The post The Fine Art of School Engagement appeared first on Education Next.

]]>
49716575
A Poor Poverty Measure https://www.educationnext.org/poor-poverty-measure-identify-children-in-need-look-beyond-free-lunch-data/ Tue, 14 Feb 2023 10:00:43 +0000 https://www.educationnext.org/?p=49716284 To identify children in need, look beyond free lunch data

The post A Poor Poverty Measure appeared first on Education Next.

]]>
Free and reduced-price meal designations are inaccurate indicators of family income.
Free and reduced-price meal designations are inaccurate indicators of family income.

In education policy and public debate, we often talk about students from “low-income” families. That descriptor is typically based on data from the National School Lunch Program, which provides qualified students with school meals for free or at a reduced price. Enrollment in the program, which is operated by the U.S. Department of Agriculture, plays a central role in identifying low-income students in U.S. schools and thus a central role in consequential education funding and accountability policies at the federal, state, and local levels.

For example, the federal Every Student Succeeds Act requires states to track gaps in student achievement by poverty status. Among the 50 states, 44 use free and reduced-price lunch enrollment to identify low-income students. These data are also commonly used to allocate federal, state, and local funding to schools serving low-income children. School and district poverty rates, as determined by free and reduced-price lunch enrollment, additionally feature prominently in social science research, school-funding lawsuits, state laws and regulations, and philanthropic investment.

Yet a close look shows that free and reduced-price meal designations in the National School Lunch Program are grossly inaccurate indicators of family income. Using administrative data from Missouri, we find that student enrollment in the program is oversubscribed by about 40 to 50 percent relative to stated income-eligibility rules. This finding is not unique to Missouri. We see the same basic pattern in an extended sample of 27 states. Moreover, this is not a recent phenomenon. Enrollment in the school lunch program was oversubscribed even before 2014–15 when the “Community Eligibility Provision” was rolled out nationally, which permits sufficiently high-poverty schools and districts to enroll all their students to receive free meals.

While it has been understood for some time that school lunch enrollment as a poverty indicator is blunt and prone to error, the magnitude of the problem has not yet been fully appreciated. In exploring the rules, features, and processes of the National School Lunch Program, we find that the program’s design, incentives, and lack of income-verification enforcement likely contribute to the oversubscription. These findings raise important questions about the administration of a program that supports the nutrition of American schoolchildren as well as key datasets driving policy and funding decisions across the country.

More Than Just Lunch

For three quarters of a century, schoolchildren from low-income families have received low- or no-cost meals under the National School Lunch Program. In 2019–20, the program provided subsidized lunches to nearly 22 million students at about 94,000 public and nonprofit private schools across the United States.

Students are enrolled in the program in two ways. Through direct certification, students are automatically enrolled for free school meals if their families receive benefits such as food assistance, Medicaid, and Temporary Assistance for Needy Families, or if they are migrant, in foster care, or homeless. Alternately, school districts administer income surveys to parents to determine eligibility. Students from families with incomes at or below 130 percent of the federal poverty line qualify for free meals, and those from families with incomes between 130 and 185 percent of the poverty line qualify for reduced-priced meals. State and federal aid programs like Medicaid and food stamps verify incomes reported by participants, while school districts usually do not. In addition, if an attempt to verify eligibility fails, a student’s enrollment in the lunch program ends, but there are no other repercussions.

Enrollment in the school lunch program is a commonly used proxy for student poverty in policy, research, and for other purposes. But is it an accurate indicator of family income? To answer this question, we analyze school meal enrollment and two alternative measures of poverty in Missouri schools during the 2016–17 school year to determine how closely they are aligned. Our data on meal enrollment are from the state education department and show students’ National School Lunch Program designations. Each student is coded as enrolled for free meals, reduced-price meals, or neither. Our alternative poverty measures are based on students’ direct certification data and estimates of school-neighborhood poverty from the National Center for Education Statistics, which are based on the incomes of households located near schools as reported in the Census Bureau’s American Community Survey.

Assessing Accuracy

A case could be made for using either direct certification or school neighborhood poverty data for the purpose of assessing the accuracy of students’ meal designations. In Missouri, direct certification applies to students from families living at or below 130 percent of the poverty line, which is the same income threshold for free-meal enrollment by the National School Lunch Program. And while school-neighborhood poverty estimates are reported as the average family income associated with a school, with some basic adjustments they also can be used to estimate the share of students living at or below 130 percent of the poverty line.

However, there is no guarantee that either alternative metric is itself accurate. Therefore, in a two-way comparison of free lunch data to either direct certification or school neighborhood poverty data, it would be difficult to know the source of any discrepancy. Given this, we first compare the two alternative data sources to each other. When we do this, we find that the estimated shares of students in a school living at or below 130 percent of the poverty line are very closely aligned, which gives us confidence that both alternative measures are accurate, on average.

We then conduct similar tests to assess the accuracy of the shares of students who receive free or reduced-price lunch. That is, we test whether the school lunch program enrollment share in a school matches the share of students living at or below 130 percent of the poverty line (free) or 185 percent of the poverty line (free and reduced price) as measured by the direct certification and school neighborhood poverty data. If schools and districts are following the National School Lunch Program rules, these numbers should line up.

This is not what we find. We conduct a series of alignment tests where a value of 1.0 indicates one-to-one correspondence; that is, 1.0 means the poverty data from the measures being compared match each other across Missouri schools, on average. While the direct certification and school neighborhood poverty measures are aligned with each other, neither of them lines up with free or reduced-price lunch enrollment (see Figure 1). Compared to direct certification, free lunch enrollment flags 39 percent more children in a school as living in households with incomes at or below 130 percent of the poverty line, on average. Similarly, compared with school neighborhood poverty, 47 percent more children are flagged as living in households with incomes at or below 130 percent of the poverty line in the free lunch data.

We then assess alignment at the free-and-reduced-price enrollment threshold of 185 percent of the poverty line. Unfortunately, we cannot conduct the alignment test at this threshold using direct certification data because that threshold is 130 percent in Missouri. However, we can use the school neighborhood poverty data, which show that enrollment for free and reduced-price meals is also substantially oversubscribed, by about 40 percent.

Next, we explore how the Community Eligibility Provision figures into our findings. This provision, which was included in the 2010 reauthorization of the program and was rolled out nationally during the 2014–15 school year, subsidizes free meals for every student at participating schools and districts. To be eligible, a school or district must have at least 40 percent of students qualify for direct certification. For districts and schools in Missouri (and many other states) that adopt community eligibility, all of their students are reported as being enrolled for free meals. Community eligibility for free meals will certainly contribute to our finding that the poverty rate is overstated by data from the National School Lunch Program, but the magnitude of the effect is unclear.

To disentangle the effect of the Community Eligibility Provision, we incorporate data from the 2013–14 school year, just before the provision was implemented. Specifically, for schools that adopted community eligibility in our data from 2016–17, we use their free and reduced-price meal enrollment rates to the 2013–14 values. We leave the enrollment rates for non-participating schools unchanged. This exercise shows that while the Community Eligibility Provision has contributed to oversubscription in the free lunch category in recent years—as expected—it is not the primary driver. The provision explains 15 percentage points of the free lunch oversubscription and 9 percentage points of the free-or-reduced-price-lunch oversubscription in our data, or about one third to one fourth of the total oversubscription rates. The implication is that even before the Community Eligibility Provision, enrollment was greatly inflated relative to the National School Lunch Program’s stated income thresholds.

Finally, we consider school lunch and neighborhood poverty data from a larger 27-state sample to show that our findings are not unique to Missouri. This exercise involves different datasets and some additional assumptions, the details of which we provide in another publication. Suffice it to say here that the average oversubscription rates in this larger sample are close to the rates in Missouri. We conclude that the oversubscription of free and reduced-price lunch is likely endemic in the United States.

Figure 1: Free Lunch Enrollment Exceeds Share of Families in Poverty

Why Is Enrollment Oversubscribed?

The degree to which enrollment in the National School Lunch Program is oversubscribed is not well understood—but perhaps it should not be surprising. Outside of direct certification, free and reduced-price lunch enrollment is based on mostly unverified surveys. These are administered by school districts to parents, and both groups have incentives that encourage oversubscription.

Districts may be motivated by concerns about child welfare and academic performance; healthy, ample lunches during school contribute to both. But districts also may be incentivized to encourage and approve parent applications in order to gain access to additional federal, state, and local funding to support low-income students. Meanwhile, parents are incentivized to enroll their children because participation lowers their food costs.

In addition, the United States Department of Agriculture does not seem particularly interested in enforcing income eligibility rules. As noted by David N. Bass, only a very small number of applications go through an income-verification process (see “Fraud in the Lunchroom,” feature, Winter 2010). In fact, according to the department’s Eligibility Manual for School Meals in 2017, attempting to verify more than 3 percent of applications without special cause is prohibited. When eligibility is checked and cannot be verified, the student’s meal subsidies are discontinued, but there are no other consequences. The incentive structure clearly favors districts and parents stretching the boundaries of eligibility.

We do not want to go too far down the path of wondering why the federal agriculture department does not enforce its income-eligibility policies more strictly, much less whether it should. The most obvious explanation is that lax enforcement is a strategy to increase meal access for students in public schools, especially considering other initiatives to promote broader access to subsidized school meals, like community eligibility. This may be an appropriate approach to policy implementation given evidence that children benefit from expanded access to free and subsidized meals.

However, this highlights a fundamental problem with using these data to inform other consequential education policies: enrollment for free or reduced-price school lunch is not a reliable measure of family income. Rather, it is a measure that can be, and seemingly is, manipulated by administrators to promote their own objectives related to meal access and program participation. The end result is that school lunch data are a poor proxy for student poverty counts. The problem is not with the National School Lunch Program’s administration of its own program, but rather the education system’s reliance on enrollment data to achieve objectives for which the program and data were never designed—and are not maintained—to support.

The United States Department of Agriculture does not seem particularly interested in enforcing income eligibility rules for the school lunch program.
The United States Department of Agriculture does not seem particularly interested in enforcing income eligibility rules for the school lunch program.

A Policy Problem

The use of data from the National School Lunch Program in consequential education policies is ubiquitous. The most prominent example is in state funding formulas, which use free and reduced-price lunch enrollment as the basis for distributing billions of dollars to school districts every year. While states’ allocations of federal Title I aid to support low-income students are based on Census data, not National School Lunch Program data, school lunch data can affect the allocation of federal aid within states and school districts. State accountability policies that track achievement gaps by poverty status also commonly use free and reduced-price lunch enrollment to identify students in the “low-income” group.

The substance and scope of these policies suggest that the consequences of inaccurate school lunch data are significant. For example, consider a funding formula designed to allocate resources to students living at or below 185 percent of the poverty line. If oversubscribed school lunch data are used to proxy for this condition, our estimates from Missouri indicate that the number of students identified as low-income would be overstated by about 40 percent. If the resources to support low-income students are from a fixed budget set aside to support the true target population, the inflated count due to oversubscription would greatly dilute the resources available for each targeted pupil. And that would shift funding away from the most severely disadvantaged students.

Our findings support the position that the education system should move away from relying on National School Lunch Program meal designations as consequential measures of income status. In addition to showing that lunch program participation is greatly oversubscribed, we also note the possibility of substantial variation in oversubscription among school districts. This can lead to a situation where districts receive funding support that is more related to their success in soliciting applications that show eligibility rather than the actual number of low-income families they serve. To the extent that this variation exists, school districts that are more aggressive in signing up students or where parents are more engaged in the application process stand to gain more than districts that are less aggressive, even when their underlying levels of true poverty are the same.

Where do we go from here? Consternation among policymakers caused by the Community Eligibility Provision has led some states to change from using school lunch data to using direct certification data to count low-income students. Our findings in Missouri suggest this shift improves accuracy. However, a caveat to this result is that different states implement federal social-assistance programs differently, and those particularities make it difficult to project how broadly our findings will generalize outside of Missouri. The primary concern is how state policies differ regarding eligibility for food assistance through the Supplemental Nutrition Assistance Program, the primary program that leads to direct certification. Some states allow families with incomes of up to 200 percent of the poverty line to qualify for the program through Broad Based Categorical Eligibility. Missouri is one of a handful of states that does not have Broad Based Categorical Eligibility.

Variation across states in their policies regarding Broad Based Categorical Eligibility has two implications for poverty measurement using direct certification data. First, it means that direct certification status conveys different information about the level of poverty in different states. This has implications both for individual state policies and federal policies that affect multiple states. Second, it is unclear whether direct certification status will accurately measure the income thresholds intended by state rules, given differences among states in program participation and how income rules are enforced. For example, in some states, participation in Medicaid can lead to direct certification, but research shows that many Medicaid-eligible families do not participate in Medicaid. Concerns have also been raised about the fidelity with which Broad Based Categorical Eligibility criteria are enforced.

A larger conceptual concern is that income metrics based on direct certification share a critical flaw with metrics based on free and reduced-price lunch: they are not policy invariant. Like with data from the National School Lunch Program, the criteria that determine direct certification status are subject to continued change as policymakers target evolving policy objectives outside of the education system, not accurate poverty measurement.

Looking Ahead

Despite these concerns, direct certification data are likely the most feasible alternative to school lunch data to identify low-income students and implement education policies to support those students—at least in the short run. Over a longer horizon, we hope for more comprehensive solutions. One aspirational alternative would be to merge education data with tax data from the Internal Revenue Service, state tax agencies, or both, which could capture family income more accurately. This merge is technically feasible, and proof of concept has been established by recent research and at least one state policy. However, to adopt this as common practice would require overcoming political barriers and establishing new avenues of data sharing between agencies in most states.

In the more immediate term, it is worth considering policies that lessen the emphasis on flawed measures of family income in favor of broader indicators of student need. For instance, we could develop generalized measures of student disadvantage to inform education funding and accountability policies. Such measures could incorporate imperfect information on poverty from subsidized meal data and direct certification data but also include information about geographic mobility, attendance patterns, test and other school performance measures, and participation in remedial programs, among other factors. By considering these many facets of disadvantage together, we can improve measurement and expand our understanding of the broad range of need among students.

Ishtiaque Fazlul is a clinical assistant professor at Kennesaw State University. Cory Koedel is a professor at the University of Missouri, where Eric Parsons is an associate teaching professor.

This article appeared in the Spring 2023 issue of Education Next. Suggested citation format:

Fazlul, I., Koedel, C., and Parsons, E. (2023). A Poor Poverty Measure: To identify children in need, look beyond free lunch data. Education Next, 23(2), 48-53.

The post A Poor Poverty Measure appeared first on Education Next.

]]>
49716284
Gifted and Talented Programs Don’t Cause School Segregation https://www.educationnext.org/gifted-and-talented-programs-dont-cause-school-segregation/ Tue, 31 Jan 2023 10:00:12 +0000 https://www.educationnext.org/?p=49716240 Uneven enrollments, but minor impacts on racial separation

The post Gifted and Talented Programs Don’t Cause School Segregation appeared first on Education Next.

]]>

For decades, gifted and talented programs have offered small, selected groups of students enrichment and faster-paced lessons. They also have stoked controversy and allegations of contributing to racial segregation and academic inequality. New York City’s program, for example, was planned for virtual elimination in 2021 based on longstanding concerns about relatively low enrollment rates for Black and Latino students, who account for about 70 percent of all city students but 25 percent of gifted and talented students. After public outcry, the program was preserved, but with major changes: more classes, including in less-advantaged neighborhoods, and more pathways for students to qualify.

Racial segregation and racial gaps in student achievement in U.S. public schools are well-documented trends. So too are race-based differences in student enrollment in general-education versus gifted and talented programs. But are gifted and talented programs drivers of racial segregation? If so, to what extent?

To explore these questions, I look at the federal education department’s Civil Rights Data Collection surveys, which provide detailed data on the existence and racial composition of gifted and talented programs at virtually every elementary school in the United States. I focus on the period between 2009 to 2018 to investigate and compare the racial compositions of gifted and talented and general-education programs. I then apply standard indices of racial segregation to determine the extent to which gifted programs contribute to within-school segregation.

Overall, gifted and talented programs do disproportionately enroll more white and Asian students and fewer Black and Hispanic students. However, they have only a minor impact on racial segregation, in part because they enroll relatively small numbers of American schoolchildren. When I track enrollment changes at specific elementary schools before and after gifted programs are initiated or discontinued, I find virtually no impact on the percentages of white and Asian students. Gifted and talented programs are not a major contributor to racial segregation in U.S. elementary schools.

Who Is Gifted?

Gifted and talented programs have been a feature of American public schools for nearly a century. Nationwide, these programs enroll a relatively small share of students. In 2017–18, for example, 1.6 million elementary-school students were enrolled in gifted programs out of 23.6 million students overall, or 6.9 percent of total enrollment.

There is no single standard definition of giftedness. Instead, states and school districts apply locally selected measures of intelligence and ability to determine which students are accelerated relative to their peers. These encompass a broad spectrum of approaches: IQ, demonstrated ability in multiple intelligences, creativity and problem solving, and focus and task commitment. Some programs use screening tests to determine entry, while others are based on teacher recommendation and portfolio review.

The structures and operations of gifted and talented programs are similarly diverse. While no official data is collected on their basic operations, a 2019 national survey of more than 1,200 gifted and talented teachers and coordinators conducted by Education Week provides some insight into common practices.

The survey found that the most common method of delivering gifted and talented instruction was in “pull-outs,” where identified students are removed from the mainstream classroom for a portion of instructional time. Some 86 percent of gifted and talented educators reported using pull-outs compared to 32 percent reporting self-contained classrooms. The most common gifted and talented services were “content enrichment,” where instruction provided deeper coverage of grade-level topics, and “content acceleration,” in which students moved more quickly to new topics compared to their general-education peers down the hall.

There is little question that segregationists historically used within-school tracking programs like gifted and talented education as an intentional strategy to subvert legally required school integration. Many school districts in the South initiated test-based classroom assignments in the wake of strong school desegregation enforcement in the 1970s, for example. While many of these programs were successfully challenged in the courts, the general practice of ability grouping was not itself ruled unconstitutional.

Contemporary implementations of gifted and talented programming are rarely seen as explicit attempts to resurrect de jure racial segregation. But racial gaps in tests scores, as well as other common features of gifted and talented screening processes, such as the availability of fee-based aptitude test-score prep programs, have the strong potential to result in de facto racial imbalances in gifted and talented programs and contribute to overall racial segregation. In addition, research suggests that many common screening processes are subject to some degree of racial bias; for example, Jason Grissom and Christopher Redding found that Black students with high tests scores are less likely than similar non-Black students to be referred to gifted programs, especially when they are taught by a non-Black teacher.

Gifted and Talented Enrollment by Race (Figure 1)

Data and Method

My analysis is based on data from the U.S. Department of Education’s Civil Rights Data Collection surveys, which are conducted biennially and are mandatory for virtually every public school in the country. I focus on the five surveys administered from 2009–10 to 2017–18, which included consistent data about gifted and talented programs. These surveys collect information on a wide variety of school characteristics including enrollment, discipline, teacher characteristics, expenditures, and curricular offerings, and most of the data is disaggregated by student race and ethnicity, sex, English proficiency, and disability status. For this analysis, I consider only whether the school operated a gifted and talented program in each year, as well as the race-specific enrollments of the gifted and talented program (when present) and of the full school. I include public charter, magnet, and alternative schools offering any grade from K through 6 in all 50 states plus Washington, D.C., but not schools that offered only a special education curriculum or were not observed in all five surveys. The resulting data set includes 46,704 public elementary schools observed five times over a span of nine school years.

I calculate race-specific enrollments using the racial and ethnic categorizations that were collected in a consistent fashion across all survey years. I divide enrollment into two main groups: Hispanic and Black students, and white and Asian students. (The group of Hispanic and Black students also includes a small number of Native American students.) This allows me to identify racial differences in gifted and talented program enrollments and analyze how the racial composition of those programs affects racial segregation between and within schools.

Enrollment rates by race confirm that widely held perceptions about gifted and talented programs are correct. These programs disproportionately enroll white and Asian students compared to Black and Hispanic students.

Nationwide, the average gifted and talented program enrolls 60.1 percent white students and 8.2 percent Asian students. These students account for smaller shares of enrollment in non-gifted programs: 50.9 percent and 5.1 percent, respectively (see Figure 1). The opposite is true of Black and Hispanic students: Black students account for 11 percent of gifted and talented students but 17 percent of non-gifted enrollments, while Hispanic students account for 19.8 percent of gifted and talented students but 25.5 percent of non-gifted enrollments.

Because the racial composition of many gifted and talented programs does not resemble schools’ overall enrollment, critics have argued that such programs essentially constitute independent, racially segregated programs within supposedly integrated schools. This argument holds that standard between-school segregation measures substantially understate the true level of racial segregation within schools, and that eliminating gifted and talented programs therefore could be an effective desegregation strategy.

To evaluate this claim, I calculate two standard indices of segregation for all schools in the sample. I then do the same calculations as if gifted and talented programs are standalone, separate schools, to see how much gifted and talented programs contribute to racial segregation within schools.

First, I calculate the dissimilarity index. When applied to schools, this index measures how evenly members of different racial groups are distributed across different schools relative to a district’s overall enrollment. It can be interpreted as the share of students from one school who would need to move to another school in order to make the racial composition of each school match that of the district as a whole. The dissimilarity index therefore ranges from 0 to 1, with larger values indicating greater segregation.

Then I calculate the exposure index, which measures how intensively one group of students is exposed to another group. It can be interpreted as the probability among members of one racial group that a randomly selected peer will be from a different racial group. My analysis focuses on the exposure of Black and Hispanic students to white and Asian students, such that the exposure index values give the share of Black and Asian students’ peers who are white or Asian.  Note that unlike the dissimilarity index, larger values indicate less segregation.

To isolate the influence of gifted and talented programs on racial segregation, I first calculate these two indices between all elementary schools in the same district. I then re-calculate each index between all elementary schools and all gifted and talented programs within the same district. The latter measure therefore reflects both between-school segregation and any within-school segregation that results from gifted and talented programs. Finally, I calculate the difference between the two measures for both indices, which shows how racial segregation would change if gifted and talented programs were discontinued and those students returned to non-gifted classrooms at their schools.

Minimal Impacts on Segregation From Eliminating Gifted and Talented Programs (Figure 2)

Results

Gifted and talented programs do contribute to racial segregation—but not by very much. In looking at school districts that have gifted and talented programs, which includes about 70 percent of the total dataset, my analysis indicates that if these programs were ended and gifted students were returned to non-gifted classrooms, the value of the dissimilarity index would decline by about 0.03, or approximately 18 percent of its mean value.

Overall, racial dissimilarity between all public elementary schools in districts that have gifted and talented programs is 0.171 (see Figure 2). When I recalculate that index as though gifted and talented programs were their own separate schools, racial dissimilarity increases to 0.201. However, because most gifted programs operate as occasional pullouts, rather than standalone classrooms, this estimate likely overstates their contribution to racial segregation.

I also conduct the same calculations for all U.S. school districts, including those that do not have gifted programs. Racial dissimilarity between schools is 0.151 overall and 0.172 when gifted and talented programs are included in the analysis as separate schools, a difference of 0.021. Finally, I calculate dissimilarity for larger, diverse U.S. districts, which I define as serving more than 35,000 students and where Black and Hispanic students make up between 10 percent and 90 percent of enrollment. In these districts, racial dissimilarity is 0.439 overall and 0.452 when gifted and talented programs are included as separate schools, a difference of 0.013.

In looking at the exposure index, I find essentially no impact from gifted and talented programs on a Black or Hispanic student’s likelihood of having white or Asian students as classmates. In districts that have gifted and talented programs, the overall exposure index is 0.649. The exposure index is 0.643 when gifted and talented programs are included as separate schools, a difference of -0.006.

This may seem incongruent with the overrepresentation of white and Asian students in gifted and talented programs. However, remember that gifted and talented programs account for only 6.9 percent of total school enrollments, a relatively small share. In addition, 27.3 percent of gifted students are Black and Hispanic. While that is a smaller percentage than overall Black and Hispanic enrollment of 47.7 percent, it is still a substantial number of students relative to overall gifted and talented enrollment.

I then look at how starting or ending a gifted and talented program affects a school’s racial composition. Public debate has focused on whether these programs disproportionately attract and retain white and Asian students who might otherwise enroll in other schools, so I focus on changes in white and Asian student enrollment in the years before and after a program is added or discontinued. About one-fourth of schools, or 12,037 out of 46,704 total, either initiated or eliminated a gifted and talented program during the study period. My analysis tracks these trends for program starts and cancellations in 2012, 2014, 2016, and 2018, resulting in eight specific event studies.

I do not find any consistent evidence that gifted and talented programs have a causal effect on schools’ race-specific enrollments (see Figure 3). None of the eight studies reveal a trend in white and Asian enrollment after the elimination or initiation of a gifted and talented program. In addition, there are no indications that gifted and talented programs are started or discontinued in response to changing racial compositions.

No Major Effect on Enrollments When Gifted Programs Start or End (Figure 3)

Questions to Consider

My analysis indicates that gifted and talented programs are a small or negligible contributor to racial segregation in U.S. elementary schools. Eliminating all gifted and talented programs nationally would have a minimal impact on standard measures of racial segregation, and the presence of a gifted and talented program does not appear to causally impact the racial composition of enrollments over time.

One caveat of these findings is that while the reductions in segregation that could potentially be achieved through modifying gifted and talented offerings may be modest overall—at approximately 18 percent with respect to the dissimilarity index and close to zero for the exposure index—policy changes related to gifted and talented programming may be more practical to implement than those affecting the sorting of students between schools. For example, busing programs or redrawing enrollment zones often decided at higher levels and are extremely contentious. Given this, although changing gifted and talented programing can certainly be controversial as well, it may be a relatively actionable step that district or even school-level policy makers can undertake to modestly remediate racial segregation.

Another caveat is that gifted and talented education is primarily a feature of elementary schools. Previous research has found that within-school segregation is less extensive in primary schools than in secondary schools. An analysis of tracking and racial segregation at the high-school level, such as in Advanced Placement or International Baccalaureate classes, might yield different conclusions. A third caveat is that there may be subtle interactions between racial segregation between and within schools. For example, an analysis of Hispanic student enrollment in North Carolina classrooms found districts with less segregation between schools also have more segregation within schools. Further research on this and related patterns would be valuable.

Nonetheless, my findings suggest that any impacts of gifted and talented programming on racial segregation at the elementary-school level are likely to be minimal. The questions facing school and district leaders, then, are whether these selective programs benefit those young students identified as gifted or harm the students who are not. While the analyses reported here do not directly prescribe whether gifted and talented programs are a desirable overall education policy, they do indicate that the effects of existing gifted programs on racial segregation should not be a first-order policy consideration.

Owen Thompson is an associate professor at Williams College.

This article appeared in the Spring 2023 issue of Education Next. Suggested citation format:

Thompson, O. (2023). Gifted and Talented Programs Don’t Cause School Segregation: Uneven enrollments, but minor impacts on racial separation. Education Next, 23(2), 54-49.

The post Gifted and Talented Programs Don’t Cause School Segregation appeared first on Education Next.

]]>
49716240
Assessing Integration in Wake County https://www.educationnext.org/assessing-integration-wake-county-north-carolina-loud-debate-muted-effect-students-schools/ Tue, 18 Oct 2022 09:00:46 +0000 https://www.educationnext.org/?p=49715914 Loud debate, but muted effects for students and schools

The post Assessing Integration in Wake County appeared first on Education Next.

]]>

IllustrationFor decades, large public-school systems in the United States have used student assignment policies to foster more diverse school enrollments. Such efforts, sometimes pursued under court order, seek to expand educational opportunity and counterbalance the patterns of residential segregation that contribute to racial and economic isolation, especially in urban centers. They may receive a new source of federal support: the Biden Administration has proposed a $100 million Fostering Diverse Schools grant program to help communities “voluntarily develop and implement strategies that will build more racially and socioeconomically diverse schools and classrooms.”

“Research suggests that diverse learning environments benefit all students and can improve student achievement, serve as engines of social and economic mobility, and promote school improvement,” Education Secretary Miguel Cardona said in his budget testimony.

One common strategy to create more diverse learning environments is to intentionally balance school enrollments according to students’ socioeconomic and demographic characteristics. Districts can achieve this by physically transporting students to schools outside their immediate neighborhoods or by prioritizing the enrollment of students in schools where they would diversify the student body. The intent is to expand opportunities for students from diverse backgrounds to learn alongside one another, in order to reduce the prevalence of segregated schools, allocate resources more equitably, and improve student outcomes. Indeed, research has found benefits for educational achievement, attainment, and other measures of well-being for Black students where schools were desegregated and resources more equitably distributed.

But school integration initiatives (sometimes referred to as “mandatory busing” or just “busing”) in New York City, San Francisco, Charlotte-Mecklenburg in North Carolina, and elsewhere have sparked substantial backlash, particularly among more affluent families. Their concerns often echo a longstanding claim that school reassignments destabilize communities and exact a social or educational toll on reassigned students and their peers. For example, in 2019, community members submitted hundreds of overwhelmingly negative comments to Maryland’s Howard County Public School System in response to a proposed plan to redraw attendance boundaries to integrate its schools. Among the comments, a prediction: “The only result you will find is more time commuting to school, humiliation, intimidation. Busing children WILL NOT increase individual grade-point averages. In fact, it may decrease all those objectives.”

Critical to informing this debate is a comprehensive answer to the question: How does reassigning students to create schools that are more socioeconomically and academically diverse affect the distribution of educational opportunity? What are the impacts on students who switch schools as a result of these policies? And how do changes in school assignments affect the students who don’t switch schools, but who experience changes in their classmates’ characteristics?

We report the results from two distinct studies of North Carolina’s Wake County Public School System, which has a long history of using student assignment policies to weaken the school-neighborhood links that exacerbate school segregation. Our research teams, one based at the University of North Carolina at Chapel Hill (UNC) and the other originating at the Center for Education Policy Research at Harvard University, have worked closely with the district to better understand how student assignment policies affect academic and behavioral outcomes and how changing the demographic characteristics of a student’s peers affects learning.

We find that, on the whole, school reassignment has somewhat muted effects. In contrast to the sharp criticism and heated controversy that integration programs often inspire, switching schools does not harm students who are reassigned. In fact, reassigned students perform modestly better on statewide tests and are less likely to be suspended. We do find some negative effects for students who switch to schools where achievement and income levels are lower, but these effects are offset by positive impacts for students when school reassignments mean they learn alongside higher-performing and wealthier peers. However, these impacts are small, because, in most cases, students’ new schools are largely similar to the schools they left behind. Put another way, the impacts of school integration rely more on the destination than the departure.

A Decade of Dizzying Growth

Our research emerges from the deep and longstanding commitment to evidence-informed policymaking by district leaders in Wake County. Multiple teams of university-based researchers have cooperated with district staff over the past two decades. Our studies are unique, in that two teams pursued partially overlapping research topics. Taken together, these studies complement one another and provide a more comprehensive assessment of a longstanding student reassignment policy in one of the largest districts to implement such a plan. Satisfyingly, where our questions align, so, too, do our findings, which we believe can serve as a model for how future evaluations of high-priority policy topics might proceed.

Both studies examine school reassignment policies and effects during an era of rapid growth and demographic change in Wake County. Between 2000 and 2010, the number of students jumped by nearly 50 percent, to 143,289 from 98,741. The share of Hispanic students more than tripled, to 13 percent from 4 percent, while the percentage of Asian students almost doubled to 6 percent from 3 percent. During that time, the percentage of white students fell to 51 percent from 64 percent, while the share of Black students shrank slightly to 25 percent from 26 percent. As a result of the population growth, the district opened 40 new school buildings during the decade, most of which were located in relatively more affluent neighborhoods in the county’s suburban fringe. Nearly one quarter of all students were reassigned during the study period.

The UNC study looks at how reassignments affect the 23.9 percent of students who were asked to change schools, including how reassignment changed the characteristics of the schools they attend. It does not find that reassigned students who change schools are adversely affected. On average, after changing schools, reassigned students travel shorter distances and attend higher-performing schools. Their academic performance does not suffer after changing schools, and, in some cases, it actually improves by a small but significant degree. Critically, these results suggest that concerns about the negative consequences of school reassignment for those who were reassigned may be overblown.

The Harvard study asks a different set of questions. What about the larger group of students who don’t switch schools, but whose peer groups are changed due to wide-
scale reassignments? Their academic performance improves too—but only if their peer group changes to include more high-achieving classmates. Students from more affluent families and already high-achieving students benefit the most from peer groups that are also academically high-achieving. But students with lower family incomes and lower baseline academic achievement also benefit from being in class with academically stronger peers. The team does find that students earn lower grades in reading when they experience an influx of higher-performing peers, possibly due to teachers’ practices of relative-rank grading. But overall, the picture is positive.

In particular, our results indicate that students who learn alongside more high-achieving students as a consequence of school reassignment policies have better academic achievement. These results suggest that a policy that reassigns students to optimize the average peer achievement level of less-advantaged students can help accomplish equity goals but, under certain conditions, may also have unintended consequences, particularly for their more-advantaged peers.

A Voluntary Desegregation Effort

Unlike many other large school districts in the South, Wake County was never the subject of court-ordered desegregation. But in the 1970s, under federal pressure to integrate, the majority-Black Raleigh City Schools and majority-white surrounding county district merged to form the Wake County Public School System. To balance the enrollments of the district’s schools, leaders used students’ race as a primary factor in school assignments until a federal court decision ended mandatory desegregation efforts in the nearby Charlotte-Mecklenburg Schools.

In 2000, Wake County shifted to using students’ socioeconomic status and levels of academic achievement in school assignment decisions instead of race. As part of its strategy for meeting these targets, the district was divided into geographic nodes of about 150 students. The district assigned each node to a base elementary, middle, and high school, which served as the default school of attendance for students in the node.

To maintain socioeconomic and achievement balance, Wake County reassigned a small share of students to different base schools each year based on their grade level and node. That is, in a particular year, the policy would assign all students in the same grade and node to the same school. The goal was to balance enrollments such that no school would serve a student body with more than 40 percent of students receiving free or reduced-price lunch and more than 25 percent of students reading below grade level.

Reassignments occurred throughout the district, including nodes in the district’s urban core as well as rapidly expanding suburban nodes at the district’s northern and southern peripheries. Nodes with high concentrations of Black and Hispanic students were more likely to experience reassignment than nodes that included predominately white students (see Figure 1). The district also used reassignments to populate newly constructed schools, most of which were located in relatively affluent, high-growth neighborhoods. These shifts affected between 2 percent and 8 percent of students annually.

Shifting School Assignments in Wake County, 2000-2010 (Figure 1)

District decisionmakers used a range of criteria when reassigning groups of students, including travel distances, capacity constraints, and diversity considerations. These recommendations were presented for community feedback alongside options for students to attend magnet and year-round schools. The annual reassignment process kicked off more than a year in advance, and parents had at least six months to decide whether to accept the newly assigned school or appeal the decision.

This process created several groups of students: students who were reassigned and moved; students who were reassigned and did not move; and students at sending and receiving schools who experienced different peer groups because of node reassignments. In addition, students could choose to attend year-round or magnet schools, though students were guaranteed door-to-door transportation to their base schools but not to magnet programs. About two thirds of Wake County students attended their base schools from 2000 to 2010.

The district’s approach to reassignment enables both research teams to overcome a key challenge in measuring the impacts of school switching or the influence of peers on one another’s learning. Simply examining the outcomes of any reassigned students who change schools or not-reassigned students who share classrooms with peers of different demographic backgrounds would fail to account for unobserved differences that could influence outcomes. In our studies, however, we can compare the outcomes of students in nodes selected to switch schools to those in otherwise similar nodes who were not chosen to change schools. After accounting for the observable characteristics used to inform the assignment process, groups of students from adjacent nodes were selected in an arguably random process to attend different schools. Similarly, to understand the effects of learning with peers from different backgrounds, we examine the outcomes of students who remained in the same school but experienced an as-good-as-random reshuffling of peers assigned into and out of their classes. As a result, both teams have plausible claims to a causal interpretation of their findings.

Impacts on Reassigned Students

Those of us on the UNC team examine how reassignment affected the new schools that students attended as well as its impact on reassigned students’ academic achievement, attendance, and school discipline. We compare student outcomes in the pre-reassignment period to those outcomes in post-reassignment periods and then benchmark those differences against the trend for nodes that were never reassigned. Our analysis is based on district data from 1999–2000 to 2010–11, which includes students’ basic demographic and academic characteristics, home addresses, geographic nodes, and school assignments.

Despite concerns about the potential harms of reassigning students to achieve diversity goals, we find no evidence that reassignment negatively affected student outcomes. In some cases, reassignment modestly boosted achievement and protected students against exclusionary discipline.

First, we examine how reassignment affects the characteristics of students’ assigned schools. Looking at the effects on distance, we find that reassignment reduces the distance between a geographic node and assigned school by one fifth to one half of a mile, on average. However, this surprising overall result masks heterogeneity across racial and ethnic groups. While distances for reassigned white students decline by roughly one mile, distances for reassigned Hispanic students increase by about one mile. There is no change in average travel distance for Black students.

Overall, reassignment results in students attending schools with somewhat higher math achievement, though we find substantial variation across racial and ethnic groups. On average, test-score performance in math at schools attended by reassigned students is 0.02 to 0.05 standard deviations higher compared to their previous schools. For white students, however, math achievement is between 0.02 and 0.07 standard deviations higher at new schools. Black students attend schools where achievement is initially lower than at their previous schools, but this effect shrinks over time. The differences range from 0.05 standard deviations lower one year after reassignment to 0.004 standard deviations lower two years later. There is no change in average school performance for reassigned Hispanic students. In addition, the proportion of students of color is lower in schools attended by reassigned Black, Hispanic, and white students, suggesting that only white students are more likely to be reassigned to schools that included more students who share their racial identity.

We then look at how reassignment affects students who change schools, and we find encouraging results. After reassignment, students’ math achievement improves by a modest amount in all three post-reassignment years, ranging from 0.02 standard deviations in year one to 0.05 standard deviations in year three (see Figure 2). Reading achievement is initially flat but improves by 0.02 standard deviations in year two. Students who are reassigned are also less likely to experience exclusionary discipline in the first post-reassignment year and are no more likely to be chronically absent than before they switched schools. The impact on suspensions is particularly encouraging in light of emerging efforts by policymakers to combat disciplinary practices that disproportionately harm students of color (see “Proving the School-to-Prison Pipeline,” research, Fall 2021).

Effects of Reassignment for Students Who Change Schools (Figure 2)

Wake County students also switch schools to attend the district’s rich set of magnet and year-round calendar schools; during the study period, about one-third of district students attend a public school of choice. We explore whether the main results differ depending on whether reassigned students attend their reassigned base school or opt to attend a magnet or year-round public school. Encouragingly, we find that the effects for achievement, absenteeism, and suspension are broadly similar whether reassigned students attend new base schools or schools of choice.

Peer Effects on Students Who Don’t Switch Schools

Those of us on the Harvard team focus here on students who do not change schools. Because students in similar geographic nodes were not uniformly reassigned, school enrollment changes serve as a series of natural experiments that allow us to compare students’ performance across years in which they experienced more- or less-affluent or higher- or lower-achieving peers.

Unlike students who switch schools, the only aspect of non-reassigned students’ schooling that changed was their peers. Thus, we can home in on just the phenomenon of changes in composition of these students’ classroom resulting from peer in- and out-flow produced by the reassignment policy. Our analysis includes district data from 2005–06 to 2011–12 (a shorter window than the UNC study), when academic standards, tests, and district-assignment policies were relatively stable. We focus on students in grades 7 and 8, who have two years of annual test-score data and typically do not change schools unless they are reassigned.

We find that middle-school students’ academic skills, as measured by standardized test scores, improve when they attend school with higher-achieving peers. Overall, when peer achievement increases by 0.10 standard deviations, students’ test scores increase by 0.04 standard deviations in math and 0.03 standard deviations in reading (see Figure 3).

Impacts of Changing Peer Groups on Students Who Are Not Reassigned (Figure 3)

We also look at non-reassigned students by family income and prior test-score performance. Peer effects are largest for wealthier students, whose test scores increase by 0.05 standard deviations in math and 0.03 standard deviations in reading when their peers’ achievement increases by 0.10 standard deviations. Students with higher test scores get the biggest gains in math from attending school with higher-achieving peers. Students with lower family incomes and lower baseline levels of achievement also benefit from academically stronger peers. When peer achievement increases by 0.10 standard deviations, lower-income students’ test scores increase by 0.02 standard deviations in math and 0.01 standard deviations in reading. Similarly, students with low prior achievement improve their math and reading scores by 0.04 and 0.03 standard deviations, respectively, when they have higher-achieving peers in their class.

In looking at students’ grades, we find differences in peer effects between math and reading. When students attend school with more higher-performing and affluent peers, their math grades go up by 0.02 and 0.20 standard deviations, respectively. We find that benefit throughout the performance distribution. But non-reassigned students’ English grades decline by 0.03 standard deviations when their peers are higher performing. This may be due to the different ways that students are graded in reading and math. In English Language Arts classes, teachers may look at how students stand in relation to other students in the class, while math grades may depend more directly on objective mastery of the material.

One important caveat to our results is that only a small number of student reassignments demonstrably changed the achievement and family income levels of students’ peers, both for students who were reassigned and those who experienced new peers but did not themselves change schools. The majority of reassignments were intended to address the rapidly expanding student population. As such, we do not present our study as an evaluation of comprehensive policies that redistribute students to schools for the purposes of socioeconomic or academic integration. Rather, it looks at the narrower topic of changing a particular child’s assigned school or peer group.

Lessons in Complexity

The Wake County Public School System’s recent history of school integration policies represents one of many such efforts occurring nationwide. The Century Foundation recently reported that 185 charter schools and school districts are actively implementing voluntary or court-ordered integration policies based on select demographic or socioeconomic criteria. Our joint research projects represent a deep dive into one large district’s policy, with implications for stakeholders and policymakers pursuing equity through integration efforts in schools.

Of course, each district’s experience is unique. For more than three decades, Wake County implemented a student assignment policy that aimed to prevent school segregation and enjoyed widespread popularity. The recent iteration of the policy resulted in roughly 2 percent to 8 percent of students being reassigned in any given year—many for purposes other than integration. This represents an incremental approach that, at times, led to loud, negative headlines but relatively muted impacts. Still, the policy ran into political headwinds and was phased out following the 2009 board election, which featured a large influx of national attention, organizing, and funding not typically seen in local education races.

Districts implementing their own policies or considering new ones should set expectations for equity and achievement that are commensurate with the scope of any particular policy levers. As decisionmakers consider various factors in the policymaking process (including parental preferences and political feasibility), they should be aware of the implications of our research. Modest reassignment policies lead to modest changes in students’ peer groups, which together produce modest—although mostly positive—results. Bolder interventions may produce more meaningful effects. But they will require a broad spectrum of stakeholder support.

Given the targeted reach of Wake County’s integration policy, we are not surprised to see empirically small achievement impacts on students who were reassigned. The results from both studies suggest that shifting small numbers of students might marginally improve achievement for reassigned students in the aggregate, but also lead to unintended impacts for some groups of marginalized students or the widening of opportunity gaps. Examples of such unintended consequences include longer travel times for Hispanic students, declines in performance for low-performing students, and some peer-learning benefits that accrue disproportionately to students from more affluent families. The relatively small impacts we detail stand in contrast to the often hyperbolic discourse that accompanies school-integration debates, with critics arguing that reassignment has large and persistent deleterious effects for students who are asked to change schools.

Our work also highlights how peers influence their classmates’ learning. Increasing the overall proportion of high-achieving and wealthy students is likely to increase student achievement. However, these benefits tend to be concentrated among students who are already high-achieving and do not have low family incomes. And relying on changes in classroom composition alone as a mechanism to improve student outcomes will pose challenges.

First, consider Matthew Kraft’s estimate of the median effect of educational interventions on student test scores: 0.10 standard deviations. By our calculations, accomplishing that median impact would require vast changes in school composition. Students would have to switch to schools where their peers performed 0.20–0.25 standard deviations higher and where the share of more-affluent families was 10 percentage points greater compared to their previous schools. This is no small feat.

In addition, our findings imply that reassignment policies such as the one we study may have some unintended consequences. Specifically, our findings imply that—absent mitigation efforts—some students may learn less when they study in classrooms with more lower-achieving peers as a result of reassignment policies. To limit this potential outcome, teachers and school leaders in locales seeking to use reassignment for equity purposes will need to attend to the needs of higher- and lower-achieving students alike. For example, higher-achieving and higher-family-income students may benefit from community-service extension activities and differentiated instruction. At the same time, lower-achieving and lower-family-income students may benefit from school leaders scrutinizing grading practices and ensuring that their schools are designed to support traditionally underserved learners.

Historically, children from different racial, socioeconomic, and achievement backgrounds leave school with different life opportunities available to them, and resource inequalities in the country’s K–12 educational system contribute to these inequities. The results of our respective studies suggest that student assignment policies that relocate students to optimize the average peer achievement level of lower-achieving or less-affluent students can accomplish equity goals. But school systems will need to employ strategies beyond reassignment to accomplish such goals, not least of which will be to build the political will to implement and sustain such integration policies in their communities.

James S. Carter III is Senior Education Data and Research Associate at the Urban Institute. Rodney P. Hughes is assistant professor at West Virginia University. Matthew A. Lenard is a doctoral candidate at the Harvard Graduate School of Education. David D. Liebowitz is assistant professor at the University of Oregon. Rachel M. Perera is a fellow in the Brown Center on Education Policy at the Brookings Institution. This article is based on the study “The Kids on the Bus,” published in the August 2021 issue of the Journal of Policy Analysis and Management, and the National Bureau of Economic Research working paper “New Schools and New Classmates,” issued in May 2022 and forthcoming in the Economics of Education Review.

This article appeared in the Winter 2023 issue of Education Next. Suggested citation format:

Carter III, J.S., Hughes, R.P., Lenard, M.A., Liebowitz, D.D., and Perera, R.M. (2023). Assessing Integration in Wake County:Loud debate, but muted effects for students and schools. Education Next, 23(1), 60-67.

The post Assessing Integration in Wake County appeared first on Education Next.

]]>
49715914
Are Two Teachers Better Than One? https://www.educationnext.org/are-two-teachers-better-than-one-effect-co-teaching-students/ Tue, 13 Sep 2022 09:00:59 +0000 https://www.educationnext.org/?p=49715700 The effect of co-teaching on students with and without disabilities

The post Are Two Teachers Better Than One? appeared first on Education Next.

]]>
5th-grade math teachers Brittney Bentley and Nicole Plowman co-teach a multiplication lesson at Lucy Laney Community School in Minneapolis.
5th-grade math teachers Brittney Bentley and Nicole Plowman co-teach a multiplication lesson at Lucy Laney Community School in Minneapolis.

For nearly 50 years, special education law has mandated that students with disabilities be served in the “least restrictive environment” possible. This often takes the form of an inclusive classroom, or a general education classroom where students with disabilities learn alongside their non-disabled peers. In some cases, inclusive classrooms are co-taught by a general education teacher and a special education teacher who share planning and instructional responsibilities.

The logic behind co-teaching is intuitively appealing. Co-teaching reduces the student-teacher ratio, and the presence of two educators, each with distinctive expertise, should make it easier to connect students at a range of abilities to grade-level content. But how well does this approach actually work on the ground? Do students with disabilities benefit from the presence of an additional teacher in the classroom? And how does co-teaching affect learning for students without disabilities?

We examine a decade of test scores for students in Massachusetts, where co-teaching has experienced rapid growth, and find positive effects on academic achievement for students with and without disabilities in the years they are enrolled in co-taught classes. For students with disabilities, attending a co-taught classroom boosts test scores by 2.6 percent of a standard deviation in math and 1.6 percent of a standard deviation in reading, on average. For students without disabilities, test scores improve by 1.2 percent of a standard deviation in math, while reading scores are not affected. This is the case even though students without disabilities who never participate in a co-teaching classroom have higher math and reading scores, on average, than their peers who do.

At the same time, the gains we find are much smaller than those reported in prior research on co-teaching, which involved small samples and focused on short-term results (see “A Charter Boost for Special-Ed Students and English Learners,” research, Spring 2020). By contrast, our analysis looks at co-teaching as it is currently implemented across an entire state, tracks many students over a long period of time, and compares their rates of learning in years they did and did not attend a co-taught classroom.

While co-teaching is broadly popular among educators, its effectiveness for improving student outcomes depends on a key assumption—that the presence of a second adult results in more effective learning opportunities for students. Our findings appear more consistent with studies suggesting that just putting two teachers in the same room does not necessarily improve the quality of instruction students receive. In practice, co-teachers often do not work in the idealized way advocates of the approach recommend. Colocation does not necessarily cause effective collaboration.

Nonetheless, we do find some evidence of positive effects on student learning, even in an apparently less-than-perfect form. Whether the impact of co-teaching at such a scale can be improved by encouraging more consistent application of best practices in co-teaching is an important area for future research.

A Growing Share of Students in Co-Taught Classrooms (Figure 1)

Examining a Team Approach

To investigate the impact of co-teaching on academic achievement, we focus on Massachusetts, where the Department of Elementary and Secondary Education encourages co-teaching but has not adopted it as a statewide initiative. Our analysis uses administrative and test-score data from 2007–08 through 2017–18 for teachers and students enrolled in grades 3 through 8. This includes students’ demographic and socioeconomic data, students’ scores in math and reading on the Massachusetts Comprehensive Assessment System exams, teachers’ job-assignment classifications, and information on classroom assignments that link students with their teachers. We also consider state job-classification data, which identifies a “teacher” as an employee who provides instruction, learning experiences, and care to students during a particular period or in a given discipline and a “co-teacher” as a teacher who is equally responsible with another teacher for providing those same services.

In most cases, schools do not adopt co-teaching as a uniform policy that applies to all students, but rather offer a mixture of co-taught and single-teacher classrooms. We identify classrooms as co-taught based on multiple teachers being assigned to a single class in the same year. In 2011, less than 10 percent of Massachusetts schools educated students with disabilities in co-taught classrooms. By 2018, nearly 30 percent of schools educated at least some students with disabilities in co-taught classes.

The percentage of elementary- and middle-school students educated in co-teaching classrooms has grown sharply in recent years. Between 2011 and 2018, the percentage of all Massachusetts students educated in co-taught classrooms grew to 9.6 percent from 1.8 percent (see Figure 1). That growth was especially sharp in 5th-grade reading, where the percentage of co-taught students increased tenfold. The percentage of 5th-grade students with disabilities educated in co-taught classrooms grew to 26.8 percent from 3.1 percent. Among 5th graders without disabilities, the percentage in co-taught classrooms grew to 19.7 percent from 2.4 percent.

Importantly, most students who enter a co-taught class do not remain in that environment for the rest of their academic careers, but rather switch between co-taught and single-teacher classrooms over time. Because the goal of our study is to understand how attending a co-taught classroom instead of a classroom headed by a single teacher affects a student’s learning, we compare these “sometimes” co-taught students against themselves, both for students with and without disabilities. Our analysis considers whether a student earns higher or lower math and reading scores on state tests in years when they experience co-teaching compared to years when they are in a single-teacher classroom, controlling for characteristics of the student’s school.

Compared to students who never experience a co-taught classroom, students who are sometimes in a co-taught classroom—whether or not they have disabilities—are slightly more likely to be Hispanic, eligible for free or reduced-priced lunch, and have substantially lower math and reading scores, on average. Because decisions about offering co-teaching classrooms are made at the school level, these differences could stem from differences in the characteristics and leadership of schools that adopt co-teaching, as well as from schools’ decisions about which classrooms should use the approach.

Higher Test Scores in Co-Taught Classrooms for Students With and Without Disabilities (Figure 2)

Results

Attending a co-taught classroom improves test scores for students with and without disabilities, especially in math. Test scores for students with disabilities are 2.6 percent of a standard deviation higher in math and 1.6 percent of a standard deviation higher in reading when they are in co-taught classrooms, on average (see Figure 2). The positive impact on math achievement is larger in middle school than in the elementary grades, at 3.6 percent of a standard deviation compared to 1.4 percent in elementary grades. The impact on reading scores does not differ between elementary and middle school.

For students without disabilities, attending a co-taught classroom leads to a significant increase in math scores of 1.2 percent of a standard deviation. We find no significant effect of co-teaching on reading scores for students without disabilities. In looking separately at elementary- and middle-school grades, attending a co-taught classroom improves reading scores by 0.4 percent of a standard deviation in elementary school and reduces scores by 0.6 percent of a standard deviation in middle-school grades. However, these results are not statistically significant.

We also examine our results based on the characteristics of students in a class, including the percentage of students with disabilities in the class, and find little impact on the estimated effects of co-teaching. The estimated effect on reading in middle-school grades appears larger for females with disabilities at 2.8 percent of a standard deviation compared to 0.6 percent of a standard deviation for males with disabilities. We also find that the impact on reading scores in elementary grades is larger for Black students with disabilities compared to white students with disabilities, at 5.1 percent of a standard deviation versus 0.7 percent, respectively. However, neither of these differences is statistically significant. In looking across disability categories, we find positive effects from co-teaching within each disability type (see Figure 3).

We also look at the test scores for students with disabilities in co-taught classes compared to those for students in two types of single-teacher classes: special education and general education. We consider classes specific to special education if they are led by a special education teacher and predominately enroll students with disabilities. In no case do we find a significant difference between the effect of attending a special education versus a general education class with a single teacher. We interpret this as evidence that the effect of a student with disabilities attending a co-taught classroom is similar regardless of whether the student would otherwise be “mainstreamed” in a general education classroom without a co-teacher or enrolled in a self-contained special education class.

Effects of Co-Teaching on Test Scores by Disability Type (Figure 3)

More Than the Sum of Its Parts

We find that co-teaching has positive effects on academic achievement for students with and without disabilities, but the size of the impact on students with disabilities is substantially smaller than those reported in prior studies. Our results show that co-teaching can produce some benefits when implemented at scale, but they do not appear consistent with the enthusiasm that surrounds the practice in special education literature.

However, our analysis is limited to the effect of co-teaching on student test scores. Some of the most important justifications for moving students with disabilities into inclusive general education classrooms, which co-teaching can facilitate, are not academic. Inclusive environments like co-taught classrooms can foster tolerance and understanding among typically developing students, while also supporting students with disabilities to practice and develop social skills and build friendships with a broad group of peers.

That we analyze co-teaching within the context of a large public-school system is arguably both our study’s most valuable contribution and its most substantial limitation. Proponents of co-teaching might reasonably argue that the magnitude of our estimates is muted by schools and teachers that have moved students into classrooms where teachers are purportedly co-teaching but, in reality, do not apply the best practices necessary for co-teaching to be effective. Indeed, differences in the fidelity of implementation when co-teaching is considered across an entire state rather than within the context of a single school or classroom could at least partially explain why our findings are so much smaller than prior estimates. It is also primarily for this reason that, though co-teaching certainly reduces the adult-to-student ratio within a classroom, we caution against interpreting our results for the effect of co-teaching as the effect of substantially reducing class size.

Co-teachers Ann Renee Evans and Cassandra Hinson lead kindergarten storytime at Connerton Elementary in Land O' Lakes, Florida.
Co-teachers Ann Renee Evans and Cassandra Hinson lead kindergarten storytime at Connerton Elementary in Land O’ Lakes, Florida.

From a policy perspective, whether our findings support a decision to adopt co-teaching as an instructional model depends on the associated costs, which are thus far unknown. Transitioning to a co-teaching model almost certainly would require additional resources and personnel, which would appear to be a costly proposition. However, if considered within the broader topic of special education spending, using co-teaching to create inclusive general education classrooms may prove to be an efficient model. Prior research has found that serving students with disabilities in inclusive general education settings with team teaching costs less than sending students to standalone programs or classrooms.

School leaders and policymakers may consider whether the relevant comparison is between single-teacher and co-taught class costs, or between inclusive and non-inclusive special education programs. We encourage the field to continue building a more robust evidence base surrounding the services schools provide to the especially vulnerable group of students with disabilities.

Nathan Jones and Marcus A. Winters are associate professors at Boston University.

This article appeared in the Winter 2023 issue of Education Next. Suggested citation format:

Jones, N., and Winters, M.A. (2023). Are Two Teachers Better Than One? The effect of co-teaching on students with and without disabilities. Education Next, 23(1), 54-59.

The post Are Two Teachers Better Than One? appeared first on Education Next.

]]>
49715700
Estimating the “Effective Teaching Gap” https://www.educationnext.org/estimating-the-effective-teaching-gap/ Tue, 06 Sep 2022 09:00:43 +0000 https://www.educationnext.org/?p=49715689 Students Experience Unequal Outcomes, but Mostly Equal Access to High-Quality Instruction

The post Estimating the “Effective Teaching Gap” appeared first on Education Next.

]]>

IllustrationInequality in educational outcomes is substantial and persistent in the United States. Students from high-income families outperform those from low-income families on achievement tests, are more likely to graduate high school, and are more likely to earn a college degree. Black and Hispanic students also earn lower scores on standardized tests, on average, and are less likely to graduate high school and go to college than white and Asian students.

While there are many possible explanations for these differences, one frequent hypothesis is that high-income white and Asian students are taught by more effective teachers. After all, evidence shows that teachers vary a great deal in their impacts on student learning, and that students taught by the best teachers have higher test scores and better outcomes in adulthood, including greater likelihood of college attendance and higher wages.

Studies also have found that teachers working with low-income students, on average, tend to be less experienced and have fewer qualifications that teachers working in high-income communities. In response, federal law currently requires states to ensure low-income students “are not served at disproportionate rates by ineffective, out-of-field, or inexperienced teachers,” and states like Washington offer bonuses to teachers with advanced credentials who work in high-poverty schools. However, more experience and better qualifications do not guarantee better teaching.

We look at student demographics and several measures of teacher quality in 26 public school districts across the United States over a five-year period. We find that, in fact, low- and high-income students have nearly equal access to effective teachers. Effective teachers are found in high-poverty schools, even if their accomplishments are often overlooked because their students typically start out far behind. Conversely, ineffective teachers can be found in high-performing schools, where the impacts of subpar instruction can be camouflaged by students’ other advantages.

Our analysis also suggests that it would take wholesale reassignment of the most effective teachers to the least advantaged students to substantially reduce inequities in learning outcomes, and that differences in the likelihood of low-income and minority students being taught by a novice teacher contribute a negligible amount to gaps in student achievement. The inequitable outcomes experienced by low-income and minority children may have less to do with their teachers and more to do with the supports and resources available to children of greater means.

Which Students Have High-Quality Teachers?

If low-income students were more likely to have less-effective teachers year after year, key questions would include how the effects of those teacher assignments accumulate over time and what contribution that would make to the student achievement gap. To explore these questions, we developed the “effective teaching gap” calculation, which measures average differences between low- and high-income students in access to effective teachers and can be extended to answer questions beyond the average gap in one year.

Data: We focus on the five-year period from 2008-09 to 2012-13, using data on teachers and students in 26 medium and large school districts. The districts are located in 15 states, distributed across all four Census regions, and operate in different geographic areas and under different conditions. The size and geographic diversity of our sample ensures that our results will not be influenced by idiosyncratic conditions in a single district or state and permits us to assess regional variation in access to effective teachers. We look at data on reading and math teachers in grades 4 to 8, students’ scores on statewide tests in grades 3 through 8, and student characteristics such as race and free or reduced-price school lunch status. Our data allow us to track teacher effectiveness from 4th to 8th grade in 12 districts. In the others, we track teacher effectiveness from 6th to 8th grade.

The students in our sample are more likely than average to live in cities and be low-income or Black or Hispanic. Some 69 percent live in large cities and 63 percent qualify for free- and reduced-price school lunch, compared to 46 percent and 53 percent of U.S. students nationwide, respectively. Forty-two percent of students are Hispanic and 29 percent are Black. On state assessments, the average student in our sample scores at the 45th percentile in English and the 46th percentile in math.

Student achievement gaps by family income mirror those at the national level. Among 8th-grade students, the typical low-income student performs at the 36th percentile on reading state achievement tests compared to the 63rd percentile for the typical high-income student, a gap of 0.68 standard deviations of student achievement. In math, the difference is 24 percentile points, or 0.63 standard deviations. In 4th grade, the student achievement gaps are slightly larger. In reading, the gap is 28 percentile points, or 0.72 standard deviations. In math, the gap is 29 percentile points, or 0.74 standard deviations.

Among the teachers in our sample, we find substantial variation in effectiveness and interaction with low-income students. The standard deviation of teacher effects is 0.13 in reading and 0.20 in math, on average. In other words, an average student with a teacher in the 90th percentile for effectiveness in reading could expect to score at the 57th percentile on an end-of-year state test. If that average student were assigned to a teacher in the 10th percentile for effectiveness in reading, the student could expect to score in the 43rd percentile. In math, this student might expect to score at the 60th percentile with a highly effective teacher compared to the 40th percentile with a minimally effective teacher.

Some 23 percent of teachers in our sample work in high-poverty schools where at least 90 percent of students qualify for free or reduced-price school lunch. Another 39 percent teach in schools where 60 percent to 90 percent of students qualify, and 38 percent teach in low-poverty schools where less than 60 percent of students qualify.

Method: Our effective teaching gap calculation starts by estimating individual teachers’ value added to student achievement as measured by statewide tests. We then link each student to the value-added estimate of the student’s teacher and find the average value added of teachers of low- and high-income students in each district. Finally, we subtract the average value added of teachers of low-income students from the average value added of teachers of high-income students.

Our analysis of teachers’ value added accounts for a range of student characteristics, including limited English proficiency, special education status, race, gender, and whether a student transferred across schools during the year. We also account for three types of potential peer effects: the average achievement of students in the classroom at the end of the prior school year, the amount of variation in student achievement within the teacher’s classroom, and the proportion of students in the classroom who are eligible for a free- or reduced-price lunch. We do this to account for the possibility that the characteristics of others in the classroom, such as their prior academic achievement, influences a student’s performance independent of the quality of the teacher.

We then calculate how the cumulative effect of the effective teaching gap translates into changes in the student achievement gap over multiple years. This takes into account the student’s incoming achievement level, contribution of family and other out-of-school factors, and the fact that the impact of an individual teacher’s effectiveness fades over time. We estimate the extent of this fade-out using estimates from the value-added model of how students’ test scores from the prior year are related to their test scores in the current year. We also estimate how student achievement gaps would change if low- and high-income students had equally effective teachers between Grades 4 and 8 (or between Grades 6 and 8, depending on what data are available).

Finally, we investigate the extent to which disproportionality in rates of placement with novice teachers could lead to greater inequity for low-income students, by documenting the proportion of teachers with less than three years of experience working at high-poverty schools, where at least 90 percent of students qualify for free and reduced-price school lunch. We compare that to the proportion of novice teachers at schools where less than 60 percent of students qualify for meal subsidies. We also examine the average difference in value added between novice and veteran teachers.

Similar Access to Effective Teachers for Low- and High-Income Students (Figure 1)

Results

Low-income students have less-effective teachers than high-income students, on average, but the differences are exceedingly small. The effective teaching gap is 0.005 standard deviations of student achievement in reading and 0.004 standard deviations in math. The average teacher of a low-income student is just below the 50th percentile of teacher effectiveness, while the average teacher of a high-income student is at the 51st percentile.

Black students also have teachers who are less effective than those who teach white students, on average, but only in math. The effective teaching gap in that subject is 0.01 standard deviations. We find no gap in teacher effectiveness in reading. In both subjects, there are no significant differences between teachers of Hispanic and white students, or between teachers of English learners and students who are not English learners.

Despite these broad similarities, pockets of inequity in access to effective teachers could exist within the study districts. To explore this possibility, we examine the likelihood that low- and high- income students are taught by teachers across the distribution of effectiveness. Here, we also find small or no differences (see Figure 1). In both subjects, 10 percent of low- and high-income students have one of the most effective teachers, on average. In looking at the least effective teachers, 10 percent of both low- and high-income students have such teachers in math. In reading, 10 percent of low-income students and 9 percent of high-income students have one of the least effective teachers.

We also investigate the effectiveness of the average teacher across schools with different poverty levels and find relatively small differences. We group schools into 10 categories based on their proportion of low-income students and calculate the average value added of their teachers. These range from 0.02 to −0.01 standard deviations across the school poverty categories for reading and from 0.03 to −0.02 standard deviations for math. In addition, there was no pattern of average value added decreasing as school poverty rates increased, although teachers in the lowest-poverty schools have the highest average value added, at 0.02 to 0.03 standard deviations.

Overall, our results indicate fairly equitable access to effective teachers. While the most effective teachers boost student achievement substantially relative to the least effective teachers, high-income students are not consistently taught by more effective teachers than low-income students. Instead, both low- and high-income students are taught by a mix of more effective and less effective teachers.

Access and the Achievement Gap

The absence of large effective teaching gaps in the districts we study implies that closing those gaps would have little effect on achievement outcomes. To demonstrate this, we first model the impact of all low-income students having teachers who are at least as effective as those of high-income students, from 4th through 8th grade. We find it would have relatively little effect.

The typical low-income 8th grader performs at the 35.4 percentile in reading while the typical high-income 8th grader is at the 60.5 percentile—a difference of 25.1 points. In math, the gap is 24.5 points. We estimate that if low-income students had teachers at least as effective as those of high-income students in grades 4-8, the student achievement gap would shrink to 24.2 points in reading and 22.3 points in math. If low-income students had teachers at least as effective as those of high-income students in grades 6-8, the student achievement gap would shrink by one percentile point or less in both subjects.

What if low-income students had more effective teachers than high-income students? To cut average income-based differences in achievement in half between 4th and 8th grade, districts would need to have an effective teaching gap of -0.102 standard deviations instead of 0.005. (A negative effective teaching gap means that low-income students have more effective teachers than high-income students.) To accomplish that, 30 percent of reading teachers would have to switch places with one another. In math, the effective teaching gap would need to be -0.080 standard deviations instead of 0.004, which would require that 11 percent of math teachers trade classroom assignments. These reductions in the achievement gap would only occur if the best teachers in classrooms with mostly high-income students were to systematically switch places with the worst teachers in classrooms with mostly low-income students.

Even though there is relatively little inequity in students’ access to effective teachers on average, there could be individual districts with greater inequity than others. We explore this possibility and find modest variation at the district level, with effective teaching gaps ranging from -0.024 to 0.023 standard deviations in reading and from -0.050 to 0.040 standard deviations in math. In other words, there are some districts where low-income students have less-effective teachers than high-income students, on average, and other districts where the opposite is true.

This raises the question of whether certain types of district characteristics are associated with greater inequity in access to effective teachers. We look at a variety of characteristics and find two that are significantly related to the effective teaching gap in both math and reading: district size and region. Districts that are larger and located in the southern United States tend to have a less equitable distribution of teachers compared to other districts. These findings are related, as districts in the South tend to be larger than those in other regions. Low-income students’ access to effective teachers is not consistently related to the other district characteristics we consider, such as the student achievement gap, the extent to which high- and low-income students are separated across schools, or the percentage of Black, Hispanic, and white students in the district. In reading, the effective teaching gap is significantly larger in districts with a greater percentage of low-income students and those with a greater percentage of minority students, but these relationships are not evident in math.

Novice Teachers

Across the study districts, 18.3 percent of teachers in high-poverty schools are novices compared with 8.9 percent of teachers in low-poverty schools. Novices are less effective than veteran teachers on average, with 0.022 lower average value added. However, we find that the presence of more novice teachers in high-poverty schools does not create substantial inequity, for two reasons.

First, although there are more low-income students in high-poverty schools than average, these schools still enroll a mix of low- and high-income students. The substantial difference between the prevalence of novice teachers in low- and high-poverty schools does not translate to a substantial difference between high- and low-income students in the probability of having a novice teacher.

When calculated at the student level, the difference between the likelihood of being taught by a novice teacher is modest, at four percentage points. Some 14 percent of low-income students and 10 percent of high-income students are taught by novices. In other words, 86 percent of low-income students and 90 percent of high-income students are taught by veteran teachers.

Second, the average difference in the effectiveness of novices and veteran teachers is also modest. Thus, even if all low-income students were taught by novices and all high-income students were taught by veteran teachers, the effective teaching gap would be 0.022 standard deviations. The actual difference in the proportion of students taught by a novice teacher is only 4 percentage points. Therefore, the component of the effective teaching gap resulting from low-income students being taught more frequently by novice teachers is approximately 4 percent of 0.022 standard deviations, or slightly less than 0.001.

Implications

Our results show that low-income and minority students have equal or nearly equal access to effective teachers in the great majority of the public school districts we analyze. While individual teachers differ substantially in their effectiveness, both high- and low-income students have a mix of the most effective and the least effective teachers. As a result, providing the two groups of students with equally effective teachers—even over a period of five years—would not substantially reduce the student achievement gap in most districts. Similarly, the disproportionate number of novice teachers at high-poverty schools contributes almost nothing to the effective teaching gap, and, by extension, to the student achievement gap.

The findings of our study—based on a cross-section of medium and large public school districts throughout the United States—suggest that a policy emphasis on correcting for an unequal distribution of “ineffective, out-of-field, or inexperienced teachers” (as required by the federal Every Student Succeeds Act of 2015) is misplaced. Value-added estimates identify effective and ineffective teachers in all types of schools. Student test-score data show that high- and low-income students are far apart in their achievement by the end of 3rd grade, and that this achievement gap grows little due to inequitable access to effective teachers.

It may not be reassuring that public schools are just holding the line on a set of unequal outcomes instead of decreasing them. However, public schools are financed and managed within a political system. Our simulation results suggest that it may be difficult to jolt this system and bring about a substantial decrease in achievement gaps through teacher mobility alone. This is not to concede that policymakers need to accept the status quo. But the best policy response likely resides outside the realm of teacher recruitment, school assignment, and retention. Although a well-planned and well-executed set of human capital policies can improve teacher effectiveness overall, that approach alone is not likely to diminish the student achievement gap.

Rather, our results might nudge policymakers to consider a broad spectrum of other cost-effective, evidence-based policies. For example, experimental evidence supports the expansion of tutoring. In addition, well-implemented early-learning programs may disrupt the predictability of student achievement gaps that are already apparent when children enter school. Other experimental evidence demonstrates that coaching teachers can boost students’ literacy levels in the early grades (see “Taking Teacher Coaching to Scale,” research, Fall 2018).

A half-century ago, James S. Coleman’s landmark “Equality of Educational Opportunity” report to Congress declared “differences between schools account for only a small fraction of differences in pupil achievement.” With more sophisticated methods, easier access to data, more computational power, and the ability to take the analysis from the school level to the teacher level, we have concluded much the same thing.

Eric Isenberg is senior study director at Westat. Jeffrey Max is principal researcher at Mathematica, where Philip Gleason is senior fellow and Jonah Deutsch is senior researcher. This article is based on the study “Do Low-Income Students Have Equal Access to Effective Teachers?” published in the June 2022 issue of Educational Evaluation and Policy Analysis.

This article appeared in the Fall 2022 issue of Education Next. Suggested citation format:

Isenberg, E., Max, J., Gleason, P., and Deutsch, J. (2022). Estimating the “Effective Teaching Gap” – Students experience unequal outcomes, but mostly equal access to high-quality instruction. Education Next, 22(4), 60-65.

The post Estimating the “Effective Teaching Gap” appeared first on Education Next.

]]>
49715689
A Half Century of Student Progress Nationwide https://www.educationnext.org/half-century-of-student-progress-nationwide-first-comprehensive-analysis-finds-gains-test-scores/ Tue, 09 Aug 2022 09:00:44 +0000 https://www.educationnext.org/?p=49715526 First comprehensive analysis finds broad gains in test scores, with larger gains for students of color than white students

The post A Half Century of Student Progress Nationwide appeared first on Education Next.

]]>

Illustration

Has the achievement of U.S. students improved over the past half century? Have gaps between racial, ethnic, and socioeconomic groups widened or narrowed?

These and similar questions provoke near-constant conversation. But answers are uncertain, partly because research to date has yielded inconsistent findings. Here we bring together information from every nationally representative testing program consistently administered in the United States over the past 50 years to document trends in student achievement from 1971 to 2017, the last year for which detailed information is currently available.

Contrary to what you may have heard, average student achievement has been increasing for half a century. Across 7 million tests taken by U.S. students born between 1954 and 2007, math scores have grown by 95 percent of a standard deviation, or nearly four years’ worth of learning. Reading scores have grown by 20 percent of a standard deviation during that time, nearly one year’s worth of learning.

When we examine differences by student race, ethnicity, and socioeconomic status, longstanding assumptions about educational inequality start to falter. Black, Hispanic, and Asian students are improving far more quickly than their white classmates in elementary, middle, and high school. In elementary school, for example, reading scores for white students have grown by 9 percent of a standard deviation each decade, compared to 28 percent for Asian students, 19 percent for Black students, and 13 percent for Hispanic students. Students from low socioeconomic backgrounds also are progressing more quickly than their more advantaged peers in elementary and middle school. And for the most part, growth rates have remained steady throughout the past five decades.

Conventional wisdom downplays student progress and laments increasing achievement gaps between the have and have-nots. But as of 2017, steady growth was evident in reading and especially in math. While the seismic disruptions to young people’s development and education due to the Covid-19 pandemic have placed schools and communities in distress, the successes of the past may give educators confidence that today’s challenges can be overcome.

Bypassing Conventional Wisdom

Scholars and public intellectuals from all sides of the political spectrum have consistently made the opposite case. Dating back to 1983’s A Nation at Risk, debate over the state of public education in the United States often has portrayed schools as failing and American students as falling behind. Books like 2009’s The Dumbest Generation and 1994’s The Decline of Intelligence in America argued that young people were so entranced by technology that they failed to develop basic knowledge and skills.

Public understanding of inequality also has assumed that racial, ethnic, and socioeconomic gaps in student achievement are universal and growing. In 2011, research by Stanford sociologist Sean Reardon appeared to show a widening of the socioeconomic achievement gap over the past 70 years. In 2012, conservative Charles Murray argued that “the United States is stuck with a . . . growing lower class that is able to care for itself only sporadically and inconsistently” even as the “new upper class has continued to prosper as the dollar value of [its] talents . . . has continued to grow.” In 2015, Harvard political scientist Robert Putnam wrote “rich Americans and poor Americans are living, learning, and raising children in increasingly separate and unequal worlds.” More recently, critiques by organizations like Black Lives Matter have identified racial inequality both inside and outside the classroom as a defining characteristic of American life.

But no study of student achievement over time has brought all the relevant data together in a systematic manner and assessed how these assumed trends are playing out. Our analysis does just that.

Our data consist of more than 7 million student test scores on 160 intertemporally linked math and reading tests administered to nationally representative samples of U.S. student cohorts born between 1954 and 2007 (see “Put to the Test“). By “intertemporally linked,” we mean that researchers in each of the testing programs have designed their tests to be comparable over time, by doing things such as repeating some of the same questions across different waves.

We estimate trends separately by testing program, subject, and grade level and report the median rather than average result to avoid giving undue importance to outliers, much as consensus projections of future economic growth typically use the median of predictions made by alternative economic models. We report changes in student achievement over time in standard deviation units. This statistic is best understood by noting that average performance differences between 4th- and 8th-grade students on the same test are roughly one standard deviation. Accordingly, we interpret a difference of 25 percent of a standard deviation as equivalent to one year of learning.

Clear Progress for U.S. Students Over 50 Years of Testing (Figure 1)

Achievement and the Flynn Effect

The surveys show a much steeper rise in math than reading performance (see Figure 1). In math, overall student performance rose by 19 percent of a standard deviation per decade, or 95 percent of a standard deviation over the course of 50 years—nearly four additional years’ worth of learning. In reading, however, the gains are only 4 percent of a standard deviation per decade, or 20 percent of a standard deviation over the same period.

The difference between the two subjects is puzzling. Mathematical knowledge and reasoning skills in the U.S. teaching force has long been a matter of concern. And mainstream math instruction in U.S. schools generally is considered inadequate relative to other developed countries, despite recent attempts to focus on developing mathematical understanding. Why is math achievement accelerating far more quickly than reading?

The answer, we believe, is found in recent research on human intelligence. Not long ago, intelligence quotient, or IQ, was considered a genetically determined constant that shifted only over the course of eons, as more intellectually and physically fit homo sapiens survived and procreated at higher rates. Then in the mid-1980s, James Flynn, a New Zealand political scientist, examined raw IQ data and found that scores were increasing by 3 points, or about 21 percent of a standard deviation, per decade. Though Flynn’s work was initially dismissed as an over-interpretation of limited information, his finding was replicated by many others.

In 2015, Jakob Pietschnig and Martin Voracek conducted a meta-analysis of 271 studies of IQ, involving 4 million people in 31 countries around the world over the course of more than a century. As Flynn did, they found growth in overall IQ scores. But they also distinguished between types of intelligence. This included crystallized knowledge, or the ability to synthesize and interpret observed relationships in the environment, which is rooted in facts, knowledge, and skills that can be recalled as needed. And it included fluid reasoning, or the ability to analyze abstract relationships, which is associated with recognizing patterns and applying logic to novel situations. In industrialized societies, for a period similar to the one covered by our study, they found that fluid reasoning grew by 15 percent of a standard deviation per decade compared to 3 percent for crystallized knowledge. This difference resembles what we observe in the achievement data: growth of 19 percent of a standard deviation per decade for math and 4 percent for reading.

That the growth rates for the two types of achievement and IQ parallel one another may be more than a coincidence. Reading draws heavily on crystallized knowledge of the observable world, and skillful readers can give meaning to words that denote features of their physical and social environment. In math, this type of knowledge is necessary to understand symbols such as 1, 2, and 3 or +, -, and =, but analyzing and manipulating relationships among symbols is more a function of fluid reasoning. Several studies have shown math performance to be more strongly associated than reading performance with higher levels of fluid reasoning. In addition, a longitudinal study of preschool children found emergent school vocabulary to be associated with gains in verbal intelligence, a form of crystallized knowledge, but not with gains in fluid reasoning.

In the meta-analysis, Pietschnig and Voracek point to the factors that affect brain development as the most likely explanation for differential growth in these types of intelligence. Studies in neurobiology and brain imaging have found that when environmental factors like nutrition, infections, air pollution, or lead poisoning damage the brain’s prefrontal cortex, it affects fluid reasoning, but not crystallized knowledge. The negative impact on brain development of, for example, growing up amid famine or war would appear to have the biggest impact on fluid reasoning intelligence, used for math, rather than crystalized knowledge, used for reading.

Over the past 100 years, mothers and babies from all social backgrounds across the world have enjoyed increasingly higher quality nutrition and less exposure to contagious diseases and other environmental risks. Pietschnig and Voracek find substantial growth in fluid reasoning and less growth in crystallized knowledge on every continent, with particularly large gains in Asia and Africa. If students’ performance on math tests depends more on fluid reasoning than crystallized knowledge, then the greater progress in math than reading may be due to environmental conditions when the brain is most malleable—in early childhood, or even before students are born.

 

Put to the Test

Our data come from approximately 7 million U.S. student observations, as well as 4.5 million international student observations, on math and reading assessments in five psychometrically linked surveys administered by governmental agencies. The surveys have administered 160 waves of 17 temporally linked tests of achievement to nationally representative cohorts of U.S. students for various portions of the past half century.

Together, these data provide information on student race and ethnicity, gender, and socioeconomic status (an index based upon student reports of parents’ education and the number of possessions in the home). Within each subject, age/grade, and assessment, we normalize each subsequent cohort’s test score distribution with respect to the mean of test scores in its initial year of administration. With a quadratic fit, we calculate the distance in standard deviations of the change in student performance for survey per decade.

1971-2012
National Assessment of Educational Progress, Long-Term Trend (LTT) Assessment
● Math and Reading – ages 9, 13, 17

1990-2017
National Assessment of Educational Progress, main NAEP
● Math and Reading – grades 4, 8, 12

1995-2015
Trends in International Math and Science Study (TIMSS)
● Math – grades 4, 8

2000-2015
Program for International Student Assessment (PISA)
● Math and Reading – age 15

2001-2016
Progress in International Reading Literacy Study (PIRLS)
● Reading – grade 4

 

 

The PISA Exception

The main exception to this pattern comes from the Program for International Student Assessment (PISA) given since 2000 to high-school students at age 15. On this test, and only on this test, both the overall trend and the math-reading comparison are the reverse of what we observe on all the other surveys. U.S. student performance declines over time, with steeper drops in math scores than in reading. In math, scores decline by 10 percent of a standard deviation per decade; in reading, they fall by 2 percent of a standard deviation per decade. This stands in sharp contradiction to student performance on the National Assessment of Educational Progress (NAEP). There, we see large gains of 27 percent of a standard deviation per decade in math among middle-school students, who take the test in 8th grade. In addition, student performance improves by 19 percent of a standard deviation per decade on another math exam, the Trends in International Math and Science Study (TIMSS). How can PISA obtain results so dramatically different from what other tests show? Is the PISA exam fundamentally flawed? Or is it measuring something different?

We cannot account for all differences among tests, but in our opinion, PISA math is as much a reading test as a math test. The goal of PISA is to measure a person’s preparation for life at age 15. It does not ask test-takers to merely solve mathematical problems, as do NAEP and TIMSS, but instead provides opportunities to apply mathematical skills to real-world situations. A 2018 analysis found that “more than two-thirds of the PISA mathematics items are independent of both mathematical results (theorems) and formulas.” A 2001 review found that 97 percent of PISA math items deal with real-life situations compared to only 48 percent of items in NAEP and 44 percent in TIMSS. Another analysis comparing the exams found that PISA questions often have more text, including extraneous information students should ignore, than NAEP questions. In addition, a 2009 study found “there is a very high correlation between PISA mathematics and PISA reading scores” and that “The overlap between document reading (e.g., graphs, charts, and tables) and data interpretation in mathematics becomes blurred.”

We do not pretend to know which testing program is administering the best exam. But we are quite certain that PISA is administering a decidedly different kind of math test, one that requires much more crystallized knowledge than other math tests.

Growth Over Time for Students of All Racial and Ethnic Groups (Figure 2)

Results by Social Group

Every test in our study shows a forward stride toward equality in student performance across race, ethnicity, and socioeconomic lines over the past half century (see Figure 2). The median rate of progress made by the average Black student exceeds that of the average white student by about 10 percent of a standard deviation per decade in both reading and math. Over 50 years, that amounts to about two years’ worth of learning, or about half the original learning gap between white and Black students. The disproportionate gains are largest for students in elementary school. They persist in middle school and, in diminished form, through the end of high school.

We don’t think this is due to outsized improvements in nutrition and medical care for Black children, because the gains are as great in reading as in math. It could be due to educationally beneficial changes in family income, parental education, and family size within the Black community. Other factors may also be at play, such as school desegregation, civil rights laws, early interventions like Head Start and other preschool programs, and compensatory education for low-income students. Regardless, the equity story is clearly positive, if still incomplete.

Hispanic student performance in math is similar: a steeper upward trend as compared to white students. However, gains in reading by Hispanic students, though still greater than the progress made by white students, are less pronounced than the math gains. This may be due to language barriers; about 78 percent of English language learners in the U.S. are Hispanic.

Overall, Asian students are making the most rapid gains in both subjects. Asian students have advanced by nearly two more years’ worth of learning in math and three more years’ worth of learning in reading than white students.

We also compare trends by socioeconomic status by building an index based on student reports of parents’ education as well as the number of possessions in the home. We compare achievement made by students coming from households in the top 25 percent and lowest 25 percent of the socioeconomic distribution. For all students, the achievement gap based on socioeconomic status closes by 3 percent of a standard deviation per decade in both reading and math.

The biggest gains occur in elementary school, where the gap closes over the 50-year period by 1.5 years’ worth of learning in math and three years’ worth in reading (see Figure 3). The differences shrink in middle school and are reversed in high school, where rates of progress by students in the top 25 percent modestly exceed those of students with the lowest socioeconomic status. The increase in the gap among the oldest students is 3 percent of a standard deviation per decade in math and 4 percent in reading.

In looking at low- and high-socioeconomic students within racial and ethnic groups, we see similar patterns for Black students in both subjects and for Hispanic students in math: achievement differences by socioeconomic background closing when students are tested at a younger age, but widening when students are tested toward the end of high school. Among Asian students, low-socioeconomic students continue to make greater progress than high-socioeconomic students in both subjects at all age levels.

What about income-based gaps in student achievement? In a widely circulated 2011 study, Stanford sociologist Sean Reardon found the income-achievement gap had increased dramatically over the past half century and more. However, the data upon which this claim rests are fragile, in that he relies for his conclusion upon results from disparate tests that are not linked and therefore are not necessarily comparable. To see whether trends from linked surveys support Reardon’s findings, we explore trends in achievement by the number and type of possessions students report as being in their homes, a plausible indicator of family income.

Overall, the evidence points in a direction opposite to Reardon’s findings, and results are qualitatively similar to the ones observed when estimated by the socioeconomic index. We find disproportionately larger gains for students in the lowest income quartile in both math and reading at younger ages. The difference is 5 percent of a standard deviation per decade in math and 6 percent in reading. However, we find that among students tested at the end of high school, the students from the highest quartile of the income distribution make greater progress than those from the lowest quartile by 6 percent of a standard deviation in math and 9 percent of a standard deviation in reading.

In sum, inferences about whether the size of the income gap, or the socioeconomic gap more generally, has increased or decreased depend largely on whether one places greater weight on tests administered to students in earlier grades or on trends for students tested as they reach the end of high school. For some, the high-school trend is most relevant, as it measures performance as students are finishing their schooling. For others, it is the least informative trend, as it could be subject to error if some older students are taking standardized tests less seriously in recent years or if rising graduation rates have broadened the pool of older students participating in the test.

But it is worth mentioning again that PISA stands out as an exception. It is the only test that shows much larger gains for U.S. high-school students from families in the lowest socioeconomic quartile than for those in the highest one. In math, the performance of the most advantaged 15-year-old students slid each decade by no less than 20 percent of a standard deviation in math and 14 percent in reading. Meanwhile, students in the bottom quartile showed notable gains of 4 percent of a standard deviation in math and 15 percent in reading. That amounts to closing the socioeconomic achievement gap by a full year’s worth of learning each passing decade. If PISA is to be believed, we are well on the way to equality of achievement outcomes.

Larger Gains for Disadvantaged Students in Elementary School, but Differences Decline and Are Reversed as Students Age (Figure 3)

Recent History

Critical assessments of America’s schools have a long history. But criticism grew sharper after the passage of the federal No Child Left Behind Act of 2001, which required annual testing and score reporting and set deadlines for improvement. In the past two decades, public opinion has been split widely between those who say the law enhanced student achievement and those who claim it made matters worse.

We split the sample into students born before and after 1990 to determine whether gains in median test scores were greater or lesser after the law was passed. Reading scores grew by 8 percent of a standard deviation more per decade among students born between 1991 and 2007 compared to students born between 1954 and 1990. In math, scores of more recent test-takers grew by 8 percent of a standard deviation per decade less than their predecessors.

Why would progress in math have slowed when progress in reading speeded up? The first half of the question is more easily explained than the second half. Trends in math achievement, as we have seen, are sensitive to changes in fluid reasoning ability. Factors that drive broad growth of that type of intelligence, such as better nutrition and decreased vulnerability to environmental contaminants, may have been changing more rapidly 30, 40, and 50 years ago compared to the past two decades. But why, then, have reading scores climbed more quickly? Did schools operating under No Child Left Behind have a more positive impact on reading performances? Or are families more capable of helping their children to read? Or both? Our data cannot say.

Recently, school closings in response to the Covid-19 pandemic seem to have had a negative impact on learning for an entire generation of students and exacerbated achievement gaps. This recalls similar educational setbacks from school closures during wars and strikes, reduced instructional time due to budget cuts (see “The Shrinking School Week,” research, Summer 2021), and broad absenteeism during weather events (see “In Defense of Snow Days,” research, Summer 2015). Indeed, Pietschnig and Voracek detect a slowdown in intellectual growth during World War II, a likely byproduct of both school closures and worldwide disruptions of economic and social progress.

But on the whole, families and schools both appear to have played a key role in reducing achievement gaps by race, ethnicity, and socioeconomic status over time. They also may have facilitated more rapid gains in reading among students born after 1990. Parental educational attainment and family incomes, both of which are strong correlates of student achievement, have risen in this more recent period. In addition, school reforms—desegregation, accountability measures, more equitable financing, improved services for students learning English, and school choice—have had their greatest impact on more recent cohorts of students.

Still, a research focus on families and schools may distract attention away from broader social forces that could be at least as important. For example, diminished progress in math for those born later than 1990 could be due to a decline in returns from improved health and nutrition in advanced industrialized societies. In addition, the greater gains of students at an early age and the recent flattening of growth in math performance all suggest that broader social, economic, and physical environments are no less important than schools and families. It is reasonable to infer from our research that policies benefiting children from the very beginning of life could have as much impact on academic achievement, especially in math, as focused interventions attempted when students are older.

Paul E. Peterson is a professor and director of the Program on Education Policy and Governance at Harvard University and a senior fellow at the Hoover Institution, Stanford University.

M. Danish Shakeel is a professor and director of the E. G. West Centre for Education Policy at University of Buckingham, U. K. This essay is drawn from an article just released by Educational Psychology Review.

This article appeared in the Fall 2022 issue of Education Next. Suggested citation format:

Shakeel, M.D., and Peterson, P.E. (2022). A Half Century of Student Progress Nationwide: First comprehensive analysis finds broad gains in test scores, with larger gains for students of color than white students. Education Next, 22(4), 50-58.

For more, please see “The Top 20 Education Next Articles of 2022.”

The post A Half Century of Student Progress Nationwide appeared first on Education Next.

]]>
49715526
The Costs of Canceling Darwin https://www.educationnext.org/costs-of-canceling-darwin-fewer-scientists-more-skepticism-science-states-limit-evolution-instruction/ Tue, 05 Apr 2022 09:00:34 +0000 https://www.educationnext.org/?p=49715177 Fewer scientists, more skepticism of science in states that limit evolution instruction

The post The Costs of Canceling Darwin appeared first on Education Next.

]]>

Illustration

The influence that attitudes about science can have on public health and the economy has been on broad display during the Covid-19 pandemic. Despite consensus in the scientific and medical communities that Covid-19 vaccines are safe and protective against a virus that has killed more than 900,000 Americans, 15 percent of U.S. adults remain unvaccinated. Most report that they don’t trust the vaccines or are worried about side effects. Complicating matters, attitudes about science are closely tracking deeply polarized partisan affiliations. A recent poll found that 34 percent of Republicans reported having a “great deal” of confidence in science compared to 64 percent of Democrats.

What contributes to this skepticism? Virtually every U.S. high-school student is required to study biology, at minimum, to earn a diploma. But the exact content of the course varies from state to state. I investigate the role of state standards for high-school science content in shaping knowledge and attitudes about science—specifically, how inclusion of lessons on evolution theory influences students’ knowledge about evolution at the end of schooling, attitudes on evolution in adulthood, as well as the probability that they work in life sciences.

I focus on evolution theory because it is both foundational to modern science and controversial among Americans. Some 160 years after Charles Darwin’s research detailing the theory of natural selection was first published, 98 percent of scientists compared to 65 percent of U.S. adults believe that humans have evolved, according to the American Association for the Advancement of Science and a 2015 poll by the Pew Research Center. In general, Republicans and evangelical Christians are less likely to agree with the idea that humans evolved over time through natural processes alone, while Democrats and independents are more likely to express agreement with that theory.

There is substantial variation across U.S. states in how evolution is covered in education standards—and the nature of this variation has changed over time. I look at the period of 2000 to 2009. During that timespan, 22 states expanded the coverage of evolution in their education standards and 15 states reduced it. I use these changes to estimate the causal impact of standards on three outcomes: whether students understand evolution theory at the end of high school, agree with evolution theory as adults, and pursue careers in the life sciences.

I find that state science standards affect all three of these outcomes. In states that require more comprehensive evolution instruction, students are more likely to answer knowledge questions on evolution correctly by the end of high school on the National Assessment of Educational Progress. In adulthood, being from a state that requires comprehensive evolution instruction as opposed to no evolution instruction increases evolution approval by 33 percentage points—a 57 percent jump. It also boosts the probability of working in life sciences by 0.04 percentage points, or 23 percent.

My analysis shows that what states require in their educational standards has long-lasting effects on individual attitudes and occupational choices—which, even outside of the challenges of managing a pandemic, can foster innovation, opportunity, and economic growth. When state education leaders require comprehensive instruction in evolution theory in high school, they are helping grow the science workforce of the future.

Dayton, Tennessee, teacher John T. Scopes volunteered to be tried for teaching evolution in 1925.
Dayton, Tennessee, teacher John T. Scopes volunteered to be tried for teaching evolution in 1925.

A Long Battle Over Evolution Education

Whether U.S. public schools should teach evolution has been a contested issue for at least a century. The scientific community reached consensus on the validity of evolution relatively soon after Darwin published On the Origin of Species in 1859, but just one quarter of the biology textbooks published between 1900 and 1919 contained any information about evolution. By the 1920s, about one third of biology textbooks covered human evolution. But the decade also marked the start of a series of legal disputes, with at least 20 states considering bills to ban evolution lessons from public schools.

In Tennessee, the Butler Act banned evolution instruction. That law resulted in the Scopes Trial of 1925. John T. Scopes, a 24-year-old high-school teacher, volunteered to admit he used a textbook that included evolution while covering a biology class in order to be charged with a misdemeanor under the law. After a high-profile trial in which Scopes was represented by the American Civil Liberties Union, he was convicted of violating the Butler Act. While the state supreme court overturned that conviction on a technicality, the justices upheld the Butler Act, and Mississippi and Arkansas soon passed similar laws. These laws remained on the books until 1967, when Tennessee lawmakers repealed the Butler Act, and 1968, when the U.S. Supreme Court’s ruling in Epperson v. Arkansas made such laws unconstitutional on First Amendment grounds.

Efforts to limit evolution teaching persisted in the years following Epperson, with advocates lobbying states to pass bills that required teaching creation theory alongside evolution. Arkansas passed such a “balanced treatment” law in 1981, which was overturned by the U.S. Supreme Court the following year in McLean v. Arkansas Board of Education. In 2007, a nationally representative survey of high-school biology teachers by researchers at Penn State found that just 51 percent said they presented evolution as scientific consensus. That percentage climbed to 67 percent in 2019, though organized efforts to influence how schools cover evolution persist.

“Academic freedom” bills, proposed in states like Florida, Oklahoma, and Arizona, would empower parents to challenge what is taught in school and require teachers to present an array of theories about the origin of life, climate change, and other issues. And in Texas, Governor Greg Abbott’s proposed amendment to the state constitution would establish a “Parents’ Bill of Rights,” including the right to review all curriculum and books teachers plan to use in the classroom. While the current conversation about such proposals is more explicitly focused on how race, gender, and climate change are taught, these bills could have broad implications for the teaching of evolution as well.

Anti-evolution books were sold in Dayton during what became known as the “Scopes Monkey Trial.”
Anti-evolution books were sold in Dayton during what became known as the “Scopes Monkey Trial.”

Shifts in State Science Standards

In the distant past, curriculum in American public schools was determined largely at the local level. However, concerns about declining achievement among U.S. students in the 1960s and 1970s prompted calls for states to establish rigorous and comparable education standards. In the 1990s, the American Association for the Advancement of Science and National Research Council published guidelines for science standards, which define the scientific knowledge and skills students are supposed to master in each grade in public schools.

State standards are just one of many factors that shape academic content. For example, local school curricula, the selection of textbooks, the knowledge, ability, and ideology of teachers, and testing requirements also influence what students learn in school. However, standards influence how local curricula and lesson plans are written and what content is tested in statewide exams.

Standards are typically drafted by advisory committees, which can consist of a panel of teachers and other stakeholders including, at least occasionally, scientists. The standards then must be considered by a state’s board of education, whose members are either appointed by the governor, elected, or some combination of the two. A board typically holds public hearings and reviews written testimony from parents, scientists, and representatives from religious groups, among others. Then, after this period of public comment, the board votes to approve or reject the proposed standards.

Clarence Darrow was the lawyer for the defense during the proceedings.
Clarence Darrow was the lawyer for the defense during the proceedings.

The political process described above implies that changes in standards are not random events. Rather, these changes reflect changing political views, either expressed by the election of a governor who subsequently appoints members of the state board or by the direct election of its members. But the exact point in time that a state’s standards change could be regarded as virtually random. If public approval of teaching evolution or of science in general changes in a given year, that will not necessarily result in a reform of state science standards within a certain amount of time. State by state, board members arrive at their posts through different avenues and serve for different amounts of time. Gubernatorial and legislative election schedules are also not standard from state to state. Further, a state could be influenced by standards changes in other states, due to their influence on public opinion, textbook content, or political will.

One thing that is constant across states is that changes to standards involving evolution typically follow heated negotiations. For example, after years of debate and drafting, in February 2008 the Florida Board of Education voted 4-to-3 in favor of new science standards that included a comprehensive discussion of evolution, replacing standards that did not mention the word “evolution” and included minimal discussion of evolutionary processes. Meanwhile, in 2009 the Texas Board of Education minimized evolution in its standards. A former chairman of the board, who said he did not personally believe in Darwin’s evolution theory, had pushed to limit evolution instruction in various ways. One proposal for state science standards would have required teachers to cover the “strengths and weaknesses” of evolution, while another would have required students to study the “sufficiency or insufficiency” of key principles of evolution. Ultimately, a version of science standards that included evolution without such qualifications but left out statements like “the estimated age of the universe was 14 billion years” was approved.

Notably, the reforms in Florida and Texas did not follow a partisan change—every governor in Florida and Texas in the 21st century was elected as a Republican. Both examples shed light on the political process behind such reforms and show that they are not simply a consequence of a change of government. Because these changes happen at some point in time for largely idiosyncratic reasons, and not because of a specific event like a shift in political power, they are useful for studying how standards affect student outcomes.

Charles Darwin, an English scientist, first put forth the theory of evolution in 1859.
Charles Darwin, an English scientist, first put forth the theory of evolution in 1859.

Data and Method

First, to measure the coverage of evolution in a state’s science standards, I make use of states’ “evolution scores” as calculated by Lawrence Lerner and Louise S. Mead and Anton Mates, who undertook detailed reviews of state science standards in 2000 and 2009. Those analyses also look at whether (and when) a state’s standards were updated between those years.

The evolution score is a composite index based on whether the word “evolution” appears in a state’s science standards; the respective coverages of biological, human, geological, and cosmological evolution; and the connections drawn between different aspects of evolution. In addition, the index takes into account the absence of creationist jargon and creationist disclaimers in approved textbooks, where applicable. Evolution scores range from 0 to 1, with 0 indicating no mention or a non-scientific creationist view of evolution and a score of 1 indicating comprehensive coverage of evolution. According to Lerner, scores between 0.60 and 0.79 are “satisfactory.” There is wide variation in how standards emphasize alleged weaknesses and critiques of evolution theory.

I compare state scores in 2000 with those from 2009 and find broad changes. Some 22 states earned higher evolution scores because their standards grew more comprehensive, while 15 states earned lower evolution scores because their standards contracted (see Figure 1). Kansas, Mississippi, and Florida had the largest increases in evolution scores, while the biggest decreases were in Connecticut, Louisiana, and Texas.

Changes in How State Science Standards Cover Evolution (Figure 1)

Second, to estimate the effect of students’ exposure to the teaching of evolution in high school on their knowledge about evolution by the end of high school, I link states’ evolution scores to individual scores on the National Assessment of Educational Progress 12th-grade science test, which includes questions about evolution. I consider scores for students based on their assumed date of high-school entry, three years before they take the test. The high-school entry year is the relevant year, as most teaching of evolution takes place at the beginning of high school. The average evolution score is 0.65, implying that students in the sample were on average exposed to “satisfactory” evolution coverage. Students on average answer just 32 percent of the evolution questions correctly, underscoring the questions’ difficulty.

Third, I link states’ evolution scores to individual results on the General Social Survey conducted by NORC at the University of Chicago. The survey monitors societal change by interviewing nationally representative samples of adults and has included questions about evolution attitudes since 2006. The main outcome variable is based on the question “Human beings, as we know them today, developed from earlier species of animals. Is that true or false?” The data include respondents’ birth years and state of residence at age 16, which allows me to estimate the effect of students’ exposure to the teaching of evolution in high school on their approval of evolution in adulthood.

Finally, I link state evolution scores with occupational fields of adults as reported in the American Community Survey by the U.S. Census Bureau. These data also include year and state of birth; I assume that students enter high school in this state at age 14 and assign the evolution score for this state-year combination accordingly. Since evolution is the fundamental theory of life sciences, I focus primarily on whether adults work in the life sciences.

To study the effects of state standards on these outcomes, I compare students who attended high school in the same state before and after standards changed. This approach addresses the concern that states with more comprehensive standards differ in other ways that matter for student’s knowledge about evolution. I also adjust for differences in a range of individual characteristics that could be associated with the outcomes, such as gender, race, parental education, and the religion in which the student was raised. (The specific set of control variables differs across outcomes based on the available data.) Finally, I take into account the year each student entered high school and the year the outcome data was gathered, in order to capture changes in these outcomes over time across the nation as whole.

State Science Standards Influence Evolution Knowledge in Grade 12 (Figure 2)

Results

In states where science standards call for more comprehensive coverage of evolution in high-school science classes, students accurately answer more questions about evolution on the 12th-grade NAEP science test (see Figure 2). Being from a state with an evolution score of 1, compared to an evolution score of 0, increases the average share of questions on evolution answered correctly by 5.8 percentage points, an 18 percent increase over the average of 32 percent correct. Looking across student subgroups, I find a large difference in effects for females, at 10.1 percentage points, compared to effects for males, at 1.0 percentage points. The largest effects are for students who do not have a computer at home, at 12.7 percentage points. The effects are 8.8 percentage points for both students who are Black and students who do not have a computer at home.

Interestingly, there are no effects on knowledge in areas of science unrelated to evolution. This finding implies that evolution instruction does not spill over to other science areas. It also suggests that the changes in state standards on evolution I use to study the standards’ effects are not associated with changes that matter for students’ broader scientific knowledge.

Scientific content in high school also shapes students’ attitudes in adulthood. Individuals who attended high school in a state with an evolution score of 1, as opposed to an evolution score of 0, are 33.3 percentage points more likely to approve of evolution in adulthood, a 57 percent increase (see Figure 3).

Impact of State Science Standards on Evolution Attitudes in Adulthood (Figure 3)

To put this overall effect in context, I look at the impact of state science standards on adult attitudes for groups of students defined based on race, community type, and religious upbringing. The effect is particularly large for Black adults, at 70.6 percentage points, and for adults who were raised in urban communities. Among religious groups, the effect is largest for individuals raised as Mainline Protestants. They are 44.6 percentage points more likely to agree that human evolution is true if they attended high school in a state with an evolution score of 1 as opposed to an evolution score of 0. The impact is 17.6 percentage points for both adults raised Evangelical Christian or Catholic. However, baseline attitudes about evolution among those two groups vary widely: compared to adults raised in non-religious households, adults who were raised as Evangelical Christians are 29 percentage points less likely to agree that human evolution is true, while Catholics are just 1.9 percentage points less likely to agree, conditional on other factors such as gender, race, and parental education.

At the same time, non-evolution scientific, religious, and political attitudes are not affected by evolution instruction. This again shows that the changes in state evolution standards do not affect outcomes in these domains and that the specific timing of the reforms I study is unrelated to general scientific, religious, and political shocks.

Finally, I look at whether state science standards influence whether students pursue careers in the sciences as adults. Being exposed to a comprehensive teaching of evolution as opposed to no evolution teaching when entering high school increases by 0.04 percentage points the probability of working in life sciences in adulthood, which includes typically high-paid occupations in medicine, biology, chemistry, and agriculture. This represents a 23 percent increase, as only 0.15 percent of all adults work in one of these fields. The effects are strongest on careers in biology, for which evolution theory is the fundamental scientific explanation of the origin of life.

At the same time, I find no effect on whether students go on to work in scientific occupations outside the natural sciences, such as jobs in the social sciences or as science technicians, and no effect on whether they pursue work in non-scientific occupations.

Standards Matter

Public debate over academic standards often focuses on the gap between standards and curriculum and emphasizes the critical role that textbook developers, chief academic officers, and teachers play in determining what content is presented to students. The notion that education standards have no meaningful impact on students is a common view. However, legal pressures on school districts, the reflection of the content of education standards in textbooks, and the gradual expansion of standardized testing covering the content of education standards have arguably incentivized teachers to follow standards. My analysis demonstrates that these standards indeed affect what students learn.

More broadly, I find that the content of school curricula and instruction lastingly shapes students—even when it comes to a politically contentious topic like evolution. In particular, when state science standards require that schools teach evolution theory in detail, students know more about evolution at the end of high school. As adults, they are more likely to agree that it is an accurate description of the origins of the human species and to pursue careers in life sciences.

I see broad implications for these findings. For example, state standards that call for more comprehensive instruction about climate change could have analogous effects on student knowledge and adult attitudes and career decisions. Similarly, standards that include detailed coverage of the history and mechanics behind vaccinations could influence knowledge, attitudes, and life choices.

These findings are of particular interest as the U.S. enters the third year of a deadly pandemic. The potential of state standards to enhance public trust in science is a worthy topic of future research.

Benjamin W. Arold is PhD Candidate in economics at LMU Munich and a junior economist at the ifo Center for the Economics of Education at the CESifo Group in Munich, Germany.

This article appeared in the Summer 2022 issue of Education Next. Suggested citation format:

Arold, B.W. (2022). The Costs of Canceling Darwin: Fewer scientists, more skepticism of science in states that limit evolution instruction. Education Next, 22(3), 56-63.

The post The Costs of Canceling Darwin appeared first on Education Next.

]]>
49715177
Major Differences https://www.educationnext.org/major-differencecs-why-some-degrees-cost-colleges-more-than-others/ Tue, 22 Feb 2022 10:01:12 +0000 https://www.educationnext.org/?p=49714612 Why some degrees cost colleges more than others

The post Major Differences appeared first on Education Next.

]]>

IllustrationHow expensive is a college degree? Usually, the answer is based on what students pay in tuition and fees compared to what they earn after graduation. As a result, policymakers often promote enrollment and applaud growth in tech-heavy programs that tend to produce high-earning graduates, like engineering and computer science—especially given the explosive growth in college prices, which have doubled in the past 30 years.

But that thinking only focuses on one side of the equation: the student’s private return on investment, based on the labor-market value of a degree relative to what the student paid. We know very little about the economic cost of running an electrical engineering program compared to, say, a history department, or the resource consequences of steering more students into these fields.

To fill this gap, we examine department-level data on expenditures, outputs, and factors of production for undergraduate, graduate, and professional degree programs at nearly 600 four-year institutions across the United States from 2000 to 2017. Our analysis compares the instructional costs per student credit hour at more than 8,000 departments in 20 disciplines, including both in-person and online study.

We establish five new facts about college costs. First, we find substantial cost differences across fields of study. On the whole, costs are higher in fields where graduates earn more and in pre-professional programs. Second, most of these patterns can be explained by differences in class size and, to a lesser extent, differences in average faculty pay. However, some fields with highly paid faculty, like economics, offset this with large classes.

Third, cost differences have evolved over time. Programs in some fields, such as mechanical engineering, chemistry, physics, and nursing, have shown steep annual declines in spending, while others like fine and studio arts, history, and political science have grown more expensive each year. Fourth, these trends are explained in part by a growing number of adjunct faculty as well as changes in class size and faculty teaching loads. Fifth, online instruction is not a cost-saver. It is neither more nor less expensive than in-person classes.

Our results underscore the potential wedge between the social and private returns to higher education. That is, the social return to investment in high-earning fields may be lower than wage premiums suggest, because high-return fields also tend to be more costly to teach. This highlights the need for policymakers to consider the cost implications of changes in the mix of fields students study.

In addition, our work suggests that while differences in production technology enable some departments to take different approaches to cost management, from changing the mix of faculty to increasing class size, online instruction does not have a meaningful association with college costs, at least in its current form. Any one-discipline-fits-all approach to addressing cost escalation in higher education, including moving more classes online, is likely to be ineffective.

Cost Drivers on Campus

Scholars have long noted the tendency for college costs to grow faster than the broader economy. Some argue that this is inevitable because postsecondary education is inherently labor-intensive and therefore has not benefitted from the kinds of productivity-enhancing innovations that have driven down costs in other industries. Other potential explanations include the proclivity of colleges to maximize revenue in an effort to compete for prestige, school spending on student amenities, and the expansion of unnecessary administrative positions.

In all cases, programs produce a set of outputs, such as undergraduate instruction or research publications, using a large set of inputs, such as faculty of different types, classrooms, office space, technology, and laboratories. The relationship between these varies across fields of study. Some disciplines require intense interaction between students and faculty to produce a given level of instructional quality—think of a debate-driven course with long extemporaneous papers to grade—while others require laboratory sessions that rely on expensive equipment and supplies. By contrast, other fields of study may be able to take advantage of economies of scale and scope, such as those that deliver “101”-style general-education courses for the entire institution. In addition, departments with undergraduate and graduate programs can tap graduate students to serve as lower-cost instructors. Such differences affect class size, faculty mix, faculty teaching load, and non-personnel expenditures—all of which determine the cost per unit of instruction.

Data and Method

To compare instructional costs by field of study, we use department-level data from 2000 to 2017 from University of Delaware,s Cost Study (also known as the National Study of Instructional Costs and Productivity). Instructional activity is measured by student credit hours, organized class sections, and faculty full-time equivalents. Student credit hours and class sections are disaggregated by instructor type, such as tenure track, supplemental, or teaching assistants, and by course level, such as lower-division undergraduate, upper-division undergraduate, and graduate. Finally, institutions report total direct expenditures for instruction, research, and public service and total undergraduate and graduate student credit hours for the entire academic year.

We work with direct instructional expenditures per student credit hour as our main measure of costs, which include salaries, benefits, and non-personnel expenses. In 2015, the study added a component to the survey to capture information about online instruction. These data contain information on online student credit hours by department at the undergraduate and graduate levels.

Institutional participation in the cost study is voluntary. Therefore, we assess how well our sample matched the broader universe of public and private nonprofit four-year institutions operating in the United States, based on the Integrated Postsecondary Education Data System. We estimate that our sample represents 23 percent of all degrees awarded between 1998 and 2015, including 32 percent of degrees at public institutions and 8 percent of degrees at private schools.

We focus on data collected from doctoral, master’s, and bachelor’s degree-granting institutions in the United States. Our analysis looks at 20 core fields of study, including the largest fields (collectively accounting for more than half of student credit hours) and fields that are particularly salient for institutional leaders and policymakers. Our final sample contains 43,819 institution-year observations from 8,221 departments in 20 disciplines at 594 institutions. The data for online programs beginning in 2015 include 3,358 unique departments from 20 disciplines at 238 institutions across 3 years.

Using these data, we construct variables that measure costs, outputs, and inputs. Our primary outcome of interest is direct instructional spending per student credit hour, which we construct by dividing annual instructional costs by annual student credit hours at the department level. We also calculate this ratio for the personnel expenditures portion of costs. Finally, to analyze the sources of differences in costs across programs, we calculate average class size based on fall credit hours, faculty per student, and faculty teaching load.

Instructional Costs Vary by Field of Study (Figure 1)

Differences by Degree Program

We find substantial variation across disciplines in average costs per student credit hour (see Figure 1). Costs range from about $434 per student credit hour in electrical engineering to $163 per hour in math. Our benchmark department, English, costs $199 per hour; the mean cost across the group of 20 fields is $228. Most social-science disciplines and philosophy are relatively less expensive, while science, technology, and pre-professional programs like nursing are more costly. This broad conclusion holds across all institutions—nursing is a more expensive program to operate at elite private research institutions and less selective public institutions alike.

To compare costs across fields of study, we use the English department as a benchmark and look at costs by discipline from 2015–17. First, we calculate the average direct instructional cost per student credit hour for English at each school. We then calculate the within-institution difference between direct instructional expenditures per student credit hour for each of the other 19 disciplines and the same measure for English. We do this for all institutions and disciplines in our sample and then compute averages for each field of study.

In many cases, we find higher instructional expenses in the fields that also produce higher-earning graduates. For example, at $434 per student credit hour, electrical engineering costs 90 percent more than English. In looking at research by Brad Hershbein and Melissa S. Kearney, we see that adults with electrical engineering degrees have substantially higher average salaries throughout their careers. One year after graduation, electrical engineering majors earn more than double the average salary paid to English majors, or $63,000 compared to $31,000. Some 15 years later, that difference is about 83 percent, or $106,000 compared to $58,000.

There are several exceptions to these overall patterns. Math costs about 25 percent less than English, yet adults with mathematics or statistics degrees still earn more: $45,000 one year after graduation, a difference of 45 percent, and $76,000 in year 16, a difference of 32 percent.

In addition, education and fine and studio arts are among the most costly programs to operate, yet their graduates are also among the lowest paid. Education costs about $291 per credit hour, about 45 percent more than English. But adults with degrees in elementary education earn less than English majors throughout most of their careers—in year 16, they earn $44,000 compared to $58,000, a difference of about 35 percent. Similarly, fine and studio arts instruction costs about $273 per credit hour, but arts majors earn about the same or slightly less than English majors, on average: both graduates earn $31,000 one year after graduation. Some 15 years later, arts majors earn $55,000 versus $58,000 for adults with English degrees.

We also look at the growth in the number of student credit hours in each field during the study period of 2000–2017 and find that, on the whole, enrollment is growing fastest in some of the more costly fields. The highest annual rates of growth are in nursing, at 5.4 percent, and mechanical engineering, at 4.9 percent. Meanwhile, we also see that student credit hours are declining in four fields: English, history, education, and fine and studio arts. If those higher costs were associated with challenges in dynamically adjusting inputs, such as instructors and course sections, we would expect that fast-growing fields would have lower costs than slow-growing or declining ones. In fact we see the opposite.

More costly fields also are more likely to have access to additional revenue sources than English departments. Almost all of the most expensive fields are typically housed in separate schools from English, such as colleges of engineering, business, and education. This permits them to generate additional revenue through differential tuition bills or fees, and through separate fundraising efforts from alumni or industry. In some cases, these fields also have access to dedicated state appropriations for instructional costs. For example, in Texas and North Carolina, programs in the sciences, engineering, and nursing are eligible for more public support than programs in the liberal arts and social sciences.

Variety in Cost Contributors Across Fields of Study

We next investigate the factors behind these cost differences, looking at faculty salaries, class sizes, faculty workload—defined as the number of organized class sections divided by the number of full-time equivalent faculty—and non-personnel costs like equipment and supplies.

Instructional cost differences across fields can mostly be explained by large differences in class size across disciplines and, to a lesser extent, differences in average faculty pay. Teaching loads and other non-personnel expenditures explain relatively little.

Instructional Style

Although each field is slightly different, a few general patterns emerge. Economics, political science, accounting, and business have high faculty salaries that are mostly offset by large classes. Engineering and nursing are more expensive than English as a result of higher faculty salaries and lower teaching loads without commensurately larger classes. Workload and non-personnel expenses are important for some of the sciences with laboratory components—namely, biology and chemistry—but otherwise explain relatively little of the observed cost differences.

Consider economics, which is approximately 8 percent less expensive than English. Economics faculty are more highly paid than English professors. Postsecondary English instructors earn about 54 percent less than economics professors, with mean annual wages of $80,340 a year compared to $123,720. Thus if all cost drivers other than average pay were equalized between the two fields, economics would be more expensive. On the other hand, economics classes tend to be much larger than English classes, so class-size differences would make economics instruction less expensive. The faculty workload is a little lighter in economics than in English, so if that were the only driver of cost differences, economics would be about 3 percent more expensive.

Putting these findings together, we see that economics departments are able to field classes that are large enough to more than offset the higher salary and slightly lower workload of economics faculty, resulting in slightly lower average costs than English. Yet economics graduates earn substantially more: one year after graduation, the average salary is $48,000, or about 55 percent more than English majors. Fifteen years later, that difference grows to about 83 percent, or $106,000 for economics majors compared to $58,000 for adults with English degrees.

Mechanical engineering, a fast-growing field that is 62 percent more expensive than English, provides a counterexample. Engineering instructors have mean annual wages of $114,130, about 43 percent more than English instructors, as well as lower teaching loads than English faculty. As a result, the average difference in faculty pay across these two fields contributes substantially to the overall cost difference. Unlike economics, however, classes are only modestly larger in mechanical engineering than in English. Class-size differences are not large enough to offset the higher salary and lower teaching load, and thus mechanical engineering remains much more expensive than English. Research shows wide differences in earnings for mechanical engineering graduates compared to adults with English degrees: $60,000 one year after graduation, a difference of about 95 percent, and $104,000 some 15 years later, a difference of about 80 percent.

Differences in Tenure-Track Faculty Across Disciplines (Figure 2)

Faculty Type and Class Size

A department’s faculty salaries are driven by the mix of tenure-track and adjunct instructors as well as the salary for that particular discipline. We find quite a bit of variation in the share of tenure-track faculty by field (see Figure 2). In nursing, nearly 60 percent of instructors are not on the tenure track, while in mechanical and electrical engineering, three quarters of faculty are in tenure-track roles. English, communications, and math also have relatively low shares of tenure-track faculty. Thus, greater use of tenure-track faculty, which is more expensive, is one explanation for higher personnel costs in engineering, economics, and the sciences. Across the board, we see declines in the share of tenure-track faculty during the study period, with departments relying more on lower-cost adjunct instructors.

The second key cost driver, beyond faculty salaries, is class size. Differences in class size are a function of the mix of course types offered, such as lower-level undergraduate, upper-level undergraduate, and graduate, as well as the average class size for those courses. Lower-level classes tend to be larger and therefore less costly, whereas upper-level and graduate courses have smaller class sizes and are therefore more expensive.

We find substantial differences in the mix of course types offered, with relatively fewer lower-division courses in professional fields like education and business, and many lower-division courses in mathematics and science disciplines like physics and chemistry. Fields with relatively little undergraduate instruction, like engineering and nursing, tend to be more expensive. Looking over the study period, we find average class size conditional on level of course is fairly steady for most fields in the social sciences and humanities, in contrast to marked increases in undergraduate class sizes in engineering and biology.

Trends Over Time

We consider the average annual change in instructional costs for each of our 20 fields of study by looking at pieces attributable to changes in faculty salaries, class sizes, faculty workload, and other expenses. Across many fields, changes in faculty salaries and class sizes account for the bulk of the changes in instructional costs between 2000 and 2017. For instance, mechanical engineering saw a 2.10 percent reduction in cost each year, which is more than fully explained by an increase in class size. Costs for accounting rose by 0.64 percent annually, driven by faculty salary growth of 1.43 percent that outpaced increases in workload and class size. Some fields saw notable changes in faculty workload: education, English, and history all saw reductions in faculty workload over this period, which increased costs, while chemistry experienced a large increase. Only for nursing did changes in non-personnel expenditures increase costs. We also find appreciable declines in expenditures in a few tech-related fields, such as physics and computer science, perhaps reflecting falling costs for technology or lab supplies.

Differences in Shares of Undergraduate Online Coursework (Figure 3)

No Real Savings from Online Instruction

We then turn our attention to online instruction, which has commanded sustained interest from policymakers and institutional leaders as a possible strategy for containing college costs and expanding postsecondary access. We look at data from 2015–17 and find substantial variation in the prevalence of online instruction in the 20 disciplines we study. In undergraduate coursework, the share of online credits ranges from essentially zero in engineering to 13 percent of credits in nursing programs (see Figure 3). The average share of online credits is 6 percent, and 51 percent of the programs in our sample have no online enrollment at all. However, some graduate programs have considerable shares of online credits. For example, online coursework accounts for about one quarter of graduate education programs and one third of graduate nursing programs.

We investigate the potential cost savings of online classes and find a negligible association between online credits and instructional costs. Our estimates imply that adoption of any online coursework is associated with a cost increase of 0.4 percent, but an increase of 10 percentage points in online coursework is associated with a cost decrease of 1.4 percent. Neither of these is statistically significant. We view these estimates as small, especially given the hype about the cost-saving potential of online instruction.

A common hypothesis holds that online coursework can cut labor costs because such classes can be larger and require less face-to-face instruction. However, there is debate about the appropriate size for online courses relative to traditional in-person ones, with some institutions imposing lower enrollment caps for online courses than in-person study. We find some evidence that an increase in the share of undergraduate coursework completed online is related to lower salary costs. But estimates for the other cost drivers suggest that any short-run cost savings on salaries are offset by smaller class sizes and an uptick in non-personnel expenditures.

We note two caveats here: first, this analysis looks only at three years of data and thus cannot illuminate long-run cost changes that might emerge from the sustained adoption of online instruction; and second, we do not observe costs shared across departments, such as capital costs or costs for technology support. We also note the open question of whether the quality of online instruction is comparable to that of in-person classes, especially for less-prepared students. For example, a study by Stanford University’s Eric Bettinger and colleagues looked at a non-selective, for-profit school and found that students earned lower grades and were less likely to persist in school when they completed their coursework online. Some fields may find online education a more useful tool than others in lowering costs without compromising quality. Better understanding this potential is a productive path for future research.

Making Decisions at the Department Level

Over the past 17 years, average instructional costs per credit hour have increased only modestly. However, this relatively flat trend in average costs obscures variation in such cost trends by field of study. We find steep declines in spending in some science and technology fields, due to larger classes and increases in faculty workloads. Other fields, such as nursing, also saw declining costs that reflect a shift in the composition of faculty, with greater reliance on non-tenure-track staff. Yet other fields, such as business and accounting, have experienced escalating costs driven by rapid growth in faculty salaries. For all its promise, online education, arguably the highest-profile change to the delivery of higher education over this time period, is not associated with short-run cost savings.

Public debate about college costs usually focuses on differences between institutions. But the wide variety in costs by field of study should be part of that conversation too. It has important implications for institutional leaders facing decisions
such as differential tuition pricing and for government leaders considering programs to support more students to study engineering or nursing, for example.

Institutions have little control over the prevailing market wages for faculty, but changes in faculty workload, class size, and mix of course types across disciplines show some of the ways that costs might be kept in check. However, changes along these margins are also likely to shape research productivity and the capacity for public service. Thus, changes aimed at reducing instructional costs must balance potential effects on other valued activities of academic departments.

Our findings highlight the broad differences in costs and cost contributors among disciplines. We see a strong need for additional research that sheds light on the effects of instructor types, class sizes, and online classes on field-specific outcomes, including measures of quality such as student performance and success after college completion. For example, it may be that the adoption of online instruction reduces average instructional costs without impinging on quality in math but compromises student performance in chemistry.

Resource allocation decisions have strong effects on learning, instructional quality, and student outcomes, and these effects are likely to differ by field. Further research should explore these differences by discipline to help policymakers and institutional leaders work to reduce spiraling college costs while maintaining the quality of education that students strive to acquire.

Steven W. Hemelt is associate professor at the University of North Carolina at Chapel Hill. Kevin M. Stange is associate professor at the University of Michigan. Fernando Furquim is Director of Institutional Effectiveness at Minneapolis College. Andrew Simon is a postdoctoral scholar at the University of Chicago. John E. Sawyer is professor at the University of Delaware.

This article appeared in the Spring 2022 issue of Education Next. Suggested citation format:

Hemelt, S.W., Stange, K.M., Furquim, F., Simon, A., and Sawyer, J.E. (2022). Major Differences: Why some degrees cost colleges more than others. Education Next, 22(2), 58-65.

The post Major Differences appeared first on Education Next.

]]>
49714612