Ep. 182: Big Ideas – “School Choice: Separating Fact from Fiction” with Patrick Wolf

Jason Bedrick: Hello, and welcome back to EdChoice Chats. I’m your host, Jason Bedrick, director of policy at EdChoice. And this is another edition of our Big Ideas series. Today, I’m excited to be joined by Dr. Patrick Wolf, professor and 21st Century chair in School Choice in the Department of Education Reform at the University of Arkansas. And the subject of our conversation will be the book that he edited, School Choice: Separating Fact from Fiction. Pat, welcome to the podcast.

Patrick Wolf: Happy to be here, Jason.

Jason Bedrick: So, before we dive into the content of your book, perhaps you could share with our listeners a little bit about the genesis of this project.

Patrick Wolf: So, every two years, I run a seminar in school choice for the doctoral students in our program. And the idea of the seminar is to make it a deep dive into the 12 to 14 crucial questions surrounding parental school choice. And basically I had to come up with a capstone assignment for the students to challenge them in terms of understanding the research on school choice and writing about it.

And I decided, “Hey, let’s see if we could put together a book that systematically and comprehensively reviews the literature on some crucial questions surrounding school choice.” And that’s sort of where we came up with this idea.

I also reached out to a few people outside of the seminar, who I knew were doing important research on key questions. So, my graduate students sort of provided the core set of chapters, and then it was augmented by Brian Ray, who’s a home-school expert, and Phil Gleason, who’s a charter school expert.

Jason Bedrick: And your book focuses on the evidence on the effects of school choice on outcomes besides test scores. Why that decision?

Patrick Wolf: Yeah. So Jason, as you know, as well as anyone, most of the research on school choice has focused narrowly on student test scores, students’ scores on standardized tests. So, there’s a lot of research already out there. There are actually a lot of summaries, meta analyses and such out there on both the private school choice side and the charter schooling side.

I just felt that that field had already been plowed. And so this book was an opportunity to bring to bear summaries of the evidence on other key elements of parental school choice besides test scores. And of course, one thing we find in one of the chapters is that test scores, aren’t essential to parent decision-making about school choice as many people thought.

Jason Bedrick: So, actually that’s a great place start the first chapter by Heidi Holmes Erickson very logically begins at the beginning: how parents choose schools. So, what are the things that are very important to parents when they’re choosing schools? And so maybe some surprising things that aren’t so important.

Patrick Wolf: Sure. Well, Heidi, in reviewing the literature on how parents choose schools and what they look for, she did confirm that academic quality is a significant consideration of parents along with safety and special programs that are particularly important for their child. So, these are sort of a constellation. So, academic quality was sort of primus inter pares, kind of first among equals, among a cluster of important factors.

But the other thing that she discovered in this research base is that parents don’t always define the academic quality of the school in terms of its aggregate test scores. They look at curriculum, they consider teacher quality. They just kind of get a sense, “Is this a rigorous school? A school offering a rigorous educational program that’s going to challenge my child, but not overwhelm them.”

That’s kind of what they’re looking for. And test scores are an information source for them, but not necessarily the information source that drives their decision on quality.

Jason Bedrick: And are they looking—aside from safety, which is a lot of upper middle income families take for granted. I think they just sort of assume their schools are going to be safe, so they don’t put it. If that part of Maslow’s hierarchy of needs is taken care of, they take it for granted. For lower-income families that might be in schools that they consider unsafe, suddenly you see that move up in terms of their stated preferences. But are there other things they’re looking for as well, even aside from the academic qualities of a school?

Patrick Wolf: Sure. There is a subset of school choosers for whom a religious school is important, a religious educational environment is important. They don’t always insist on a perfect match with their sectarian religious affiliation and such a lot of families sort of want a Christian school, a school with some Christian affiliation Catholic or Protestant for their child to be in. Because they feel that the school will then reinforce certain moral values and moral commitments and religious commitments of the family and allow children to grow in their spirituality, as well as in their learning of secular subjects.

So, religious commitments and a religious school is important for a subset of parents. And that includes the Catholic, Evangelical, Christian, Jewish, Muslim, etc. That is an element. It’s not the most important element for the majority of parent choosers, but it is the most important element for a subset of choosers. So, that’s important.

Some measures of convenience like location certainly plays a factor. The evidence suggests it’s not the dominant consideration, that parents often we’ll choose a school that isn’t the closest school of choice to their home. But if they find what they’re looking for in other areas like academic quality and safety in a school that’s close to their home, then they will choose that school because they get the extra convenience of location.

Jason Bedrick: And then once they’ve selected a school, how satisfied are parents with the schools that they’ve chosen relative to parents that are not participating in school choice programs?

Patrick Wolf: Well, Jason, it’s probably one of the most consistent findings in the literature on parental school choice is that if parents are given the opportunity to select their child’s school, they tend to be much more satisfied with that school. Now, the question is, is it really because the school is better for the child or is it maybe because the control group, parents who lose voucher or charter lotteries have sour grapes, they’re very disappointed?

So, they rate the schools that their child has to stay in very low, artificially low. So, that creates this sort of false advantage of the choice for students. So, Evan Rhinesmith was very clever in how he designed his chapter on parent satisfaction. He looked at the analysis that are based on these lotteries, lottery winners and losers where you might have the sour grapes factor biasing the parental satisfaction differences uncovered.

But he also then looked at studies of parent satisfaction where families were more sort of, it wasn’t the artificial circumstances of a lottery. It was just sort of a natural, “Well, these parents wanted to enroll in a school of choice. These other parents wanted to stay in the school that their child was in, the public school that their child was in.”

And so he looked at differences in satisfaction rates there and they’re very similar in both groups of studies. Lottery studies find large positive effects on parental satisfaction for parents who are given the opportunity to choose their child’s school, non-lottery studies, observational studies also find very similar, a large positive effects on a parent’s satisfaction with schools.

And there’ve been a few studies that have sort of drilled down and look at the specifics. And the specifics kind of makes sense in terms of parents’ satisfaction with specific elements of their child’s school, tend to reflect the strength of the private school sector or the charter school sector relative to the traditional public school sector.

I’ll give you a couple of examples. So, in a study that Brian Kisida and I did the effect of the D.C. Opportunity Scholarship Program on parent satisfaction. The areas in which they were most satisfied are areas where the private schools in Washington have a strong reputation, and that is academic quality teacher quality and the opportunity to practice the family’s religion.

Those were the highest and safety too. So, those four were sort of the highest specific components of satisfaction that the parents expressed. The areas where the parents in the private school choice program really did not have significantly higher satisfaction than the control group parents were things like special education services or the facilities of the school.

And if you’re familiar with private schools, those tend to be areas where they don’t have a lot of bells and whistles. And so it would be perfectly logical for parents to not be dramatically more satisfied with those elements of their school of choice, because the school sector that they’re choosing private schooling is just not known for those.

So, in a way, this logical alignment of parental satisfaction advantages for choice parents, with specific components that the school sector is known for or not known for, it’s kind of lays any fears that parents are simply inflating the true satisfaction they have with their school because they chose it.

Now, there’s a real logic to what they are expressing in these surveys and it suggests that they’re much more satisfied with schools that they’ve been able to choose to fit their child’s needs.

Jason Bedrick: Now, I think parental satisfaction should be one of the most important metrics that policymakers look at because it does capture so much. Parents have a variety of different legitimate preferences. And so that satisfaction question really captures how well schools do at meeting all of those different types of preferences. But those of a more technocratic bent are skeptical of too heavy a reliance on parental satisfaction.

And they want to know, “Well, do these parents really have good cause to be so satisfied?” So, aside from test scores, because your book doesn’t address those. That’s been covered before so many times. Do we have evidence that choice students are succeeding more than their non-choice peers in other areas like attainment or non-cognitive skills?

Patrick Wolf: Yes. And there are actually three chapters that look at outcomes that are non-test score outcomes that are sort of outcomes of schooling. There’s also a chapter on racial integration, which is not necessarily an outcome of schooling. It’s a condition of schooling. But in terms of the non-test score outcomes of schooling, Malachi Nichols examines the evidence on the short-term effects of school choice on student non-cognitive conditions.

And these are both statements or expressions that the student makes about their own character skills like grit and conscientiousness and such. And also actual behavioral measures of these kinds of character traits that we know are associated with long-term success in life. And what Malachi found is that there is some research suggesting that students in schools of choice describe their character traits as lower than comparison group students.

But the measures of their actual behaviors suggest a school choice advantage regarding character skills and Malachi reasons that this might be a sort of a reference group bias for the choice students. That they’re switching from a situation where there are kind of low expectations for conscientiousness and effort and grit and such to a school where there are high expectations for it. And there are a lot of peers around them who are working very hard and are committed to their academic success.

And so the child in comparison says, “Hey, I’m not very strong on these non-cognitive skills.” Because they’ve got a different reference group, a much higher reference group of peers around them, that are the source of their own internal comparison. But then when it comes time to measuring their actual effort, effort on surveys, effort exerted on tests and other sort of tests that people have devised for examining character skills in students.

The choice students actually do better than the comparison group students in public schools. So, the public school students think they’re better, but the private school choice students actually demonstrate that they’re better on these measures.

Jason Bedrick: I think this was actually something that was highlighted at the beginning of the movie, Waiting for Superman, where they have that montage. They’re talking about American students doing really poorly on international exams like the PISA, relative to other nations. And you see them skateboarding and doing a bunch of silly things, getting themselves into trouble, getting themselves hurt. And the voiceover says, “But there’s one area where American students are the best in the world, and that’s in their confidence.”

In other words implying they’re overconfident, but don’t really perform that well. And here you’re saying, “Well, actually choice programs start to reverse that trend.” They perform better. They have higher levels of achievement when it comes to non-cognitive skills. So, conscientiousness and grit and whatnot. But because they’re aiming higher, they actually have a more modest evaluation of themselves. So, that’s a very interesting finding and not one that usually makes the front page of the papers.

Patrick Wolf: No, no, definitely not. Corey DeAngelis looked at the effects of school choice, private school choice programs in particular on civic outcomes. And there’ve been several reviews of this question. I’ve done a couple of myself. Corey’s approach was to focus on studies. So, not just individual findings, but characterizing an entire evaluation as showing that private school choice either had a positive, a neutral or a negative effect on student civic outcomes.

Jason Bedrick: And just to be clear, first of all, you’ve been doing research on the effects of civic outcomes for decades. And I used to highly recommend people look at your literature review civics exam, but that’s now 12 years out of date. There’s so much more that’s happened since then.

But when we’re talking about civic outcomes, we’re talking about things like political tolerance, civic engagement. And recently you and Corey and others have started to broaden the research on civic outcomes into areas like social order, crime, teen pregnancy and other things. So, it’s a very broad and rich literature that you’re talking about here, but go ahead.

Patrick Wolf: Yeah. Yes, Jason. And in the school choice debate, the question of the civic and community effects of an intervention are really important because a major justification for universal schooling and the traditional public school system in America is the idea of preparing young people for their citizenship duties in our democratic republic.

I mean, young people have a lot of responsibilities to keep this country going, including educating themselves about civic matters, voting, being active in their community. These are all the responsibilities of free citizens in our country.

And so the traditional public school system has often justified itself and argued against school choice in terms of the claim that only government-run schools will produce effective and involved citizens. So, these virtues of political tolerance, civic engagement, and sort of successful involvement in the community and adherence to social norms, legal norms, etc.

I mean, these are really vital outcomes of education. We’ve always described as important outcomes of our K-12 education system. And so it’s a crucial question, whether an intervention like school choice, like private school choice, has positive, neutral or negative effects on civic outcomes.

The critics of school choice would have a think that the effects of private school choice would be negative on civic outcomes consistently negative and highly negative. Supreme Court Justice Stephen Breyer in his descent in the Zelman v. Simmons-Harris Supreme Court case, which confirmed the constitutionality of private school choice programs, wrote that the expansion of these programs would transform American society into a sort of balkanized system where different tribes will war against each other and will be because of private school choice and the decline in tolerance that will bring that about.

So, Corey looks at the actual evidence from 11 studies of this question and finds that the effect of private school choice on political tolerance is consistently neutral to positive. So, some no findings, no difference. Many positive findings of private school choice actually enhancing the political tolerance of young adults. He then looks at civic engagement and again, finds that the effects of going to a private school as opposed to a public school are neutral to positive in terms of involvement in your community in civic matters, including voting and other civic behaviors.

And then he looks at the few studies that have examined the effect of school choice, private school choice on crime and sort of social order and finds that the initial results from that pioneering research is positive.

So, it does look like students don’t have to attend government-run schools to emerge from their K-12 education, with a robust set of civic virtues and commitments to basically practicing their civic responsibilities in our democratic republic.

Jason Bedrick: So, one of those common critiques of school choice programs, particularly of charter schools, but also of private schools, is that the schools engage in cream skimming. Right? So yeah, you’re finding all of these positive effects. Sure. But it’s not because you’re doing anything better.

That’s because you’re just selecting the top students or when you have lower performing students, especially students with special needs, you’re counseling them out. So, is there any evidence to support or contradict these claims?

Patrick Wolf: Yeah. So, Kaitlin Anderson did a deep dive into the literature on this question. There are quite a few empirical studies of how selective students are when they participate in different kinds of choice programs. And basically Kaitlin’s takeaway is it depends. There’s a lot of variation, but there really isn’t any consistent and sizable pattern of cream skimming and selectivity in schools of choice.

There are some types of schools of choice, and there are schools of choice in certain areas where the students participating are somewhat more advantaged in terms of achievement, somewhat less likely to have disabilities and somewhat less likely to be English language learners. Those are sort of the three areas where there’s some evidence of cream skimming, but it’s very modest in magnitude and it’s not terribly consistent.

In contrast, there are other areas, other choice systems, both private school choice systems and public charter school communities or systems where there’s negative selection. Where there’s evidence that say there’s public charter school that only serves kids with disabilities. Or a private school that really focuses on kids who have been expelled from school, dropped out of school.

So, they are targeting their school mission and their student recruitment on the most disadvantaged students. So, Jason, I mean, this is what happens when you have school choice, right? Is the schools want to stand out and be distinctive in some way in the population they serve in their approach to education. And so parents then look at a distinctive set of schools and try to match their child’s needs with what the school is offering.

And so you’re going to see a lot more differentiation within school sectors of choice, private schools, charter school sectors in communities. And they’re going to be some of those schools that are attracting more advantaged students who want a different approach to education. And then there’ll be other schools that are specifically targeting and attracting the most disadvantaged students. And so when you add it all up, just not consistent patterns of cream skimming in most of the choice studies that are out there.

Jason Bedrick: Another chapter by Brian D. Ray examines homeschooling, which is always been something of a black box in education research. His chapter contains the first systematic review of the peer reviewed scientific literature on homeschooling. So, what did he find?

Patrick Wolf: Yeah, so this is a big question because almost 5 percent of children across the country are educated in their home. And homeschooling is now legal in all 50 states and the District of Columbia. There’s a long period of time in the 20th century when homeschooling was illegal in most states. You could be arrested for educating your child at home, which seems a little bizarre when you think about it.

But Brian examines first the research about the demographics of home schoolers and that shatters a lot of myths that are out there. I mean, there’s a lot of thinking that home schoolers are overwhelmingly sort of these cultish kind of people. They are separated from society. That’s not the case at all.

They’re somewhat anti-institutional in their approach, but they’re not antisocial. And typically home-school families are very closely ensconced in their community, very active in their community, very heavily networked in the educational program that they deliver to their children.

The one distinguishing feature about home-school families that stands out is they are overwhelmingly two parent families. It’s just very difficult to home-school your children if you are a single parent. Obviously, you have to have a source of income, you have to be working, and it’s extraordinarily difficult to arrange your child’s education in the home while also being the only breadwinner in the family.

So, other than that, they are somewhat more likely to be white than to be people of color. But actually the largest growing, the fastest growing segment of home schoolers demographically right now are African American families. So they’re starting to catch up.

In terms of the effects of homeschooling, what effects do they have on student achievement, student attainment and student social skills? The research suggests they’re overwhelmingly positive effects even on social skills, which is often kind of a dig against homeschooling is that children won’t be prepared to socialize effectively, that their social development will be stunted because they’re in their home only interacting with parents or perhaps siblings. But that’s really not how most homeschooling works.

Most home schooling, a lot of it, the learning takes place outside of the home, in the community, in networks of other home-school students. So, there are lots of opportunities for socializing in the home-school day of most students. And the research that’s available suggests that students who are home-schooled actually have somewhat stronger social skills than students who are conventionally educated.

Now, all of the home-school studies are non-experimental. You just can’t randomly assign children to be home-schooled or not home-schooled. So, there’s some self-selection component that is influencing the comparisons and the differences. The literature is just not as rigorous on homeschooling as it is in other forms of school choice, but the evidence that’s there does suggest a net positive effect of homeschooling.

Jason Bedrick: And of course, I mean, when we’re thinking about home-schooling, the question often is just, “Well, are these kids doing well, period?” Not, “Is this environment having a particular effect?” We just want to make sure that they’re not actually falling behind.

So, in some sense, I think that should be sufficient to allay the concerns of some that while we don’t really know what’s going on, so we need to have much more government involvement and monitoring of home schools. There just doesn’t seem to be evidence of the systematic failure to provide a proper education at home.

Patrick Wolf: That’s right, Jason. Now, there have been a few horror stories. I mean, there have been a few cases where abusive parents have used homeschooling as a tool to delay the discovery of their child abuse. And that’s horrible. It’s terrible, it’s tragic, but those are extreme cases, extreme outlier cases.

And there are other tools that we can use to recognize and stop child abuse without sort of undermining the opportunity of the 99.999 percent of home-school families who are contributing to their child’s well-being by homeschooling not undermining their efforts to do so.

Jason Bedrick: You’re one of the foremost researchers on school choice and yet you say in the introduction to your book, that you will learn a lot from the papers that eventually became the books chapters. So, before we close, were there any findings that you found particularly surprising or illuminating?

Patrick Wolf: Oh, that’s a good question. I thought that the grit findings on Malachi Nichols’ chapter were interesting. I suspected that they would be sort of more consistently positive. So, I thought it was interesting that he discovered this pattern where the student self-reports were lower for the students in schools of choice. And that sort of provoked this whole idea in discussion about reference group bias.

So I thought that was an interesting finding. I also thought a lot of the home-school findings—especially the home-school findings on social skills were surprising. I thought it’d be more null-ish.

It would be more like home-school kids have social skills on average that are comparable to the conventionally educated kids. But there actually are some studies suggesting that there are positive social effects of home schooling, which is kind of interesting and cuts against the conventional wisdom on that question.

I thought some of the research on how parents choose schools was interesting. And I think it’s just very important that research is starting to unpack the nuance when it comes to parents school quality and test scores. And how these approaches are not as automatic as a lot of people would assume that parents have a more global idea of school, quality and factor in a lot of different components.

They almost construct a balanced scorecard in their head where they consider aggregate test scores. They consider their impressions of the teachers that are in the school. They consider just how well the organization is run. They consider how focused the mission is on student academic success.

And they sort of do this balanced scorecard calculation in their head when comparing different schools, and might choose a school for their child based on academic quality that doesn’t necessarily have the highest test scores in the set of schools they’re looking at. Because they see other elements of school quality and school academic influence, and focus in the school itself.

They see it with their own eyes in their school visits and consider that to be a metric of quality that isn’t fully represented in test scores. So I think another finding that’s really important in the How Parents Choose and What They Look for in Schools chapter is that there aren’t big differences between a higher income parents exercising choice and lower income parents exercising choice.

They tend to look for the same things. They tend to engage in a similar process. The one exception is that higher income families and two parent families tend to visit more schools than lower income families and single parent families when making their school choice. And that kind of makes sense because they have the time and the resources to be a little more comprehensive in their school search.

So, the low-income families do rely somewhat more heavily on written material, summary material and less heavily on eyes on the schools on the actual school visits. But the differences are modest and subtle. And generally, families are looking for the best school for their child.

They use a variety of factors to figure it out, and they use a very sort of logical search process for informing their decision. And being a low income parent doesn’t restrict, significantly restrict one’s ability to make a good school choice for your child.

Jason Bedrick: Well, certainly evidence that cuts against the prevalent canard that low income families don’t know what’s best for their kids.

Patrick Wolf: Yeah. Don’t tell them that. Because they think they know their kids well and of course they do. They are the world’s foremost expert on their children and their children’s needs. And they understand schooling and what schooling can offer and can distinguish schools that are going to deliver what their child needs and schools that aren’t going to deliver what their needs.

So, it does sort of fly in the face of this idea that lower income families aren’t capable of exercising school choice. Many of them are highly motivated to do so and very capable.

One last thing I want to add, Jason, is throughout several of these chapters in this book, what really stands out is that school choice is special for African-American families. I mean, we’ve seen this in a number of studies. We’ve seen it in polling. They are the one demographic, African-American families is the demographic for which school choice programs are the most popular when you poll them.

Private school vouchers, charter schools are more popular for African-American respondents than any other demographic group. Their participation in school choice is higher than their representation in the community. So, the population of students who participate in private school choice programs, charter school programs are disproportionately African-Americans.

And in some cases where there are large enough samples to look at the effects of school choice, just on African American populations, we see that choice tends to have its most positive effects on African-American populations of students.

So, that’s sort of one element that comes through here is that African-American families are particularly interested in school choice and securing the benefits that it delivers to them.

Jason Bedrick: My guest today has been Patrick Wolf, professor and 21st Century Chair in School Choice in the Department of Education Reform at the University of Arkansas. And his book is School Choice: Separating Fact from Fiction. Pat, thank you for coming on the podcast.

Patrick Wolf: Glad to do so, Jason.

Jason Bedrick: This has been another edition of EdChoice Chats. If you have any ideas for authors, you’d like us to interview for the Big Ideas series, please send them to media@edchoice.org and be sure to subscribe to our podcast.

Follow us on social media @edchoice. And don’t forget to sign up for our emails on our website edchoice.org. Thank you. We’ll catch you next time.