A few authors of our annual report, The 123s of School Choice, discuss what you’ll find in this year’s edition. Click here to download the full report, or order your free, print copy today.
Mike McShane: Hello and welcome back to another episode of EdChoice Chats. My name is Mike McShane, and I’m director of national research here at EdChoice. I’m here to talk with two of my colleagues, Dr. Marty Lueken, director of fiscal policy and analysis, and Drew Catt, director of state research and special projects, about a new paper that we have out today—and a great resource that we had EdChoice provide—called The 123s of School Choice. It’s a summary of all of the existing research that we have on private school choice programs, and I’m probably going to start by kicking it off to its primary author. All of us contributed some part to this, along with our colleague, Mike Shaw, who wasn’t able to join us on the line today, but I want to start by kicking it over to my friend and colleague, Marty Lueken. And maybe, Marty, if you could start by giving us just a little bit of overview. How did this paper come about? What’s its primary purpose?
Marty Lueken: Sure. Well, it’s good to be here chatting with y’all. Basically, this is the second edition now of The 123s of School Choice, the 2020 or coronavirus edition, if you will. This is basically a publication that is designed to be a resource or kind of a general audience—policymakers, researchers, stakeholders and folks who work on the issue of school choice—but a resource that synthesizes the research on how private school choice programs work and how they affect children, how they affect families, communities and society. And this publication is also a continuation of an older publication that we have had called the Win-Win Solution, the summary of the research on school choice, which was a series of reports authored by Craig Forster.
Mike McShane: Marty, it looks like the paper has seven domains, seven areas in which research has been collected, and I think this is an important point. People frequently ask this question, which is like, “What does the research say on school choice?” As if there is sort of one answer. Generally speaking, I think they’re talking about participant test score. Do kids who participate in school choice programs have higher test scores as a result of that? But it turns out there’s actually a lot of research on school choice, not just on the test scores of kids that participate. The seven domains in this report, obviously, the first one is program participant test scores, but we also have attainment for students who participate in programs, and parent satisfaction for students that participate in programs, but also the public school student test scores, often thought of as the competitive effects of these programs.
What happens to the children who are left behind in public schools? Civic values and practices, obviously people care a great deal about not just test scores but how these programs might affect the civic fabric of our society. Racial and ethnic integration, obviously also something that we really care about, and the fiscal effects of these programs. Maybe, just quickly, we don’t have to summarize the whole thing, but going line by line through it, maybe just understanding what the sort of state of the research was before this report and what new studies came out in this last year. Maybe we’ll start at the top. When it comes to program participant test scores, what was it before this year and were there any new studies that came out this year that sort of changed the way we think about that question?
Marty Lueken: Sure. There have been some studies that came out this year, participant test scores, nothing that I think that’s groundbreaking or new, for example. Before this edition, we found that most studies kind of defined that students participating in these programs benefited in some way on math or reading test scores. There were a couple of studies that found negative effects from programs. Notably, those were D.C. and Louisiana, the two voucher programs in those places. Now, those D.C. and Louisiana are also being studied by a larger longitudinal evaluation and they both concluded this year these longitudinal evaluations. The D.C. program underwent a three-year evaluation, and Louisiana just concluded a four-year evaluation. These are conducted by different teams of researchers, one from the University of Arkansas and another from IAS from the U.S. Department of Education.
What we’ve found in those updates—and these were very comprehensive evaluations, too. Test scores weren’t the only outcome and only aspect of the programs that were studied. But in Louisiana, we found that in year three there were these large negative effects and the gap was closing and then, in the last year, we found that the gap in the test scores increased a little bit. And then, in the D.C. evaluation, we found that the program attained parody, basically that there were no discernible or statistically significant effects on test scores due to the voucher program in D.C. That to say that when evaluating the body of this research, because these programs vary significantly in terms of how they’re designed, there has to be some kind of nuance taken with evaluating these programs. It’s hard to say, overall, that choice itself is passé or something but I think that we’re finding it’s effective in some places and it hasn’t been as effective in some other places.
Mike McShane: And now is the story the same with the second metrics or the program participant attainment? Obviously this is looking at things like high school graduation rates, college matriculation rates, college graduation. Was there some new stuff that came out this year related to that?
Marty Lueken: There were. There were some evaluations that have been conducted previously, which were updated because new data became available. Three of those, it was conducted by and released by the Urban Institute. Those were studies on college attainment and degree attainment outcomes in Milwaukee and Florida, Florida’s Tax-Credit Scholarship Program, and D.C. as well. And, again, the story hasn’t really changed. The story remains pretty much the same. With the Florida study, we found positive effects on college attainment, largely in two-year colleges. In Milwaukee, they also detected positive effects on college attainment. They’re mostly in four-year colleges. And then in D.C., The researchers weren’t able to detect any effect on either college enrollment or degree attainment.
Mike McShane: Now, Drew, the next metric that’s in there is parental satisfaction—so polling and surveys to measure how parents whose children participate in these programs are satisfied. This is a literature that you yourself have contributed to so I’d be interested, what’s the sort of state of that question? Was there anything new that came out this year that sort of changed our views on it?
Drew Catt: Yes. Up until this edition and up through the mid points or early last year, everything was extremely positive. There was one study that showed no visible effect that we were able to track down since the last edition of the 123s, and there’s now one study that shows both any positive effect and any negative effect, and that is my own research of Arizona’s private school choice programs that we released on our website that I worked on with Albert Chang. That one’s a little nuanced so when it comes to satisfaction with the school, ESA and tax-credit scholarship parents were less satisfied than the charter, the fee-paying, private and the traditional public school parents.
But when it came to various aspects of the school climate, things were a little more mixed with on three of the five measures that we looked at. The ESA and or tax-credit scholarship parents actually scored higher than the other parent groups. For both of those, we kind of had to label them both any positive effect and any negative effect. I guess all that to say that there has yet to be any research that is purely negative.
Mike McShane: And now you’ve also contributed, or are in the process of contributing, to research on competitive effects—so this is that question of what happens to students who are left behind in public schools? So, was there any new research on that or what’s the sort of state of the literature there?
Drew Catt: Yes, there’s only one real piece of new research, which I’ll get into momentarily. In terms of the state of the research, it is overwhelmingly positive. There are two separate studies of Florida programs, one that showed no visible effect and one that showed a slight negative effect. However, there are, overall, 25 of the 27 studies that do show positive effects. In terms of the new research, there is some work by David Figlio and his team, which they David Figlio’s been doing competitive effects research for over a decade and a half. He’s very well known in the competitive effects world and private school choice research, but he had a paper with his colleagues that came out through the National Bureau of Economic Research or NBER. They looked at the effects of student eligibility expansion of the Florida tax credit scholarship program. And it really showed that, as the program evolved and expanded, that the math and reading test scores were actually increased for the students that remained in the public schools and, interestingly enough, the rates of absenteeism and school suspensions for the students that remained in the public schools decreased, which that’s great to see and it’ll be interesting to see if more research in the competitive effects literature kind of goes down this way of looking at more than just test scores.
Mike McShane: That’s great. Now, the next two things that are in the paper looking at civic values and racial and ethnic integration, it’s my understanding, and either of you can correct me if I’m wrong, that there was just one new study related to the sort of civic values or sort of questions around practices, civic values, civic practices, and that was the study out of Milwaukee that looked at students who had participated in the voucher program and later criminal activity. It’s my understanding they found, I believe it… Well, I’m actually quoting the study right here so I know what it found, “exposure to private schooling through a voucher is associated with lower rates of criminal activity, such as committing misdemeanors, felonies and theft.” That generally adds to a positive skew of the 11 studies of civic values and practices. Six found some positive effect, five of them found no effect but there are no studies that have zero effects. It’s also my understanding, with respect to racial and ethnic integration, there were no known new studies that came out in 2019. Marty, am I correct in saying that?
Marty Lueken: Yes. Yeah, that’s correct.
Mike McShane: OK, great. That leads us to the seventh and final look, which I can’t think of a better person to talk about this than our director of fiscal policy and analysis, but that is the fiscal impact. And I think this is a particularly important question as we look at the economic devastation that is being wrought by the coronavirus. School budgets are going to be in a bind, sales-tax revenue down, income- tax revenue down, property taxes will probably be okay, but lots of the tools that state governments have to fund such programs. Was there a new research on the fiscal effects? What do we know about the fiscal impacts of private school choice?
Marty Lueken: Yeah, so actually one of those papers was one that I authored on two tax-credit scholarship programs in Pennsylvania and that was actually an update of the educational income tax credit program, the ITC, which was an update, and then the OSTC, the Opportunity Scholarship Tax-Credit Program, which was a new analysis, a fiscal analysis, on that program. And, basically, the analysis estimated that those programs generated fiscal benefits combined overall for both the school districts and for the state that was worth between about $4,000 up to $6,800 per scholarship,. Cumulatively over the lifetime of those programs, that estimate is about $3 billion to $5 billion. And then there was also an analysis, and it came out recently on Arizona’s tax-credit scholarship program as well, and that analysis also estimated a net fiscal benefit worth up to $285 million just in fiscal year of 2018 alone.
There’s been a lot of analysis on the fiscal effects of these programs and it’s not really surprising, or it shouldn’t be surprising, because intuitively, if you compare how much the public K-12 system is funded with the level of funding for private school scholarship programs, there’s a huge gap. The scholarships are often, on a per-pupil basis, worth about from a third to a half, in many cases, of the total cost, per- pupil cost, for the public K-12 system. If you look at that gap, somewhere there’s savings in there. How those savings are distributed across different groups like different tax payers, local state taxpayers, school districts, that’s a really complicated question.
Mike McShane: Taking a step back and looking at the big picture of this report, obviously 100 north of, I think, 150 studies are summarized here. I want to sort of conclude with two questions for each of you, and I’m happy to jump in on these as well, but the first question I have is what do you think is the most common misconception about the school choice literature? As you see it discussed in the public sphere, you hear legislators talking about it and maybe even you hear other researchers talking about it, what do you think is the biggest misconception out there. Maybe, Drew, I would ask you first and then, Marty, you can thank Drew for buying you some time and I’ll start with you for the next question. But, Drew, what do you think is that biggest misconception out there?
Drew Catt: Yeah. Honestly, I think it’s the cream skimming. That’s, in my opinion, one of the biggest misconceptions that’s been kind of pushed back against.
Mike McShane: Explain that, though. When you say cream skimming.
Drew Catt: That is that these programs take the best of the best students into the private schools and leave the “lower-performing” students and the public schools. It’s great to show what the competitive effects literature, to show that the students that do stay in the public schools actually improved test scores overall. But really, there’s been some research focusing, at least here in Indiana, showing that, well, actually the students that are participating in the program in my first state are typically lower income and lower performing when they’re entering the program. There’s some more research coming out, and for other programs as well, showing that, yes, these aren’t necessarily the best and brightest students on average that are participating in these programs. I definitely think that’s one misconception.
Mike McShane: Yes. Thank you, Drew. Marty, what do you see as the biggest misconception? Is that the cream skimming? Is it something else? What do you think?
Marty Lueken: I think that there’s a lot of misconceptions out there. One of the biggest ones that I see, certainly in my line of work, is the fiscal question—the argument that these programs somehow drain resources from district schools. It’s true that districts lose revenue when students leave, for any reason though, but it’s also true that when enrollment changes, whether it goes up or down, that costs go up or down as well. And so, when the neuroma declines, the cost for a school will go down as well. Now, it’s a little nuanced to what extent that is. If you’re talking about a couple of students leaving a school or a classroom, there’s only so much that a school could do to adjust to that, but that’s well within the margin of fluctuation for schools that they deal with that type of change on a regular basis.
When you lose a lot of students, then you will have more opportunities to reduce costs as well. In addition to that, when students leave, states also are usually no longer obligated to provide state aid to those districts, too. So ,the states, they fund the private choice program, but at the same time, there is usually an offset of funding when students leave districts and that usually results in a net fiscal benefit because you usually have these huge gaps in funding the private school choice program and public K-12 system.
Drew Catt: Yeah, and this is Drew again. I thought of another thing that it’s not necessarily a misconception of the research, it’s just a misconception of these programs and private schools in general, and that is that these are dollars going to “unregulated” schools. Like to dive back into my first EdChoice report, Public Rules on Private Schools, showing that private schools in general are already very regulated and, in fact, in some of these programs such as like Indiana, the schools that are participating in the program are actually more regulated than the public district schools. That’s less of a misconception of the research and just, I would say, a misconception of private schools in general, especially private schools participating in these programs.
Mike McShane: I think those are two great points and you stole even what I was going to say. My final question, summing this up maybe, let’s say, to our educated lay readers. So, a smart person who maybe that doesn’t know this research or maybe even school choices, as well as people who spend their whole day working in this. If you wanted to summarize one key takeaway, after all of the work that you all have put into this report—I guess I put a little bit of work in, but not nearly as much as you all did. What one takeaway that you hope that people could walk away from this and knowing something to be true? Marty, I’ll start with you. And so, this doesn’t have to be necessarily the one thing, but just one thing that you hope people take away from this report.
Marty Lueken: Sure. For me, that would be that when you look at the whole body of research—of rigorous research—that these programs in general, the studies tend to be skewed towards the positive overall. But I think it’s also important to realize that no two programs, no two choice programs, are created equally and design really matters, right? It’s important when evaluating the body of research to also consider how these programs are designed and what the effects of different parameters of these programs can have on students.
Mike McShane: Drew, your one takeaway.
Drew Catt: Yeah, I’m going to kind of focus a little bit on the side of the parent satisfaction research and that to borrow an old adage, you can satisfy some of the people all the time or all the people some of the time, except when it comes to schooling. And that is you are not going to find a single school that every single person in there is completely satisfied with. Everyone’s going to have an issue with everything, even the negative research. There are students that perform positively. We’re just looking at what is negative on average or no visible effect on average, and with every single one of these programs, regardless of the research finding, there are multiple students for which these are kind of a lifeline and for, I guess, for whom without these programs they would, in the eyes of themselves and their parents, be suffering. It’s great that these programs do exist to provide an opportunity and a choice.
Mike McShane: Well, Marty Lueken, director of fiscal policy and analysis, Drew Catt, director of state research and special projects, thanks so much for sharing your insights about this report. Friends, if this conversation has piqued your interest, as I hope it would, you can order your own copy of this report. Head over to www.edchoice.org/order123s. Folks, as always, please make sure to subscribe. Go to your favorite podcast sending service, however you get your podcasts, make sure to subscribe, give us some reviews, preferably good ones, but I don’t know, just follow your heart, go with what you feel. Tell us if we can do any better and always make sure, head to our website, www.edchoice.org, you can sign up for our email list where you can get awesome content like this report, but also keep abreast with school choice developments all across the country. I’m Mike McShane, director of national research, and it’s been great spending time with you and having another EdChoice Chat.