We are joined by Ian Kingsbury, an education policy fellow with the Empire Center for Public Policy. He discusses his recently-authored EdChoice study, Online Learning: How do Brick and Mortar Schools Stack up to Virtual Schools?
Jason Bedrick: Hello, and welcome back to EdChoice Chats. I’m your host, Jason Bedrick, director of policy at EdChoice, and this is another edition of our Big Ideas series. Today, I’m excited to be joined by Dr. Ian Kingsbury, an education policy fellow with the Empire Center for Public Policy, who is the author of a new EdChoice study titled, Online Learning: How do Brick and Mortar Schools Stack up to Virtual Schools, which is the subject of today’s conversation. Ian, welcome to the podcast.
Ian Kingsbury: Thank you. Thank you for having me.
Jason Bedrick: Our pleasure. Look, when COVID-19 hit earlier this year, it forced our K-12 schools to close down physically, and we entered what was perhaps the world’s largest experiment in virtual learning. Your study compares the effectiveness of virtual schools that were already operating as virtual schools, to brick and mortar schools that had to switch during the pandemic, in terms of how effective they were on a variety of metrics. So how did you make that comparison, and what did you find overall?
Ian Kingsbury: I partnered with K12 Inc., which is a provider of online instruction and curriculum, and they sent out a survey to parents of students enrolled in their virtual schools, and the survey asks them about their student’s virtual learning experience last year. So it asks them the performance of the schools across a few different constructs that we know are critical to student learning. The other thing is that it asks the parents whether they have students who were enrolled in brick and mortar schools last year, and then it asks them to answer the same questions that they also answered for their student who was in the virtual school. So that way, we have a one-on-one comparison of those brick and mortar students and their siblings who were in virtual schools.
Jason Bedrick: Why was it important that you’re asking the same family that has siblings in each? I mean, in some sense, that seems to be a control. What are you controlling for there?
Ian Kingsbury: Well, there’s some level of judgment or arbitrariness to the whole thing, when you ask questions like, “Do you think your child earned a lot?” and this is great because now we have parents who are applying the same universal standard. So it’s direct comparison of those experiences, and you cut down on the arbitrary nature of those judgments.
Jason Bedrick: Now, you mentioned that you partnered with K12 to do this. I know they’re one of the largest providers of K-12 virtual learning, it may even be the biggest provider nationally.
Ian Kingsbury: I believe they are.
Jason Bedrick: Yeah, they might be, yeah. But are they representative of the sector as a whole?
Ian Kingsbury: It’s hard to say. They’re certainly, as you allude to, they’re an important player and I believe they dominate market share. Whether they’re representative, I don’t know, but absent being able to pull students in every existing virtual school, it’s a pretty good logical place to start.
Jason Bedrick: Certainly. Now you found that the K-12 students, for clarification, I will say online students, because it can get confusing. The students who were already doing online with the K-12 provider, did better on a variety of metrics, but critics might still say, “So what? Is this really a surprise?” Of course, we should expect that virtual schools that had been intentionally designed to be virtual schools, would be doing better than those brick and mortar schools that are set up to be brick and mortar schools, that at the last minute, due to a totally unforeseen event, have to scramble to figure out how to implement online schools, right? So I mean, so why should we be surprised?
Ian Kingsbury: Yeah, you’re absolutely correct about that, and in fact, if we saw the opposite, if we saw that parents held in higher regard the experience that the students in brick and mortar schools had after the school switched to online learning, that would be really troubling, that would almost be like an emergency break glass moment, that those schools that transitioned rapidly were outperforming those virtual schools. But it’s notable for two reasons, one in the magnitude or the gulf in their responses. So, the question that I keep coming back to is that one, “Did your child learn a lot?” Well, for brick and mortar students, you have like 13 percent of parents who agree or strongly agree that their children learned a lot after the school switched to online learning, and for K-12, it’s something like 88 percent of parents who agree with that assertion. So, 13 percent versus 88 percent who say that their child learned a lot. So it’s not just a marginal difference, it’s a gigantic difference.
The second reason it matters is when we talk about school choice programs and we evaluate them, we defer to a counterfactual and traditionally, that is a student assigned to their local public school. But right now, obviously amidst the COVID pandemic, that’s not asking the right question. For millions of students, that’s not the right counterfactual. They’re not sitting in classrooms, they are sitting at home learning virtually. And so, the question in evaluating the effectiveness of these virtual schools, and again, by that I mean schools that were virtual before, the question is no longer how they compare to those schools where kids are attending in person, it’s a question of how it compares to the schools that are operating virtually.
And this is important to understand, because for example, The Philly Inquirer put out an editorial about two weeks ago saying that virtual charter schools are not the answer for our current educational problems. Well, I would agree with that to an extent. I think education savings accounts are the best answer, but absent that, certainly the idea of capping enrollment in any sort of school that was operating as a virtual school beforehand, is just absolutely foolish.
Jason Bedrick: So let’s dive into your findings. You break up the survey into four different areas: active learning, communication, pedagogical efficacy and classroom management. So let’s take these one by one. What do you mean by active learning, and what did the survey find?
Ian Kingsbury: Sure. So by active learning, I mean that students are really actively engaged in the learning process. So the opposite of this, if it’s helpful, would be a transmissionist approach, or rote memorization where the teacher is just conveying things to the class for them to memorize or to write down. So active learning involves collaborating with your peers or presenting work to your class, things of that nature. And what we saw was that the students that were in schools affiliated with K12 Inc., were frequently participating in active learning activities.
So, on average, maybe once a week or so, they were discussing topics with their peers, or they were presenting work to their other peers, and by comparison, these were rare in the schools that operated virtually, and we saw a lot of media accounts of this. And this is not a criticism of the teachers necessarily because it was obviously a really difficult situation, but a lot of accounts of teachers handing out packets, and that was sort of the extent of their involvement. And it turns out unsurprisingly, that’s just not a good way for students to be learning, and this type of active learning is really important for keeping students engaged, and important for promoting student achievement.
Jason Bedrick: The second measure you discussed is communication. And obviously, in media reports, there have been just a lot of parents expressing frustration with the lack of communication, or the mixed communications that they are receiving from their schools. What did you find in the survey surrounding communication?
Ian Kingsbury: So, this construct is specifically looking at how effectively teachers communicated, how clear they were in terms of their academic standards, and how accessible they were in terms of communications. And the magnitude of difference between the two groups, once again, is large. So, on average, those parents say that the students in K12 Inc. schools, that the teachers were very clear at communicating, they were doing it frequently. On average, they agree to strongly agree that those things were occurring in the schools, and by comparison, in thinking on the experience of those brick and mortar students, they say that they’re neutral towards slight disagreement on whether those same things are happening. So there’s a lot of frustration in terms of unclear standards, and even being able to communicate with teachers, and being able to text and receiving a timely response.
Jason Bedrick: The third area looked at was pedagogical efficacy, and that’s an area that has received quite a bit of media attention, especially with that McKinsey report that was cited in the New York Times, which said that students who were receiving an average level of online instruction, and here they were primarily talking about online instruction from those brick and mortar schools, because that’s what most of the students were in, would be losing several months of learning this year, and that students who were subject to lower quality, I think one standard deviation lower than the average, would be losing almost up to a year’s worth of instruction. So how did K12 Inc. fare versus the brick and mortar schools that were doing emergency distance learning?
Ian Kingsbury: Sure. Yeah, so this construct of instructional efficacy simply asks, “Did teachers do a good job of delivering content?” And once again, there’s a theme here, obviously. The magnitude of the difference between the two groups is enormous. So to give you an example, there’s a question that says, “Instructional materials worked well for learning in an online or virtual setting.” For those K12, Inc. parents, about 50 percent of them, a little over 50 percent, agreed that the materials worked well for that format of that modality, and that slightly over 50 percent compares to about 5 percent of parents who agree that the materials worked well for their kids who were in brick and mortar schools. So again, this isn’t a criticism of traditional public schools. It’s very hard to do these things on the fly, and that’s the entire point, is that virtual schools, obviously they’ve had years of experience doing these kinds of things, and adapting their practices for that particular format. So it shouldn’t be a surprise that we see something like instructional materials are much higher quality for that format in those schools, but that kind of finding obviously needs to guide policy right now.
Jason Bedrick: The final area you look at is classroom management, which is interesting in and of itself, given that there’s no classroom, right? It’s a virtual classroom. And again, you see all sorts of new stories where parents are frustrated with either too much or too little of this sort of thing, too little in the sense that sometimes kids are just off doing other things, and the teachers don’t notice. In other cases, parents are frustrated that there’s too much control being exercised over their own home environment, where schools are saying, “You need to be in your uniform and you need to have your shoes on your feet, and we don’t want you on a bed, you need to be on a chair.” And some parents think that’s great like, “Yes, they should be in a uniform. I want them feeling like this is school,” others are saying, “Look, this is my own home. My kid’s going to sit where he’s going to sit, and if they don’t want to have shoes on, they shouldn’t have to have shoes.”
So that’s obviously been a challenge with a wide variety of expectations, but how are the families that you surveyed viewing the difference in the ability of teachers in each sector to manage the classroom?
Ian Kingsbury: Yeah, so I think the most extreme example we heard in terms of managing a classroom was instances where a student shared lewd content, or even someone who wasn’t supposed to be in there, somehow was able to access the course and share lewd content. And hopefully, that wasn’t the majority of classrooms, but those anecdotes are emblematic of the reality that managing a virtual classroom actually does take some level of skill, it’s not exceptionally straightforward. And you don’t have one student punching another, but there are still disruptions and bad behavior. So one question, for example, asks whether instructors were skilled at preventing and managing disruptions, and for brick and mortar students, you have fewer than 20 percent of parents who strongly agree or agree that instructors were skilled at preventing and managing disruptions, compared to those K-12 Inc. schools, where it’s about 85 percent of parents who agree or strongly agree that those disruptions were well-managed.
Jason Bedrick: What sort of advice based on the survey findings do you have for school leaders in the brick and mortar sector? That could be public, private, charter, what have you, to prepare for another, if God forbid there’s another type of event like this, or if the COVID-19 or 2021 comes back at some point and forces a shutdown again in the middle of the year, what can they learn from K12 Inc.?
Ian Kingsbury: Well, that’s hard for me to say because I don’t work for K12 Inc. and I don’t have that kind of insight, but the obvious answer is that they could get in touch with them and try to collaborate with them. I think more often than not, schools don’t see each other as competitors, but as well-intentioned folks trying to do the right thing, right? And again, I don’t speak for K12 Inc., but my guess is that a lot of these virtual schools would be willing to lend a hand. They know what they’re doing, this moment obviously doesn’t affect them the way that it affects other schools, and so they just have a really good handle on this and a lot of insight that could be shared. So I do hope that there’s more collaboration between the virtual sector and the brick and mortar sector. Anecdotally, I’ve heard a little bit about this in higher ed, where some institutions have reached out to institutions that have really robust online learning for some guidance. So, that’s the same kind of thing really, hopefully will come to primary and secondary education.
Jason Bedrick: And what about policymakers? What should they be thinking about? How should they internalize the findings of your study when it comes to creating policies that make it easier for schools to provide online instruction, if necessary, or just for parents to access online instruction from existing providers, in the event of a pandemic or event like this?
Ian Kingsbury: The most obvious I think is to not cap enrollment at virtual institutions, and some states have either deliberated this or possibly done this, because there was such strong demand to switch kids over from brick and mortar schools into these types of virtual schools. And again, it’s very hard to argue that that’s in the interest of students, when you look at the data that was collected here. It’s worth pointing out that this was obviously done in the spring, and schools have had an entire summer in which to prepare possibly for online learning, but I would caution against the temptation to say maybe brick and mortar schools have figured it out by now.
For one, the summer is just, three months is not a great deal of time, and second, it’s worth keeping in mind that a lot of schools didn’t provide guidance on how they would be teaching until July or August. In New York City, for example, parents and students have whiplash because they keep going back and changing their opening date, and what their plans are. So really, teachers in schools have not had a tremendous amount of time to improve upon the experience that was delivered in the spring. And so, these types of results are probably going to play out again during this coming academic school year, if schools are closed. So let’s make sure that as many students as possible have access to what appears to be a better option.
Jason Bedrick: My guest today has been Dr. Ian Kingsbury, education policy fellow with the Empire Center for Public Policy. He’s the author of a new EdChoice report titled, Online Learning: How do Brick and Mortar Schools Stack up to Virtual Schools? Ian, thanks so much for joining the podcast.
Ian Kingsbury: Thank you, Jason.
Jason Bedrick: This has been another edition of EdChoice Chats. If you have any ideas for authors you’d like us to interview for the Big Idea series, please send them to media@edchoice.org, and be sure to subscribe to our podcast. Follow us on social media @edchoice, and don’t forget to sign up for our emails on our website, edchoice.org. Thank you, we’ll catch you next time.