Is Mayonnaise an Educational Expense?

Four lessons I learned studying Arizona’s ESA transactions

“Numbers don’t lie,” it’s said. To me, that’s just because they can’t talk at all.

I’m here to share some stories and lessons from my effort to calculate a pretty important number in school choice world: the rate of misspending in Arizona’s ESA program.

I’ll start with a little background for those unaware of what’s been happening in Arizona. Arizona’s ESA, the oldest in the country and the second to be made available for all K-12 students in the state, has been receiving a lot of attention recently. A local NBC station did some reporting on a risk-based audit of the program and made some pretty sweeping claims about the whole program based on its sample. The Arizona Department of Education (ADE) responded with a new self-audit of its own with a more representative sample.

We at EdChoice had already started working on an analysis of our own when ADE released its report, and after its release we still thought it would be beneficial to provide a more comprehensive picture than what either previous report could do alone. We released that brief at the beginning of April. You can read it on the EdChoice research library, and you can read a summary on our blog.

I’m not here to rehash everything from that effort. I’m not talking about why the problem is as small or large as it is. I’m not here to explain why our numbers are credible. Rather, I want to share some things I learned that helping put these numbers together.

Here’s what I mean.

Lesson #1: Subjectivity is unavoidable

It’s called an education savings account for a reason. Everyone understands ESA money is supposed to go toward educational expenses, although people may disagree on what counts as educational (LEGO sets come to mind).

Like other states, Arizona dedicates several pages in its ESA Parent Handbook to listing examples of allowable and unallowable expenses. The more items that fit within these listed categories, the more objective administrators can be when discerning whether an expense is educational or not.

Because we wanted to be as objective as possible when determining an unallowable spending rate, we relied heavily on ADE’s allowable/unallowable item lists. It didn’t matter if we thought something was probably educational or not—if ADE said an item was allowable or unallowable in the parent handbook, that’s how we judged it.

What I didn’t fully appreciate before working on this brief was how many items sit in some kind of gray area between these categories, despite everything ADE does list out.

Let me give you an example.

In the ESA Parent Handbook, ADE allows “audio/visual players.” Fair enough, I think. But then I flip to the unallowable item list, and I see “audio/visual equipment.”

That’s all I have to go on. Now, it is up to me—and the ADE administrators—to decide what is a “player” and what is “equipment.” Sure, headphones may be fine, particularly if you are in an online class. But how much do you pay for better sound quality before it becomes “audio equipment?” A portable CD-ROM player seems sensible, as most laptops do not come with them built in anymore. You need one if you want to play educational DVDs for a course. Does a sub-$100 projector fit the bill as well?

You may have impressions one way or another in response to this example, but you can see how you are required to use some judgment to make an allow-or-not binary choice.

It’s a bit of a problem for both ESA supporters and detractors. The more subjective administration is, the more likely it is for some random bureaucrat’s personal persuasions—with all due respect to those just doing their job to the best of their abilities—to sway what they allow or don’t allow. It also increases the likelihood of inconsistency, when parents’ choices might be approved or unapproved depending on which person is reviewing their account on a given day.

It’s also a problem for me as someone trying to analyze with integrity. There are a lot of cases where I could understand how a freedom-first mind could find our judgments too strict or a choice skeptic could find us too lenient.

Frankly, in Arizona’s case, I think ADE can do a lot more to clarify what parents, and the taxpaying public, should expect out of the program.

Lesson #2 Transactions matter way less than spending

We had to mark “Bounty Select-A-Size Paper Towels” as an unallowable expense. ADE clearly labels “Household cleaning supplies” as an unallowable expense category. It cost a little over $3 for a roll.

We also had to mark an above-ground pool as an unallowable expense. ADE explicitly prohibits pool purchases as well. It cost about $1,100.

Both items equally affect the program’s unallowable transaction rate. If we hone in on identifying the number of accounts making unallowable purchases, as the news outlet originally did, $3 on paper towels is the same offense as $1,100 on a pool.

And that’s why I question how helpful an unallowable transaction rate really is.

We care about catching unallowable purchases because we care about responsible stewardship of taxpayer dollars. Elected representatives decided this program would serve educational purposes, and when I send my money to the government, I would like to know that the money is going to what the democratic process decided it should go toward.

That’s why not even the harshest ESA cynic is talking about paper towels, and why they’ll instead sort highest expenses to the top of the data file and write from there. It is the egregious examples that offend us the most.

That’s why I’m advocating people focus on the unallowable spending rate instead. Instead of focusing on the share of transactions that are unallowable, find the percentage of dollars spent that go to unallowable items. I think that gives you a much better picture of how well or poorly the government is allowing educational funds to be spent.

Lesson #3 Some unallowable items are allowable for public schools

Speaking of paper towels and pools, it’s worth mentioning that both items have been lawfully purchased by Arizona school districts. Janitorial services, including cleaning supplies, are something every school must have a budget for, and a number of districts pay for swimming pools to be used by high school swim teams.

I can bring up a number of other examples. My personal favorite was a very, very large jar of mayonnaise. For an ESA student, this is not allowed because a jar of mayonnaise is food. However, “food service” is its own public school spending category, and I have to imagine cafeterias have bought gallons and gallons of condiments in the last year.

ESA parents are not allowed to buy first aid kits because they cannot buy medical supplies with ESA funds. But public schools purchase a variety of medical supplies to comply with Arizona law, and nurses are among the most commonly employed support staff.

Of course, there are different economies of scale between ESA families and public schools, so the contexts behind some of these purchases are rather different. So I’m not necessarily saying everything a public school buys should be allowable for an ESA family. But the shock factor of whatever unallowable spending statistic you see gets tuned down quite a bit when you see all the items that would have been seen as perfectly fine if they were made by a public school.

Lesson #4: “Fraud” is fearmongering

Your perception of what is generally happening in the program will be shaped heavily by how you sort your data. If you sort your data by placing the most expensive items highest, or if you CTRL+F search the purchases most worrisome to you, you’ll get concerned very quickly. If, instead, you keep your data randomly sorted and sift through purchases one by one, you will get bored very quickly. Even the unallowable expenses will get boring—things like paper towels show up way more than things like pools.

By the time you get to your two thousandth transaction, the “fraud” narrative around ESA families will feel ridiculous. So many unallowable transactions are generally understandable confusions about what ADE permits.

I want to quote a local Indianapolis reporter I respect, who said it well:

Let’s be clear — errors aren’t necessarily synonymous with fraud. The fraud narrative in public assistance programs is growing and misleading at times. Of course, there is fraud in all programs in life. There will always be a person out there trying to game a system or breaking laws. But the majority of errors are misinterpretations of a rule, data entry mistakes or issues with missing paperwork.

Actually, this reporter said those things about SNAP and Medicaid. This wasn’t written as an apologetic for school choice at all.

But I’m fully with her. I think we should be consistent. For decades, cynics have used extremely rare cases of misspending to demonize entire groups of people using public assistance. Beginning in the 1970s, certain politicians began circulating an image of a Cadillac-driving welfare recipient to foster opposition to public programs. When a stat about the “error rate” in a welfare program was circulated, that was the image you were supposed to have in your mind. Now, in Arizona, cynics have paired an unallowable transaction rate with pictures of the most insane parent decisions in the 1.9 million transactions in the program.

I’m not saying weird purchases don’t exist, and I think much more can be done to minimize those cases. I am saying that if you hate how the “welfare queen” stereotype conflated fixable administrative problems with evil fraud and misunderstanding with malice, you should be skeptical of the narrative opponents are building around unallowable transaction rates.

Conclusion

That mayonnaise thing in the title was not a random joke. Someone did buy a four-gallon pail of mayonnaise. And yes, we marked it as not allowable.

For me, it’s a perfect microcosm of this whole discussion. Even when you do find unallowable items, they are so much more likely to fall on the mundane or quirky side than they are genuinely concerning, much less “fraudulent.”

Our brief is just one chapter in what is sure to be an ongoing evidence war, with supporters and detractors arguing they have the “real” statistics. As someone who put a lot of time and effort into calculating as credible a stat as possible, I’d encourage you to really consider what those numbers are representing.

This was originally published to our Substack.