Since ancient times, people have feared and suffered at the hands of tax collectors. The ancient writer Lactantius tells us that Roman tax collectors would always double the tax when they were sent to collect it, in order to make sure that their efforts were rewarded. Tax collectors were so pervasive, he says, "there was no threshing-floor without the taxman there, no vintage without a guard on the spot." And the ancient Greek philosopher Philo tells of tax collectors carrying off the wives and children of those who could not pay what they owed. He also tells of one tax collector who tortured those owing taxes "with racks and wheels, so as to kill them with newly invented kinds of death."
Public Opinion. Little wonder many Americans view the Internal Revenue Service as a malevolent force. While there is no evidence of IRS torture to coerce tax payments from recalcitrant taxpayers, there is plenty of evidence of taxpayer abuse. As a result, the IRS is easily the most hated agency of government.
- According to a Techno-Politics/American Viewpoint poll last year, the IRS was cited as the "most annoying" agency of government by 31 percent of respondents, far greater than the second-place finisher, Congress, cited by just 4 percent.
- More recently, a Fox News/Opinion Dynamics poll found that 81 percent of Americans think the IRS has too much power and 57 percent favor abolishing it and building a new tax collection agency from the ground up.
There are those who believe that some of the most egregious IRS abuses are in fact deliberate. The IRS knows it can audit only a small number of taxpayers – last year it examined just 1.63 percent of returns. Therefore, it must depend on intimidation and fear to ensure "voluntary" compliance with the tax laws. Press reports of heavyhanded IRS tactics actually support the IRS's collection efforts. Yet at least 14 percent of all taxes owed still go uncollected, according to the IRS itself.
Increasing Costs of Collection. When confronted about the low collection rate, the IRS response has always been that it needs more money for computers and staff. In the 1980s this led to big increases in the IRS budget, which more than doubled between 1980 and 1990. The agency now spends more than $7 billion a year and employs more than 100,000 people to collect the federal government's revenue.
- While the IRS notes that this is just 49 cents for every $100 collected, it is a significant increase over the 1981 collection cost of 41 cents.
- Further, according to the IRS's own figures, it costs taxpayers $8.67 just to pay $100 in taxes. That is equivalent to an invisible 8.7 percent addition to every tax rate. [ See the figure .]
Unkept Promises. At least as far back as 1948, the IRS was promising Congress an enormous return on new funding for audits. Then-commissioner Joseph Nunan went so far as to promise the House Appropriations Committee that the Treasury would get $20 for each $1 invested. Although the IRS later backed away from the 20-for-1 figure, every commissioner since nevertheless has promised that substantial additional revenues would be forthcoming if only the IRS had more money.
Even Ronald Reagan was seduced by those promises. In 1982 he was told that there was $95 billion of revenue going uncollected simply due to lack of resources. "As we struggle to trim government spending, it's hard not to think of how close that unpaid tax could come to wiping out the deficit," Reagan commented. This led him to support almost-annual increases in the IRS's staff and budget.
- From 1981 to 1988 the total number of IRS employees rose one-third, from 86,156 to 114,873.
- Over the same period, the IRS budget doubled, from $2.5 billion to more than $5 billion.
The IRS budget has continued to rise since. Yet despite the massive increase in resources over the last 15 years, the amount of money the IRS says it is not collecting has only risen.
- In 1982 the IRS estimated the "tax gap" for individuals at $51.9 billion.
- By 1992, the latest year available, uncollected monies had virtually doubled to $95.3 billion.
IRS Abuse: An Old Story. Over the summer, the Senate Finance Committee hearings on Internal Revenue Service abuses made national headlines. The committee heard from both taxpayers and IRS employees, many of whom testified in disguise to avoid retaliation from IRS management. In the words of Newsweek , "They told of an IRS that is a virtual police state within a democracy, a Borgia-like fiefdom of tax terror at the heart of the U.S. economy." Congressional leaders promised quick action on legislation to reform the IRS.
This is not the first time Congress has promised to get control of the agency. The Finance Committee held similar hearings 10 years ago, garnering similar headlines. During those hearings, an IRS agent from Los Angeles testified about a sign on his manager's door reading, "Seizure Fever – Catch It." A Baltimore agent testified that agents were routinely rewarded for having the best collection statistics. Taxpayers told of how their lives were destroyed by IRS excesses. This led to passage of the 1988 Taxpayer Bill of Rights.
The legislation did not solve the problems. Nor did a second Taxpayer Bill of Rights in 1996. The reason is simple. For decades, congresses and administrations have pressured the IRS to collect more revenue. And the IRS always has promised to comply – provided it received more money to audit and investigate recalcitrant taxpayers.
In reality, it is a pipe dream to think that the IRS could ever collect more than a fraction of the tax gap through increased auditing and enforcement. To get anywhere near 100 percent compliance would require such draconian measures the American people would revolt en masse.
Steps to Reform. Congress seems poised to pass another Taxpayer Bill of Rights that would change the burden of proof in tax court cases, award civil damages to taxpayers abused by IRS employees and extend the privilege of confidentiality to accountants and other nonlawyer tax advisers. Such legislation has passed the House of Representatives and is expected to clear the Senate in early 1998.
More needs to be done. An essential prerequisite for improving relations between taxpayers and the IRS is to sharply reduce the burden of federal taxation. Unfortunately, the recently passed Taxpayer Relief Act made only a very modest step in that direction. The federal government currently takes more money out of the pockets of American taxpayers than at any other time in its history – 21.3 percent of GDP in the second quarter of 1997, compared to the historical level of 18 to 19 percent. As long as this is the case, it is going to be harder and harder for the IRS to compel compliance.
It is also essential for Congress to fundamentally reform the tax code. As long as the tax law is so incredibly complex and indecipherable, conflicts between the IRS and taxpayers are inevitable. With tax rates as high as 39.6 percent, taxpayers have a powerful incentive to claim every possible exemption, deduction and exclusion. The remedy is to replace our current tax system with something like a simple flat rate tax on a consumption base. That would make it easier for both the IRS and taxpayers to calculate tax payments, and do so in a way that is both fairer and more conducive to economic growth.
The urge to reform the IRS is strong. But unless the tax code also is reformed – and the tax burden reduced – problems will remain.
This Brief Analysis was prepared by NCPA Senior Fellow Bruce Bartlett