Even if you haven’t read the whole thing, I will draw your attention to this passage:
But what is government itself, but the greatest of all reflections on human nature? If men were angels, no government would be necessary. If angels were to govern men, neither external nor internal controls on government would be necessary. In framing a government which is to be administered by men over men, the great difficulty lies in this: you must first enable the government to control the governed; and in the next place oblige it to control itself. A dependence on the people is, no doubt, the primary control on the government; but experience has taught mankind the necessity of auxiliary precautions.
For a dead guy, James Madison knew an awful lot about the twenty-first century. Read a description of how the IRS seizes assets of innocent Americans with secret warrants, drains bank accounts, and acts in a way that would be criminal if it were done by a private citizen (even in the process of enforcement of a valid debt).
Due process protections ought to prevent this; and once such abuses occur, criminal sanctions (akin to those imposed upon private citizens) ought to be used. But we live in a society with neither a front-end control on government abuses nor a back-end system of punishment and deterrence.