The issue of the firing of people within the Executive branch for political purposes came up during a debate in 1789 about how to create agencies within the Executive branch that would be consistent with Article II, Section 2 of the US Constitution, which says that the President can appoint people (like US Attorneys/prosecutors), but they couldn't take office unless the Senate votes to confirm each individual appointment:
He [the President] shall have Power, by and with the Advice and Consent of the Senate, to make Treaties, provided two thirds of the Senators present concur; and he shall nominate, and by and with the Advice and Consent of the Senate, shall appoint Ambassadors, other public Ministers and Consuls, Judges of the supreme Court, and all other Officers of the United States, whose Appointments are not herein otherwise provided for, and which shall be established by Law: but the Congress may by Law vest the Appointment of such inferior Officers, as they think proper, in the President alone, in the Courts of Law, or in the Heads of Departments.
Madison's logic was straightforward, and came about in one of his first major speeches before the House of Representatives, on 17 June 1789, just a few months into his first term as a Congressman (he would later become Secretary of State and President). A bill was put forward to create what is today known as the State Department in a more formal fashion than had existed when George Washington had become the new nation's first President just three months before and appointed Thomas Jefferson as his Secretary of State Affairs.
As with other agencies brought into law by Congress, this new Department of Foreign Affairs being debated would have to exist under the oversight and supervision of the Executive Branch of government, led by the President. And, as such, how, Congress wanted to know, could they make sure that no President would ever allow good members of his various departments ("meritorious officers") to be fired for purely political purposes ("wanton removal")?
The Congressional Register from that day lays out Madison's entire speech. Because it's thorough and detailed, and offers a brilliant insight into the thinking of the most important of the Framers of our Constitution (Madison), I reproduce it in its entirety below. Because it's rather long, however, I've also bolded and italicized those parts that get right to the nub of the matter so you can skim it first, and then go back and read the entire thing in context.
The essence of the debate is over whether Congress or the President would have the power to fire people employed below the level of a Cabinet officer in the George Washington and future administrations. The conclusion of the majority - and thus the way the law is today - is that Congress felt that the Senate's approval of the hiring of federal officers was critical (a power the Patriot Act took away, and the Senate voted last week to restore), and that when it comes to firing them, the President has the power. However, if the President were to abuse that power to fire federal officials through the "wanton removal of meritorious officers," he should be immediately impeached.
This was particularly relevant since, in 1789, there were no federal crimes that had yet been defined. So when the Constitution said that a President could be impeached for "High Crimes and Misdemeanors" there were none specified at the time. (The first federal crime was specified in 1790.) So Congress, at this point, was in the process of both creating new executive offices and of defining impeachable "crimes." They were establishing precedents, and this was a grave matter. It would echo forward for centuries.
The Congressional Register 17 June 1789
The house went into a committee of the whole on the bill for establishing the department of foreign affairs, and resumed the consideration of the clause "to be removable by the president."
Mr. Madison [is called upon to speak].
However various the opinions which exist upon the point now before us, it seems agreed on all sides, that it demands a careful investigation and full discussion. I feel the importance of the question, and know that our decision will involve the decision of all similar cases. The decision that is at this time made will become the permanent exposition of the constitution; and on a permanent exposition of the constitution will depend the genius and character of the whole government. It will depend, perhaps, on this decision, whether the government shall retain that equilibrium which the constitution intended, or take a direction toward aristocracy, or anarchy among the members of the government. Hence how careful ought we to be to give a true direction to a power so critically circumstanced. It is incumbent on us to weigh with particular attention the arguments which have been advanced in support of the various opinions with cautious deliberation.
I own to you, Mr. chairman, that I feel great anxiety upon this question; I feel an anxiety, because I am called upon to give a decision in a case that may affect the fundamental principles of the government under which we act, and liberty itself. But all that I can do on such an occasion is to weigh well every thing advanced on both sides, with the purest desire to find out the true meaning of the constitution, and to be guided by that, and an attachment to the true spirit of liberty, whose influence I believe strongly predominates here.
Several constructions have been put upon the constitution relative to the point in question. The gentleman from Connecticut (Mr. Sherman) has advanced a doctrine which was not touched upon before. He seems to think (if I understood him right), that the power of displacing from office is subject to legislative discretion; because it having a right to create, it may limit or modify as is thought proper. I shall not say but at first view this doctrine may seem to have some plausibility: But when I consider, that the constitution clearly intended to maintain a marked distinction between the legislative, executive, and judicial powers of government; and when I consider, that if the legislature has a power, such as contended for, they may subject, and transfer at discretion, powers from one department of government to another; they may, on that principle, exclude the president altogether from exercising any authority in the removal of officers; they may give it to the senate alone, or the president and senate combined; they may vest it in the whole congress, or they may reserve it to be exercised by this house. When I consider the consequences of this doctrine, and compare them with the true principles of the constitution, I own that I cannot subscribe to it.
Another doctrine which has found very respectable friends, has been particularly advocated by the gentleman from South-Carolina (Mr. Smith). It is this; when an officer is appointed by the president and senate, he can only be displaced from malfeasance in his office by impeachment: I think this would give a stability to the executive department so far as it may be described by the heads of departments, which is more incompatible with the genius of republican governments in general, and this constitution in particular, than any doctrine which has yet been proposed. The danger to liberty, the danger of mal-administration has not yet been found to lay so much in the facility of introducing improper persons into office, as in the difficulty of displacing those who are unworthy of the public trust. If it is said that an officer once appointed shall not be displaced without the formality required by impeachment, I shall be glad to know what security we have for the faithful administration of the government. Every individual in the long chain which extends from the highest to the lowest link of the executive magistracy, would find a security in his situation which would relax his fidelity and promptitude in the discharge of his duty.
The doctrine, however, which seems to stand most in opposition to the principles I contend for, is that the power to annul an appointment is in the nature of things incidental to the power which makes the appointment [e.g. the President]. I agree that if nothing more was said in the constitution than that the president, by and with the advice and consent of the senate, should appoint to office, there would be great force in saying that the power of removal resulted by a natural implication from the power of appointing. But there is another part of the constitution no less explicit than the one on which the gentleman's doctrine is founded, it is that part which declares, that the executive power shall be vested in a president of the United States. The association of the senate with the president in exercising that particular function, is an exception to this general rule; and exceptions to general rules, I conceive, are ever to be taken strictly.
But there is another part of the constitution which inclines in my judgment, to favor the construction I put upon it; the president is required to take care that the laws be faithfully executed. If the duty to see the laws faithfully executed be required at the hands of the executive magistrate, it would seem that it was generally intended he should have that species of power which is necessary to accomplish that end. Now if the officer when once appointed, is not to depend upon the president for his official existence, but upon a distinct body (for where there are two negatives required either can prevent the removal), I confess I do not see how the president can take care that the laws be faithfully executed. It is true by a circuitous operation, he may obtain an impeachment, and even without this it is possible he may obtain the concurrence of the senate for the purpose of displacing an officer; but would this give that species of control to the executive magistrate which seems to be required by the constitution? I own if my opinion was not contrary to that entertained by what I suppose to be the minority on this question, I should be doubtful of being mistaken, when I discovered how inconsistent that construction would make the constitution with itself. I can hardly bring myself to imagine the wisdom of the convention who framed the constitution, contemplated such incongruity.
There is another maxim which ought to direct us in expounding the constitution, and is of great importance. It is laid down in most of the constitutions or bills of rights in the republics of America, it is to be found in the political writings of the most celebrated civilians, and is every where held as essential to the preservation of liberty, That the three great departments of government be kept separate and distinct; and if in any case they are blended, it is in order to admit a partial qualification in order more effectually to guard against an entire consolidation. I think, therefore, when we review the several parts of this constitution, when it says that the legislative powers shall be vested in a Congress of the United States under certain exceptions, and the executive power vested in the president with certain exceptions, we must suppose they were intended to be kept separate in all cases in which they are not blended, and ought consequently to expound the constitution so as to blend them as little as possible.
Every thing relative to the merits of the question as distinguished from a constitutional question, seems to turn on the danger of such a power vested in the president alone. But when I consider the checks under which he lies in the exercise of this power, I own to you I feel no apprehensions but what arise from the dangers incidental to the power itself; for dangers will be incidental to it, vest it where you please. I will not reiterate what was said before with respect to the mode of election, and the extreme improbability that any citizen will be selected from the mass of citizens who is not highly distinguished by his abilities and worth; in this alone we have no small security for the faithful exercise of this power. But, throwing that out of the question, let us consider the restraints he will feel after he is placed in that elevated station. It is to be remarked that the power in this case will not consist so much in continuing a bad man in office, as in the danger of displacing a good one.
Perhaps the great danger, as has been observed, of abuse in the executive power, lies in the improper continuance of bad men in office. But the power we contend for will not enable him to do this; for if an unworthy man be continued in office by an unworthy president, the house of representatives can at any time impeach him, and the senate can remove him, whether the president chuses or not.
The danger then consists merely in this, the president can displace from office a man whose merits require that he should be continued in it. What will be the motives which the president can feel for such abuse of his power, and the restraints that operate to prevent it? In the first place, he will be impeachable by this house, before the senate, for such an act of mal-administration; for I contend that the wanton removal of meritorious officers would subject him to impeachment and removal from his own high trust.
But what can be his motives for displacing a worthy man? It must be that he may fill the place with an unworthy creature of his own. Can he accomplish this end? No; he can place no man in the vacancy whom the senate shall not approve; and if he could fill the vacancy with the man he might chuse, I am sure he would have little inducement to make an improper removal.
Let us consider the consequences. The injured man will be supported by the popular opinion; the community will take side with him against the president; it will facilitate those combinations, and give success to those exertions which will be pursued to prevent his re-election. To displace a man of high merit, and who from his station may be supposed a man of extensive influence, are considerations which will excite serious reflections before hand in the mind of any man who may fill the presidential chair, the friends of those individuals, and the public sympathy will be against him.
If this should not produce his impeachment before the senate, it will amount to an impeachment before the community, who will have the power of punishment by refusing to re-elect him. But suppose this persecuted individual, cannot obtain revenge in this mode, there are other modes in which he could make the situation of the president very inconvenient, if you suppose him resolutely bent on executing the dictates of resentment. If he had not influence enough to direct the vengeance of the whole community, he may probably be able to obtain an appointment in one or other branch of the legislature; and being a man of weight, talents and influence in either case, he may prove to the president troublesome indeed.
We have seen examples in the history of other nations, which justifies the remark I now have made, though the prerogatives of the British king are great as his rank, and it is unquestionably known that he has a positive influence over both branches of the legislative body, yet there have been examples in which the appointment and removal of ministers has been found to be dictated by one or other of those branches.
Now if this is the case with an hereditary monarch, possessed of those high prerogatives and furnished with so many means of influence; can we suppose a president elected for four years only dependent upon the popular voice impeachable by the legislature? little if at all distinguished for wealth, personal talents, or influence from the head of the department himself; I say, will he bid defiance to all these considerations, and wantonly dismiss a meritorious and virtuous officer? Such abuse of power exceeds my conception: If any thing takes place in the ordinary course of business of this kind, my imagination cannot extend to it on any rational principle.
But let us not consider the question on one side only, there are dangers to be contemplated on the other. Vest this power in the senate jointly with the president, and you abolish at once that great principle of unity and responsibility in the executive department, which was intended for the security of liberty and the public good. If the president should possess alone the power of removal from office, those who are employed in the execution of the law will be in their proper situation, and the chain of dependence be preserved; the lowest officers, the middle grade, and the highest, will depend, as they ought, on the president, and the president on the community. The chain of dependence therefore terminates in the supreme body, namely, in the people; who will possess besides, in aid of their original power, the decisive engine of impeachment.
Shortly thereafter, Mr. Madison yielded the floor to Mr. Elbridge Gerry of Massachusetts, who argued against impeaching a President for firing an honorable man. "It is said that the president will be subject to an impeachment for dismissing a good man," Gerry noted. "This in my mind involves an absurdity."
Gerry then went on to build a case that no President would do such a thing, because his new appointment would also be subject to confirmation by the Senate. When the Senate considered the new appointment, it would be able to ask what had happened to the last person, thus bringing accountability of the President into the picture.
Mr. Gerry, of course, had not foreseen a day when a certain Senator Specter would be forced by ideologues in the White House to hire a right-wing operative. He never imagined such an operative could then slip into law over Senator Specter's name (but, according to the good Senator, without his knowledge) a change in the Constitutional requirement of Senate confirmation of presidential appointments to the position of federal prosecutors known as US Attorneys. And Congressman Gerry (after whom Gerrymandering would later be named - no saint himself) would, no doubt, have been boggled at the idea that that same operative would then would himself end up as a federal prosecutor.
Madison was prescient. The remedy for this High Crime against American democracy is impeachment.