Knopf was born into a Jewish family in New York City. His father Samuel Knopf was an advertising executive and financial consultant, his mother was Ida Japhe, and his brother Edwin H. Knopf, who worked for Alfred briefly, then became a film director and producer. Alfred attended Columbia University, where he was a pre-law student and a member of the Peithologian Society (a debating and literary club) and the Boar's Head Society. He began to show an interest in publishing during his senior year, becoming advertising manager of an undergraduate magazine. His interest in publishing was allegedly fostered by a correspondence with British author John Galsworthy. After visiting Galsworthy in England, Knopf gave up his plans for a law career, and upon his return went into publishing.
After receiving his B.A. in 1912, Knopf worked as a clerk at Doubleday (1912–1913), then as an editorial assistant to Mitchell Kennerley (1914). He founded his own publishing house in 1915. The company initially emphasized European, especially Russian, literature, hence the choice of the borzoi as a colophon. At that time European literature was largely neglected by American publishers; Knopf published authors such as Simone de Beauvoir, Albert Camus, Joseph Conrad, E. M. Forster, Sigmund Freud, André Gide, Franz Kafka, D. H. Lawrence, Thomas Mann, W. Somerset Maugham, T.F Powys, Wyndham Lewis and Jean-Paul Sartre.
Knopf also published many American authors, including Conrad Aiken, James Baldwin, James M. Cain, Theodore Dreiser, Shirley Ann Grau, Dashiell Hammett, Langston Hughes, Vachel Lindsay, H.L. Mencken, George Jean Nathan, John Updike, and Knopf's own favorite, Willa Cather. From 1924 to 1934, he published the famous literary magazine founded by Mencken and Nathan, The American Mercury. He often developed a personal friendship with his authors. Knopf's personal interest in the fields of history, sociology, and science led to close friendships in the academic community with such noted historians as Richard Hofstadter, Arthur Schlesinger, Jr., and Samuel Eliot Morison. A prominent Republican until Watergate, Knopf often drew legislators into lengthy correspondence by mail. He was also a member of the Peabody Awards Board of Jurors from 1940 to 1946.
Knopf himself was also an author. His writings include Some Random Recollections, Publishing Then and Now, Portrait of a Publisher, Blanche W. Knopf: July 30, 1894-June 4, 1966, and Sixty Photographs.
When the Knopfs' son Alfred A. Knopf, Jr. left the company in 1959 to found Atheneum Publishers, Alfred and Blanche became concerned about the eventual fate of their publishing house, which had always been a family business. The problem was solved in 1960, when Knopf merged with Random House, which was owned by the Knopfs' close friends Bennett Cerf and Donald Klopfer. Knopf retained complete editorial control for five years, and then gave up only his right to veto other editors' manuscript selections. The editorial departments of the two companies remain separate, and Knopf, Inc., retains its distinctive character. Knopf called the merger "a perfect marriage."
Random House itself eventually became a division of Bertelsmann AG, a large multinational media company. The Knopf imprint remains in existence.
Blanche Knopf died in June 1966. Alfred remarried in April of the following year, to Helen Norcross Hedrick. He died of congestive heart failure on August 11, 1984, at his estate in Purchase, New York.
Knopf had little enthusiasm for most of the changes that took place in the publishing industry during his lifetime. "Too many books are published, and they are overpriced," he told The Saturday Review. These are things "about which all publishers agree, and about which no publisher does anything." The most fundamental change he noted was the increased importance of the editor. "In the early days, things were quite simple. The books came in; we published them as written... A publisher was regarded—and so, in turn, was the writer—as a pro. A writer's job was to write a book and give it to you." And he remarked to Shenker: "I guess business became more complicated and publishers less literate. It ceased to be the fact that publishers publish and authors write. Today authors submit manuscripts and editors write books." The editor is now hired largely to acquire books, "and if he can't get good books, he usually takes what he can get--books that are not so good. And then he sometimes wrecks himself trying to make a silk purse out of what can never become anything but a sow's ear."
Knopf was generally unimpressed with current literature, though he admired John Hersey, John Updike, Jorge Amado, and a few other contemporary authors. In Publishing Then and Now he wrote: "Frequently... our American author, whatever his age, experience in life, and technical knowledge, simply can't write. I don't mean that he is not the master of a prose style of elegance and distinction; I mean that he can't write simple straightforward and correct English. And here, only an exceptional editor will really help him." American authors are not very durable, he said in 1964, and "there are no giants in Europe now." And though twelve Knopf authors had won Nobel Prizes, Knopf acknowledged that "some Nobel Prize books aren't very good," calling Doctor Zhivago, for example, "incredibly tedious... If Krushchev had banned it for dullness instead of its political implications, he might have been in the clear."
Among other authors he rejected were Sylvia Plath, Jack Kerouac, Anne Frank, George Orwell, Jorge Luis Borges, Vladimir Nabokov, Isaac Bashevis Singer and Anaïs Nin. He turned down an early novel by Ursula K. Le Guin but encouraged her to keep writing.
Knopf also lamented the "shockingly bad taste" that he felt characterizes much modern fiction, and warned of the danger of a "legal backlash" against pornography, a possible revival of censorship.
This outspoken aspect of his character sometimes found voice in letters of complaint to hotels, restaurants, and stores that failed to meet his high standards. These letters grew increasingly frequent and more severe as he aged. One striking example is the six-year-long war of words he waged against the Eastman Kodak Company over a roll of lost film.