Factor grew up in a housing project in Bridgeport, Connecticut. He graduated from Wesleyan University in Middletown, Connecticut.
Factor's first job after graduating was serving as a supervisor of management consulting services at Coopers & Lybrand.
Factor is the John C. West Professor of International Politics and American Government at The Citadel, The Military College of South Carolina in Charleston, South Carolina. He held adjunct professorships at the School of Continuing and Professional Studies at New York University from 1992 to 1996, and at the Graduate School of Management and Urban Professions at the New School for Social Research from 1985 to 1992.
Factor joined the FOX News Channel in 2013 and currently serves as a contributor.
He was Chairman of the New York Public Asset Fund from 2002 to 2006 and Vice-Chairman of Governor’s Island Preservation and Education Corporation from 2006 to 2007. Factor was a member of the Banking Board of the New York State Banking Department from 2001 to 2007. From 1987 to 1988, he was a member of the Federal Savings and Loan Advisory Council for the Federal Home Loan Bank. He has been an underwriting member of Lloyd's of London since 1987.
In addition to his books, Factor has written widely on economic and financial issues for publications including the Wall Street Journal, Christian Science Monitor, National Review and newspapers nationwide and is also a Fox News and Forbes magazine contributor.
He was president of Mallory Factor, Inc., an independent merchant bank and financial relations consultancy that he founded in 1976.
Factor is the host and co-founder of The New York Meeting and the host of The Charleston Meeting, both nationally-recognized gatherings of elected officials, journalists, business leaders and authors in New York City and Charleston, South Carolina. He was previously the co-founder and co-chair of The Monday Meeting (with James E. Higgins).
He co-founded and was the chairman of the Free Enterprise Fund, a free market “do” tank advocating economic growth, lower taxes and limited government.
He is a member of the Council on Foreign Relations and recently served as Vice-Chair of the Council on Foreign Relations Task Force on Terrorism Financing. He has testified in front of the U.S. House of Representatives and the United States Senate on terror financing, regulation of the financial services industry and other economic issues. Factor chaired the Economic Roundtable for the Chairman of the Joint Chiefs of Staff and led the 2009 Economic Summit for the U.S. House Republican Conference and Policy Committee in Washington, D.C.
Factor has served on the Tony Awards Administration Committee since 2003, served on the Tony Awards Management Committee from 1999 to 2003, and served as a Tony Awards nominator from 1996 to 1999. He also served as a member of the US Air Force's Air Mobility Command Civic Leader Group. Factor served as a Senior Fellow for the Republican Policy Committee Chairman of the United States House of Representatives from 2007 to 2011.