Born and brought up in Hamburg, Germany, he was the eldest son and second of six children of an affluent Jewish-German citizen of Hamburg. His younger siblings included Otto Beit. Alfred Beit was an unpromising scholar and was apprenticed to Jules Porgès & Cie, the Amsterdam diamond firm where he developed a talent for examining stones.
Beit made his first fortune in property speculation. Responding to a demand for business premises, he bought a piece of land and built twelve corrugated iron sheds for offices and rented eleven out monthly and kept one for himself. Twelve years later he sold the land for a considerable profit.
Beit was sent to Kimberley, South Africa in 1875 by his firm to buy diamonds—following the diamond strike at Kimberley. He became a business friend of Cecil Rhodes through his role in the Kimberley Central Company. Beit was captivated by Rhodes' talk of 'big schemes'. Together, they proceeded to buy out digging ventures and to eliminate opposition such as Barney Barnato. He rapidly became one of a group of financiers who gained control of the diamond-mining claims in the Central, Dutoitspan, and De Beers mines. Rhodes was the active politician and Beit provided a lot of the planning and financial backing.
Beit's diamond interests were mainly concentrated on Kimberley mine. He focused his main attention on the Kimberley Central Company aiming to expand its interests. He had a major role in the rise of Kimberley Central Company.
In 1886 Beit extended his interests to the newly discovered goldfields of the Witwatersrand and met with great success. In his business ventures there he made use of financiers Hermann Eckstein and Sir Joseph Robinson. He founded the Robertson Syndicate and the firm of Wernher, Beit & Co. He imported mining engineers from the USA and was among the first to adopt deep-level mining. Rhodes purportedly was granted concessions by Lobengula, as a result of which Beit founded the British South Africa Company in 1888.
Beit became life-governor of De Beers and also a director of numerous other companies such as Rand Mines, Rhodesia Railways and the Beira Railway Company.
In 1888 Beit moved to London when he felt he was better able to manage his financial empire and support Rhodes in his Southern African ambitions. Beit moved into Tewin Water, Tewin, near Welwyn, a large Regency house with Victorian additions and 7,000 acres (28 km²), and a few miles away Julius Wernher bought Luton Hoo, with 5,218 acres (21.1 km²). In the 1890s, he had a mansion built in Park Lane – Aldford House.
Inspired by Rhodes' imperialist vision, he took part in the planning and financing of the unsuccessful Jameson Raid of late 1895 which was intended to trigger a coup in the South African Republic in the Transvaal. As a result of this debacle, Rhodes resigned as Prime Minister, and both he and Beit were found guilty by the House of Commons inquiry. Beit was obliged to resign as director of the Charter Company, but was elected vice-president of the British South Africa Company a few years later. With the death of Rhodes in 1902, Beit, as one of the trustees, helped control the enormous estate, currently being helped by the Oppenheimer family of De Beers and Anglo-American.
Beit never married and had no children. He died at Tewin Water on 16 July 1906 after seeing a rapid deterioration in his health. He left an estate of £8,049,886 (equivalent to £4.88 billion in 2016).
During his lifetime, Beit made generous donations for scientific work and education. In 1905 he founded a chair of colonial history at the University of Oxford, which is now the Beit Professor of History of the British Commonwealth. In 1906 he made the donation of two million mark to the stock capital of the Hamburgische Wissenschaftliche Stiftung, a charity dedicated to spend its interest for the benefit of a precursor of the University of Hamburg.
In his will he set up the Beit Trust through which he bequeathed large sums of money (£1,200,000) for infrastructure development in the former Northern and Southern Rhodesia, later modified to university education and research in Zimbabwe, Zambia and Malawi.
Significant infrastructure projects financed by the Trust include the Birchenough Bridge in the former Southern Rhodesia, named after Sir Henry Birchenough, chairman of the Beit Trust from 1931 until 1937 and whose ashes are buried beneath the structure of the bridge. Ralph Freeman, the bridge's designer, was also the structural designer on the Sydney Harbour Bridge and consequently the two bridges bear a close resemblance, although Birchenough is only two-thirds as long as the Australian bridge. It was built by Dorman Long and completed in 1935. At a length of 1,080 feet (329 m) it was the third longest single-arch suspension bridge in the world at the time.
In recognition of his bequests the Royal School of Mines, a faculty of Imperial College London, erected a large memorial to Beit flanking the entrance to its building. The Imperial College residential halls on Prince Consort Road was named Beit Hall after him.