Gilbert was born in Zanesville, Ohio. He graduated from West Point in the famed Class of 1846, finishing 21st out of 59 students. His classmates included twenty future Civil War generals, including George B. McClellan, Stonewall Jackson, George Stoneman, Darius N. Couch, and George Pickett. During the Mexican-American War he served in the 3rd U.S. Infantry and 1st U.S. Infantry and fought at Veracruz and Mexico City. Then he served in Texas for two years. Returning to West Point in 1850 as the Assistant Professor of Geography, History and Ethics he was promoted to first lieutenant on June 10, 1850. In 1855, now a captain in the 1st Infantry, he returned to Texas for frontier duty and Indian fighting. He was also the first member of the Beta Theta Pi Fraternity chapter at Ohio University (Beta Kappa).
When the Civil War started, Gilbert commanded the post of Fort Cobb, Oklahoma. Gilbert rejoined the regiment and fought in the Battle of Wilson's Creek, where he was wounded. He was appointed Inspector General in the Army of the Ohio during the Battle of Shiloh and the Siege of Corinth.
During the Confederate Heartland Offensive, Maj. Gen. William "Bull" Nelson was wounded in the battle of Richmond and his Army of Kentucky severely mauled. Department commander Horatio G. Wright needed to select a replacement for the wounded Nelson and the two ranking officers, Generals Charles Cruft and James S. Jackson, refused the promotion. Therefore, at the recommendation of both Cruft and Jackson, Wright promoted Gilbert to fill the vacancy, and Gilbert was elevated to acting major general pending the approval of the president. Several days later, on September 9, President Abraham Lincoln promoted Gilbert to brigadier general of U.S. Volunteers. Wright's acting promotion to Major General, however, gave Maj. Gen. Don Carlos Buell enough leverage to appoint Gilbert to a corps command in his Army of the Ohio over such generals as Jeremiah T. Boyle, Jefferson C. Davis, and Albin F. Schoepf. Buell later denied knowing Gilbert had not actually received an official appointment.
Proudly wearing two stars in his shoulders, Gilbert was temporarily placed in command of the corps-sized Army of Kentucky in the absence of Nelson. About the time Nelson was well enough to resume command he was murdered in Louisville. When the Army of Kentucky was assimilated into the Army of the Ohio it was redesignated as its III Provisional Cors, and Gilbert retained the command. A week later, Gilbert was engaged in the Battle of Perryville. His troops were successful in checking the last of the Confederate attacks and driving a Confederate brigade back through Perryville, but Gilbert was criticized for his slow action in battle and he was widely despised by the men in his corps for his actions as a martinet.
Gilbert's appointment to brigadier general was not confirmed by the Senate and it expired on March 4, 1863. Some officers in the Army, including Buell's Chief of Staff James B. Fry, were surprised to find out Gilbert had not officially been promoted to major general. Leaving the Volunteer service and returning to the regular army he was appointed Major in the 19th U.S. Infantry. He also was brevetted as lieutenant colonel and colonel.
Maj. Gen. William S. Rosecrans took command of the Army of the Ohio and the subsequent reorganization as the Army of the Cumberland left Gilbert without a permanent command. He commanded the 10th (Provisional) Division in Tennessee at the Battle of Harpeth River. In July 1863 he was assigned to the East on administrative duty, serving as Assistant Provost Marshal General and Chief Mustering Officer in Philadelphia, Pennsylvania and later Hartford, Connecticut, for the remainder of the war.
Gilbert served on the frontier serving in the 28th U.S. Infantry and 7th U.S. Infantry Regiments. When he retired in 1886 he was the Commanding Officer of the 17th U.S. Infantry with the rank of colonel. He died in Baltimore, Maryland on January 17, 1903; and was buried in Cave Hill Cemetery in Louisville, Kentucky. He was the brother of Union Brig. Gen. Samuel A. Gilbert and uncle to Samuel's son Cass Gilbert.