After graduating from Jesuit High School in New Orleans at age 16, the Phillies signed Caballero to a contract worth $10,000. Following a few short stints at the major league level and playing parts of three seasons in the minor leagues, Caballero was named the Phillies starter at third base in 1948. In his only season as an everyday player, Caballero batted .245 in 380 plate appearances at age 20.
After another stint in the minors in 1949, he was a backup infielder, pinch hitter, and pinch runner for the 1950 Phillies, and continued in that role through 1952. Following three more years in the minors, Caballero retired from baseball after the 1955 season. After his baseball career ended, Caballero worked as an exterminator. His Louisiana home was destroyed in 2005 during Hurricane Katrina, whereupon he then lived in Lakeview, New Orleans.
Caballero attended Jesuit High School in New Orleans, Louisiana, where he played baseball during the 1943 and 1944 seasons. During his first season, the Jesuit High team won both the city championship and the American Legion championship. The Philadelphia Phillies signed Caballero as an amateur free agent on September 9, 1944, the same year that fellow Whiz Kid shortstop Granny Hamner joined the organization.
Caballero planned on attending Louisiana State University, where he had been offered a scholarship to play baseball and basketball. Mel Ott, a scout for the New York Giants and former outfielder, wanted Caballero to sign with the Giants, but counseled him to sign with "'whoever offers you the most money'". The Phillies offered Caballero a $10,000 signing bonus ($171,000 today), and his father advised him to sign the contract offered by general manager Herb Pennock and owner Bob Carpenter.
At 16 years and 314 days old, Caballero appeared in four games for the 1944 Phillies, getting four at-bats without reaching base. In so doing, Caballero became the youngest third baseman ever to play in Major League Baseball, a record he still held at his death.
For the 1945 season, the Phillies sent Caballero to the Class-A Utica Blue Sox of the Eastern League, where he played in 130 games at third base, batting .272 with 132 hits, 9 doubles, and 10 triples. While there, he was managed by Eddie Sawyer, and played with Hamner and Richie Ashburn, with whom he lived for two seasons. Caballero appeared in nine games with the Phillies in 1945, scoring his first major league run and batting in a run as well.
In 1946, Caballero lost playing time as he and Hamner were both summoned to military service; however, he did play in 76 games with the Terre Haute Phillies in the Class-B Three-I League and one game with the Blue Sox. That year, Caballero hit the first two home runs of his minor league career with Terre Haute, along with nine doubles and four triples. Most of Caballero's 1947 season was played at Utica as well, where he posted a .287 batting average, 114 hits, and 13 doubles while splitting time with Hamner at second base. His last cup of coffee with the Phillies came at the end of 1947, when he notched his first major league hit and first walk in eight at-bats.
1948 was Caballero's first full season in the major leagues, playing 79 games during the year at third base, where he was the regular starter over Willie "Puddin' Head" Jones, and also made 23 appearances at second base. At the plate, Caballero had a .245 batting average, the second-lowest among the team's starters that season—higher only than catcher Andy Seminick. He posted 86 hits (13 for extra bases) in 380 plate appearances. In the field, Caballero played 102 games, making 18 errors in 363 chances for a .950 fielding percentage. He started a triple play in a game against Cincinnati when Virgil Stallcup hit a sharp line drive to third base; Caballero forced Ted Kluszewski out at second base with a throw to Hamner and Danny Litwhiler was picked off first trying to return to complete the third out. Additionally, Caballero notched the team's only hit in a one-hit pitching performance by Brooklyn Dodgers starting pitcher Rex Barney.
In the middle of the season, the Phillies promoted Caballero's former Utica manager Sawyer from their Triple-A farm team, the Toronto Maple Leafs of the International League, to replace Ben Chapman at the helm. Teammate Robin Roberts described Caballero, as well as Hamner, Ashburn, and catcher Stan Lopata, as having "the highest respect for [Sawyer]"—all having played under his tutelage at Utica; Caballero himself gave Sawyer the credit for his development, along with that of several other Whiz Kids.
Caballero regressed in 1949, playing in only 22 games with the Phillies, spending the majority of his season playing second base for the Maple Leafs in Triple-A. He played in 48 games, amassing a .318 batting average (the second-highest total on the team) and eight extra-base hits.
Entering the Phillies' 1950 season at 22 years old, Caballero was described as having "great potential", but the Phillies wanted Mike Goliat to take over as the everyday second baseman. Caballero went to the major league club's spring training as a non-roster invitee; his .379 average in the Grapefruit League won him a roster spot and a utility infield role with the Phillies. Media reports speculated that the Phillies would challenge the Dodgers for the National League pennant, and one of Caballero's most valuable roles as a utility player was as a pinch runner off the bench.
In the second game of a July 25 doubleheader against the Chicago Cubs at Shibe Park, starting pitcher Roberts batted in the bottom of the ninth in a 0–0 shutout. With Bob Rush on the mound for Chicago, Roberts managed a walk and was replaced on base by Caballero, who promptly took second base on a hit and run groundout by first baseman Eddie Waitkus. A single to center field by Richie Ashburn allowed the speedy Caballero to score from second and completed the doubleheader sweep for Philadelphia—Bubba Church having pitched a three-hit shutout in the first game earlier in the day.
As a pinch hitter, Caballero came through with hits in clutch situations, notably against the St. Louis Cardinals on August 30. Behind 6–3 at Sportsman's Park in St. Louis, the Phillies sent Caballero to the plate to pinch hit in the top of the seventh inning. He led off the inning with a single, scoring the first run of a seesaw come-from-behind rally that ended with a 9–8 Phillies victory, closer Jim Konstanty's 13th of the season. Caballero displayed his running skills again in September, replacing Goliat in the ninth inning of another game against the Cardinals, and scoring a late-inning run in place of Lopata against the Cincinnati Reds several days later to tie a close game that the Phillies eventually lost in 18 innings. On September 26, Caballero scored another crucial run against the Boston Braves when he pinch-ran for outfielder Dick Sisler. Waiting on third base after advancing from first on a Del Ennis single to center field, Jones hit a high ground ball to Boston shortstop Sibby Sisti, who booted the ball and allowed Caballero to score the game-tying run; the Phillies won, 8–7.
In the final game of the season, in which the Phillies defeated the Dodgers, Caballero appeared as a pinch-runner, replacing catcher Andy Seminick in the ninth inning. Philadelphia advanced to the 1950 World Series to play the New York Yankees, where they were defeated, four games to none. In the series, Caballero made three appearances: two as a pinch-runner—for Seminick in Game 2, and for Goliat in Game 3—and one as a pinch-hitter for Konstanty in Game 4, where he struck out. For the season, Caballero appeared in 46 games and posted a .167 batting average.
Caballero continued as a utility player after the Whiz Kids' defeat in the World Series, but those talented young players would never return to the Fall Classic with the Phillies. He played mostly second base during the 1951 season, fielding the position in 54 games. He posted only a .186 batting average at the plate and hit his only major league home run, along with 3 doubles and 2 triples in 84 games played. With 10 hits in 42 at-bats in what would be his final season in the major leagues, Caballero achieved a .238 average in the 1952 season, hitting three doubles. On June 15, Caballero had a three-hit game against the Pittsburgh Pirates in the second game of a doubleheader, one of the top performances of his career. He also batted in three runs in one game against the Giants on September 27.
In 1953, Caballero played for the minor league Baltimore Orioles, where he batted .268 in 133 games, the most played in his professional career. He added 25 doubles, 2 triples, and 5 home runs to his total, but did not earn a promotion back to the major leagues.
He returned to the Phillies system, playing for the Syracuse Chiefs. After batting .258 with 2 home runs and 41 runs batted in during the 1954 season and declining to a .225 average in 1955, Caballero retired from baseball after that season.
After retiring from baseball, Caballero returned to Louisiana, where he worked as a salesman for an exterminating company, partnering with E.J. D’Arensbourg in 1963. Eventually, he opened his own company in Metairie, Louisiana. Caballero lived in Lakeview, New Orleans, where his home was destroyed by Hurricane Katrina. He lost memorabilia signed by Babe Ruth, Eddie Mathews, and Pete Rose in the flood, calling the hurricane damage "a catastrophe".
Caballero lived with his wife, Clare, and they had seven children and nine grandchildren. He rejoined some of his old teammates in 2008 to commemorate the Whiz Kids on an Alumni Weekend. He was inducted into both the Greater New Orleans Sports Hall of Fame and the New Orleans Professional Baseball Hall of Fame. He died on December 8, 2016 at the age of 89.