Born in Honolulu, Boyd was the son of Edwin Harbottle Boyd (1834–1875) and Maria Punapanaewa Adams Boyd (1841–1891). His family background was of British and Hawaiian descent. His middle name honored his two-time great-grandfather John Harbottle (1781–1830), a British naval officer, who was one of the first foreign residents in Hawaii and Kamehameha I's port pilot. Harbottle's wife, High Chiefess Papapaunauapu, was the adoptive granddaughter of Kamehameha I. Boyd's maternal grandfather, Alexander Adams, of Scottish descent, was another well-known foreign advisor of the King and has been credited by some historians as the designer of the flag of Hawaii. His paternal grandfather Robert Lopaka Boyd (1785–1870) had served as the King's shipbuilder, and it is believed that he came from the British West Indies.
The Boyds were considered among the most prominent families of Hawaii, and their residence in Maunawili was often frequented by Hawaiian royalty, visiting foreign dignitaries, and world-famous writers and artists. Boyd's father made his fortune as a merchant and cattle rancher and served many political posts in the Kingdom's legislature, so it was understandable that James would follow in his footsteps.
In 1877, Boyd served as Captain on the Staff of the Governor of Oahu, John Owen Dominis. He was elevated to Colonel on King's Staff in 1880. During this period of service he became better acquainted with Princess Liliʻuokalani, the Governor's wife. Boyd was quite popular and gained the confidence of the Hawaiian royal family. During the coronation of King Kalākaua in 1883, he was trusted with the position of bearer of the royal crowns. He also served as Marriage License Agent in 1880, Private Ways and Water Rights in 1882, Captain Co. C, Prince's Own in 1885, and the King's Special Commissioner in 1886.
On September 20, 1886, Boyd took possession of Ocean Island, later renamed Kure Atoll, for the Hawaiian crown as King Kalākaua's Special Commissioner. Kalākaua had done this in response to frequent shipwrecks along the surrounding reefs, including the notorious 1870 wreck of the USS Saginaw. He ordered that a crude house be built on the island, with tanks for holding water and provisions for any other unfortunates who might be cast away there. But the provisions were stolen within a year and the house soon fell into ruins.
In 1887, he served as secretary and attaché to Col. Curtis P. Iaukea and accompanied Princess Liliʻuokalani, Governor Dominis, and Queen Kapiʻolani to England for the celebration of Queen Victoria's Golden Jubilee. The Hawaiian party was graciously received and given the same honor as every nation in attendance. In London, James was reunited with his younger brother Robert Napuʻuako, who had been studying in the Italian Royal Naval Academy for seven years under Kalākaua education programs for Hawaiian youths. Robert requested to return home, and James agreed and sent a letter to Hawaii requesting for his return. It would take another three months before his brother could return to Hawaii. In the meanwhile, the royal party was to continue their tour in Europe, but their mission was cut short by the unrest back in Hawaii, where King Kalākaua was forced to sign Bayonet Constitution.
A few months afterwards, his brother Robert returned home along with other Hawaiian students who had been studying abroad under the financial support of King Kalākaua, which ceased because the new legislature refused to continue to fund the King's many frivolous ventures. A rebellion led by Robert and his classmate Robert William Wilcox failed to change the political atmosphere in Hawaii and only served to anger the annexationists even more.
After the King's death in 1891 and his sister Liliʻuokalani accession to the throne, Boyd was reappointed as Colonel on the Queen's Staff. On January 14, 1893, a large crowd of Hawaiians gathered in front of ʻIolani Palace demanding for a new constitution, but Queen Liliʻuokalani addressed the crowd saying that she couldn't give them the new constitution at that time because of her ministers' refusal to sign and ratify the document. The Queen insisted that they return home peacefully.
This action would have a significant impact on the course of Hawaiian history. Three days later, two of her ministers betrayed the Queen to the Committee of Safety, and she was overthrown and the Provisional Government was established by the annexationists.
Boyd's family strongly opposed the Provisional Government and the Republic of Hawaii that followed, because of Boyd's friendship with Liliʻuokalani and relation to Princess Kaʻiulani, as her brother-in-law. But after the annexation of Hawaii by the United States, Boyd served the Territorial government as Superintendent of Public Works from 1901 to 1902.
Boyd died on August 14, 1915 in Waimea, on the Big Island. The funeral was held on August 18 in Honolulu, and he was buried in the Oahu Cemetery.
On August 16, 1888, Boyd married Helen Caroline Maniʻiailehua Cleghorn (1863–1927), the second daughter of Archibald Scott Cleghorn and his first wife, Elizabeth Pauahi Lapeka. (It is interesting to note that all of Cleghorn's daughter from his first marriage married men named James.) Helen was the half-sister of Princess Kaʻiulani, Cleghorn's youngest daughter with his second wife, Princess Likelike. His marriage brought Boyd closer to the King and the royal court. He and Helen had seven children: Cecil Archibald, James, Norman, Helen, Irene, Hilo, and Edwin Harbottle. Their descendants survive to this day.
According to legend, the famous song Aloha ʻOe was inspired by a visit by the then-Princess Liliʻuokalani to Maunawili in 1877, as part of her tour of the island of Oahu in her first act as the newly appointed heir-apparent. As the riding party prepared to return to Honolulu, Liliʻuokalani glanced back at gates of Maunawili and witnessed Colonel Boyd receiving a wreath of lei and a loving "farewell" from a young Hawaiian girl on the ranch, which he returned with a kiss and "one fond embrace". Helena G. Allen, in her biography of the Queen, The Betrayal of Liliuokalani, last Queen of Hawaii, 1838-1917, disagrees. She maintains that the song was in fact based on a romantic affair between Boyd and Princess Likelike, who was his future wife's stepmother, during the aforementioned visit.
Boyd's family residence at Mānoa was called Waola. His parent's estate at Maunawili was sold to William G. Irwin in 1893. The piece of land changed hands multiple time until its last resident vacated the premises in 1985. Currently the estate is owned by HRT Ltd. The Boyd's house dating from the 1860s still stands and is called The Queen's Retreat (Boyd/Irwin Estate), but has suffered major vandalism in recent years due to inefficient surveillance. Efforts have been made to save and restore the historic site due its importance in Hawaiian history, as the alleged place where Queen Liliuokalani composed her immortal song Aloha ʻOe.