A special election was held in New Hampshire's at-large congressional district on June 22, 1789 to fill a vacancy left by the resignation of Representative-Elect Benjamin West, who had declined to serve. This was the first special election in the history of the United States House of Representatives.
References
New Hampshire's at-large congressional district special election, 1789 Wikipedia(Text) CC BY-SA