The Azimganj Raj family was a zamindari clan that descended from Raja Chauhan, the ruler of Ajmer (135BC to 110BC). Raja Dudhor Rao, around 165AD, converted to Jainism and gave his name, Dudhoria, to his descendants. They migrated from place to place, until in 1774, they left Bikaner and arrived in Azimganj in Bengal, where they commenced trading in country made cloths. He first patriarch of the clan was Babu Harjimal Dudhoria who married and had issue, whose descendants governed as Zamindars till the 1950s. They were titled as Babus (Gentry) and Rajas (King).
References
Azimganj Raj Wikipedia(Text) CC BY-SA