Abba Yossi appears in a Leviticus Rabba (one of the Midrashim of Judaism) around the fourth century, in both Hebrew and Aramaic and in several sources.
Abba Yossi was at a spring in Zeitur, which appears to be in Galilee, when a female spirit appears to him and reveals to him that now, after her years of peace and service, there is a male spirit who is demanding her attentions and trying to drive her away from her spring. The Abba asks what he can do and the spirit tells him to bring the townspeople and any iron tools that they can find. They are to wait for a disturbance on the water and then they must beat the water and shout "Our Triumph, Our Triumph" (also translated as "We Win"). He gathers them up and they banish the evil spirit.
It was a difficult time, and the lesson they brought away from this parable was that the spirits were created to not need help, and even they are asking for help, so it is no weakness for man to need assistance in these times.