I have understood that Vinayak Rao feeds dolls made of flour to Hastar. I haven't understood what is the substance that he used to make the protection circle. Surely, both the substance used to make the doll and the protection circle can't be flour.
Circle was made of flour. Here's why,
Go back to scene where Vinayak has taken Pandurang(his son) into Goddesses' womb to practice their act, while Pandurang makes protection circle with flour this is their conversation:
PANDURANG makes a protective circle with the flour on the heaving floor.
VINAYAK: Yeh Lakshman-Rekha ki tarah hai. Hastar isse paar nahi kar sakta. [This is like forbidden line. Hastar cannot cross this.]
PANDURANG: Aur kiya toh? [And what if he did?]
VINAYAK: Bhasm ho jayega. Yaad kar... Sona toh utha liya tha usne Devi ka, par anaaj nahi utha paya tha...Issi liye usse iss aate ki bhook bhi hai, aur darr bhi. [Then he will burn to ashes. Remember! He took the Goddesses' gold but wasn't able to take foodgrains, that's why he has hunger for food as well as fear of it.]
This is evident from the final scene where one "copy" of Hastar turns to ashes when it crosses the protection circle(sphere).