From IMDb:
Some of the black characters are played by white actors with make-up, particularly those characters who were required to come in contact with a white actress. The person playing the Cameron's maid is not only clearly white, but is also obviously male.
Indirectly, this confirms that not all black characters were portrayed by white actors.
I also found some screenshots from the movie. The two men in the middle do not appear to be white actors in costume:
I could also see, given the subject matter and the time that not only were no Blacks hired but none would be willing to act in such a movie.
This is touched on in the quoted IMDb trivia. The issue wasn't so much that black people didn't want to be in the movie (at least, it's not mentioned, as far as I can find on Google), but rather that the white actors didn't want to come into contact with real black people.