Throughout the movie, we see a man has been talking to James Cole with a mysterious voice, Cole asked him about his identity, but he denied to express it. Once Cole doubted a beggar in 1996 to be owner of that voice, but later Railley understood he was not that "person". So who was he? And why was he calling Cole "Bob" all the time?
This character is referred to as Louie/Rasping Voice on IMDb, and feelings are mixed on the interpretation of the character.
In the feedback section of the movie's IMDb page, there is a lot of discussion over the idea that Louie is a fellow time 'volunteer', who has extracted his own teeth to avoid being tracked.
However, other sites, including this one, suggest that Cole might have been hallucinating the entire time and that he has made up the Louie character and the rasping voice in his head.
The reason he was called 'Bob' could be because this is a form of greeting (although the correct pronunciation would be along the lines of "hey, Bub") - however, Cole is called Bob in the future segments, which leads me to believe that he was being mistaken for a different volunteer.
Here is a sample of dialogue that coroborates the fellow traveler theory:
TL/DR: The raspy voice Cole hears without seeing its speaker is an audio hallucination akin to a voice heard by a schizophrenia sufferer. Louie, the toothless time traveler, is a real person -- and after meeting Louie, Cole -- in a state of extreme mental fatigue -- retroactively associates the raspy voice with Louie.
FULL ANSWER: The raspy voice Cole hears without seeing its speaker is an audio hallucination akin to the voices heard by schizophrenia sufferers, who have described the voices as "spoken by an external, commanding authority" (quote from New York Times article about schizophrenia). Note below that in all three scenes with the raspy voice, the voice seems exactly like an external, commanding authority:
This leaves the scene where the voice appears to come from the homeless man Louie:
I believe Louie is telling the truth: he is a time traveller who has avoided returning to the future by removing his teeth. At the same time, Louie's use of "Bob" and the way he seems to sound like the raspy voice is explained earlier by Railly when Cole tells her she is in his dream:
Railly was wrong then -- it was her in Cole's dream. But the phenomenon she describes does occur when after meeting Louie, Cole -- in a state of extreme mental fatigue -- retroactively associates the raspy voice with Louie.