While this might not be a complete answer already (and it's been quite a while since I saw this one), Wikipedia says that it was first and foremost a kind of convenience partnership between the movie and the song that grew into a necessity after the fact:
When director Mike Nichols and Sam O'Steen were editing the film The Graduate, they initially timed some scenes to this song intending to substitute original music for the scenes. However, they eventually concluded that the song could not be adequately substituted and decided to purchase the rights for the song for the soundtrack. This was an unusual decision for the time, as the song had charted over a year earlier and recycling established music for film was not commonly done.
So they seemed to have at first primarily chosen the song because its pacing and its mood were fitting to the movie, not directly intending the song to be used in the final movie, which is probably why the lyrics are not directly related. But it probably fit so well (and they grew accustomed to its place in the movie) that this was kept as the final solution (and nowadays I guess no-one would say this was a bad choice).
That being said, I'm not entirely sure what to make out of the lyrics of the song in particular, as I'm not a big expert on the matter and not too sensitive to sub-text. But to me the song's style and its lyrics have quite a melancholy and isolating overall feel to it, which ties in well with the whole coming-of-age theme of the movie and Benjamin's drifting state.