Besides just being a well read guy, his former mentor Declan Gage (who is involved in at least two episodes, one being the great finale of the 7th season) is a profiling expert (if not pioneer), which explains his excellent knowledge in profiling and psychology.
Other than that I don't think he has been anything else than an investigator, or at least I cannot remember it to be stated in the show at any point. But he sure was in college.
What you have to keep in mind in addition to that, though, is that police work is his life. It is clear that his job as a detective is everything for him and even more so that he doesn't actually function without it and he fails at building any kind of personal life (in fact the most romantic and intimate relationship he has, for an interesting definition of "romantic", is with his "white whale" Nicole Wallace and is inherently job-related).
So he doesn't actually have many distractions in his private life (though, he earlier had his brother and mother occupying him to some degree). It would be wrong to classify him as having Asperger's or any similar characterization, he very well knows how interpersonal relationships work, but he definitely is a lone wolf, primarily due to and in turn for his job as a detective. And while you don't necessarily need too much knowledge in physics or literature for that, there's not too much else occupying his time either.