I likely missed the bit where it's mentioned or not, but I was just curious is there any mention of his family in the movie? I assume his wife and son probably had enough after what happened in The Dark Knight, and separated/moved out of Gotham City.
Arnaki you are correct his wife and kids left and went to Cleveland. This statement is made during the Harvey Dent day events at Wayne manor. The line was spoken by Brett Cullen's character the Congressman to the police chief Foley.