Yes, there is an explanation for that. Dr. Mann was in some kind of hypersleep. This is apparently an established technology in the film. You can see Cooper and the rest of the Endurance crew enter hypersleep during their 2 years of travel from Earth to Saturn.
It is also mentioned that Romilly could have gone to hypersleep, too, but chose not to anymore at some point since he was unaware of when Cooper and Brand would return, which is why he did age significantly (even if maybe not the entire 23 years), and why Brand assumes that he didn't go to sleep.
Brand (seeing old Romilly): Why didn't you sleep?
Romilly: Oh, I had a couple of stretches. But I stopped believing you were coming back. And something seemed wrong about dreaming my life away.
And later Dr. Mann also explains how he entered hypersleep waiting for someone to rescue him:
My supplies were completely exhausted. The last time I went to sleep, I didn't even set a waking date. You have literally raised me from the dead.
Now of course it's unclear if that hypersleep is able to completely halt any aging at all, but given the science-fiction setting it's not a big stretch to accept that it slows down aging to a large degree.