From a legal perspective, it's worth noting that the criminals he's referring to aren't those that Dent has already convicted in the past but the 549 criminals that Dent convinced Commissioner Gordon to arrest in scene 91.
Not only are the criminals involved not yet convicted (far from it) but Dent makes it abundantly clear to the Mayor that the only reason why Judge Surrillo is entertaining this "farce" is because of Dent's personal involvement:
MAYOR : 549 criminals at once?! How did you convince Surrillo to hear this farce?
DENT : She shares my enthusiasm for justice. After all, she is a judge.
MAYOR : Even if you blow enough smoke to get convictions out of Surrillo, you'll set a new record at appeals for quickest kick in the
DENT : It won't matter.
The whole thing is a dramatically risky move (basically a gigantic nuisance suit) and one that seems doomed to failure. Since all parties recognise that the majority of the criminals will be out on the streets within a few months anyway due to lack of evidence, the Mayor states that he is relying on Dent's public popularity to smooth over the obvious recriminations about the wasted time, effort and cost.
No Dent (or if Dent turns out to be dirty), no trials.
MAYOR : The public likes you, Dent. That's the only reason this might fly. But that means it's on you. They're all coming after
you, now. Not just the mob... politicians, journalists, cops -
anyone whose wallet's about to get lighter. Are you up to it?
You better be. They get anything on you... those criminals will be back on the streets...