The people of Gotham City faced a very hard time. The mob was still on its height and was only slowly taken its powers (to a large part by Harvey Dent), there was corruption all over the governmental institutions. While the Batman did his best to fight crime, it was still a very dark situation when a masked vigilante has to do what the police and the government are unable to do. And in contrast to this, Harvey Dent as district attorney was a man, who fought this corruption and organized crime, an unmasked and official crime-fighter. He was the white and shiny knight contrasting the Batman and was a spark of hope at the horizon of Gotham city:
Bruce Wayne: You know that day that you once told me about, when Gotham would no longer need Batman? It's coming.
Rachel Dawes: Bruce. You can't ask me to wait for that.
Bruce Wayne: It's happening now. Harvey is that hero. He locked up half of the city's criminals, and he did it without wearing a mask. Gotham needs a hero with a face.
Now the Joker wanted to show, that not just each and every citizen of Gotham was able to be dragged onto his level and into the chaos, but especially one of the best and shiniest of them all, and with this he succeeded.
Batman: Because you were the best of us. He wanted to prove that even someone as good as you could fall.
Two-Face: And he was right.
Nobody cares if a masked vigilante goes postal and kills people he thinks to be evil, but after this hard time the people of Gotham had to endure, if they would have known that their white knight Harvey Dent was as evil and crazy as everybody else, they would have completely lost their hope in this city and a future where masked individuals are not needed to keep the city from falling apart. Harvey may not have been the hero Gotham deserves, but the one it needs right now.