In Red Dragon Hannibal received a fan letter/note written on toilet paper. He destroyed just the part which states how he should contact the dragon. So why didn't he destroy the whole thing?
This is explained in both the film and the book. Here are the direct quotes.
In the film:
Graham: It was full of compliments. He couldn't bear to part with them.
And in the book:
Graham: It was in the middle of a paragraph full of compliments. [...] He couldn't stand to ruin them. That's why he didn't throw the whole thing away.
Although Lecter didn't keep any souvenirs from his murders (as Sterling correctly notes, he ate them), he often takes pride in them. He also boasts of his knowledge, and his only Achilles heel is humiliation. So it's understandable why he wants to keep these compliments around (especially since any fan mail he'll ever get would probably be screened).