At first Lecter considers Clarice just another "rube" to be manipulated...
LECTER: You know what you look like to me, with your good bag and your cheap shoes? You look like a rube. A well scrubbed, hustling rube with a little taste. Good nutrition's given you some length of bone, but you're not more than one generation from poor white trash, are you, Agent Starling?
....but a defining characteristic of Hannibal Lecter is his desire for good manners.
(All media in which Lecter appears portray him as intellectually brilliant, cultured and sophisticated, with refined tastes in art, music and cuisine.
He is deeply offended by rudeness, and frequently kills people who have bad manners. Prior to his capture and imprisonment, he was a member of Baltimore, Maryland's social elite, and a sitting member of the Baltimore Philharmonic Orchestra's board of directors.)
Consequently, although Clarice is initially sent away, when the other inmate (Miggs) flings semen on her he takes offence on her behalf as, essentially, she is his guest.
It's later revealed that Lecter talks Miggs into committing suicide.
As for the lambs screaming, he's still a psychiatrist and the lamb incident was an defining moment for Clarice so he's interested in it. She had a need to save the lambs even though they were destined for slaughter.
LECTER: And you think if you save poor Catherine, you could make them stop, don't you? You think if Catherine lives, you won't wake up in the dark ever again to that awful screaming of the lambs.
So when Clarice manages to kill Buffalo Bill and save his victim Lecter wonders whether that was enough to dispel (at least in part) the childhood trauma and perhaps, now, the lambs have stopped screaming in her mind.
There's nothing personal in his escape and her attempt to capture him, it's just what they do. He wouldn't come after Clarice as an act of revenge as, she says, it would be "rude" in his mind...that what she figures out after initially saying she "can't explain it". She can't explain it in any other terms.
Plus. he still finds her interesting..
LECTER: I've no plans to call on you, Clarice. The world is more interesting with you in it.