In a sense it's like a more perverted Stockholm Syndrome where the victim becomes infatuated with their captor.
Stockholm syndrome, or capture-bonding, is a psychological phenomenon in which hostages express empathy and sympathy and have positive feelings toward their captors, sometimes to the point of defending and identifying with them. These feelings are generally considered irrational in light of the danger or risk endured by the victims, who essentially mistake a lack of abuse from their captors for an act of kindness.
Theon's case is probably one of the most tragic in the series when looked at on a whole.
- Taken from his homeland as a boy to essentially be a hostage so that the Greyjoy's don't rebel again
- Sent back to his "homeland" in order to recruit his blood family to the Stark's cause only to be rejected wholly by his own father.
- Attempted to give a real name to himself by sacking Winterfell but failing to do so with any kind of actual authority/real battle and resorts to killing 2 innocent farmboys to pose as Brandon & Rickon.
- Gets betrayed by his own men and given to the hands of Ramsay to do with him as he pleases.
At this point in the story, Theon tries to scrounge up just a small bit of pride and dignity and attempts to outlast Ramsay's torture. But because of either his willingness to do things half-assed or because he just isn't as strong as he thinks he is, he chooses to be subjugated by Ramsay instead of being killed or tortured any longer.
As far as realism goes, while not a psychologist of any regard, I think this kind of conditioning is completely within the realm of possibility. Coming from Theon's perspective, ever since he adopted the "Reek" personality, his life has actually increased in quality, in a sadistic relativity. No longer tortured, as long as he obeys, etc.
While I do feel the portrayal of his refusal to go with Yara was a bit much, I feel that if Yara showed him that Ramsay was dead/killed Ramsay, it might have had a bigger impact on Theon going with her back to Pyke. But because he did not know if Ramsay was alive or dead (later being confirmed as alive, obviously) he stuck with the "safe" route of denying his own freedom.
