I've just watched S02E08. Beforehand I thought Catelyn Stark was a strong and sensible woman in the sense that she should strive together with her son to win this war, just as shown in the previous episodes. But in this episode she released Jamie Lannister in the hope that her daughters would be returned, and she did it in the absence of Robb. I feel this is so reckless and even stupid, not only because of reasons Robb told her but it also put shame on the King-to-be. In the end it seems to me that all she cares about boil down to personal feelings. It dumbfounded me that she possesses such kind of nature because it's never foreshadowed in the former episodes (or did I miss it?). I can't bear this kind of plot for it upsets me quite a bit for I'm not sure if I should blame on the character or the director.
You're forgetting one important character trait: Catelyn is a mother who would do anything for her children. First example from the show was her fighting the assassin barehanded who tried to murder Bran.
Another example: spoilers if you haven't reached season 3 episode The Rains of Castamere
When the Red Wedding massacre happens, out of desperation, she grabs one of the Frey women (Joyeuse Erenford) and demands to spare Rob's life in exchange of hers (in the books it was one of Frey's sons).
Catelyn grabbed a handful of Jinglebell Frey’s long grey hair and dragged him out of his hiding place. ‘‘Enough, I say. You have repaid betrayal with betrayal, let it end.’’ When she pressed her dagger to Jinglebell’s throat, the memory of Bran’s sickroom came back to her, with the feel of steel at her own throat. The drum went boom doom boom doom boom doom. ‘‘Please,’’ she said. ‘‘He is my son. My first son, and my last. Let him go. Let him go and I swear we will forget this . . . forget all you’ve done here. I swear it by the old gods and new, we . . . we will take no vengeance . . .’’
‘‘I take you for a father. Keep me for a hostage, Edmure as well if you haven’t killed him. But let Robb go.’’
So it isn't out of her character at all. She's thinking about her children's safety even if there's a tiniest chance.