There is clearly nothing in the episode to definitely conclude what happened to her. If there would be any definite answer to her fate, it would have been spelled out. So you are correct in your statement that this is basically just a cliffhanger. What else could make you bite your fingernails more than not knowing if the little baby made it or not.
But the inconclusiveness of this is also what makes it pretty likely that she somehow got out of there alive. She just disappeared not to be seen again in the episode and a huge stain of blood could easily suggest that she's dead. So if you've ever seen any TV show or any kind of cliffhanger, this makes it pretty evident that this exactly not what happened to her. And even in a TV show that doesn't hesitate to kill off people, including children, she still has the bonus of being a baby, especially the child of a major (if not the primary) protagonist. All this, albeit far from conclusive, doesn't leave much doubt for me that there didn't happen anything heavily terrible to her.
And to finally conclude this, as the episode "Inmates" (S04E10) shows later, she indeed made it and it was Tyrese who brought her out of the prison alive, together with Lizzie and Mika.