Yoren was taking Arya back to Winterfell because he knew she would be in danger in King's Landing (whether Ned lives or not), and because Ned had (essentially) asked him to do it
We know that Yoren is the same black brother that Ned allowed to "recruit" from the prisons and streets of King's Landing. Arya recognizes him, and he mentioned "Lord Stark" giving him run of the prisons. He was in the square when Ned's execution happened because he expected Ned to be going with him:
"It wasn't supposed to happen like it did. I was set to leave, wagons bought and loaded, and a man comes with a boy for me, and a purse of coin, and a message, nevermind who it's from. Lord Eddard's to take the black, he says to me, wait, he'll be going with you. Why d'you think I was there?"
As far as I can tell, we never find out who wrote the message, but clearly someone (and probably not Ned, since he was locked up) knew enough to get Robert's bastard son out of King's Landing. Whoever that was also expected Eddard to go with him.
As Ned is being led to his execution, he passes Yoren (as seen here at about 0:30) and says the word "Baelor" to him. This is the name of the statue that Arya's standing on; when she tries to run from the statue to her father, Yoren grabs her. At the time Ned was being led to the stage, he doesn't think he's going to be executed, but remember that Arya was already on the run from the Kingsguard. Even if he lived in exile, she would still be a valuable political prisoner (like Sansa was) if she was found in King's Landing. Most likely, Ned was trying to ask Yoren to catch Arya and keep her safe so she could accompany them as they went North, presumably dropping her off at home. When Ned died, Yoren decided to take Arya with him anyway.
In the novels, Yoren's decision appears more spontaneous; there's no indication that he and Ned spoke after Ned was imprisoned, and certainly not as he was being led away. It appears that he just coincidentally saw Arya in the crowd as her father was executed, and decided to take her back with him North on his own.