Annie has run away from the orphanage before. The other girls and Mr. Bundles all seem pretty familiar with her tricks, and we know Annie is clever and mischievous.
So even if he doesn't know her name, the police officer recognizes her as the girl who's always running away. He probably walks the same beat regularly, so he's brought her back from at least some of her previous escapes. I can't find the clip on YouTube, but he has a "Hey, that's that girl again" expression on his face when he first starts following her. Remember, she's pretty distinctive looking with her curly red hair and (apparently) one brightly-colored sweater that she wears every day. He starts following her at a walk; when she starts running away from him it confirms that she's the one.
When he brings her back to the orphanage, he says "Look what I found under a paving stone" in a joking tone, not "Isn't this one of your girls?" or "Did you know this child was missing?", indicating that it's a familiar occurrence that he can be casual about.