In the first part of the movie he kills her family and she escapes. He is known as the Jew Hunter and never lets any Jews escape him. Later when he meets her under her assumed name Emmanuelle he makes references that suggests that he does recognize her. However the conversation could also have meant nothing. If he did actually recognize her I find it very strange that he didn't do anything about her.
I believe it depends on the viewer whether to take it as an indication to the accident or not. When we as viewers of the movie see the whole conversation happening in every moment we feel the fear of revelation of Shoshana's original identity to the Colonel, given we are already been introduced with his cunning and cold but cruel behavior. We feel such because the whole story is shown from the start how she survives from the unnecessary chasing of the Colonel (from her point of view). So when after a long time Shoshana meets with him, we as well as her, feel fear to see the executioner of her family and if he might recognize her! So maybe the Colonel did not identify her (chances is 99%). But we (Shoshana and us) automatically feel fear. His ordering milk for her or the last sentences by him:
pushes us in the wrong direction out of this fear. As long as the conversation went on, we hold our breath with her and when he leaves, we loose it! So I think the conversation was nothing regarding him identifying her. He was just trying to dig into her individuality as a quality security officer (Though bad in nature, we can't ignore his responsibility to his work!). But the fear of blowing her cover makes us make out unnecessary meaning and that is how a winning script is!
I'm almost positive he knows it's her.
Those little things he says are just like his personality to poke at her. Inquiring deeper than most people about her aunt and uncle and how they died. Ordering milk for her knowing she had been taken in and saw her family murdered at a dairy farm, watching her as he said it.
Jews in hiding weren't hiding because no one would notice them, they probably made the mistake of leaving behind pictures, The state probably had photo ideas. And Landa doesn't seem like the type to forget a face.
And that last thing he couldn't remember that must have just not been important. He ended the movie betraying Hitler. That's not a split second decision. He had probably known at that time the state of things in the war and was planning his betrayal at that time
Why exert himself more than needed?