In The Dark Knight's fundraiser scene where Bruce Wayne invites many people for Harvey's fundraiser, Joker attacks the place with his goons, looking for Harvey. When Bruce sees this, he catches Harvey from the back and locks him up in a safe room.
When he is dragging Harvey, Rachel says in a surprising manner "What are you doing?" and Harvey was not unconcious at that time. So clearly Harvey knows that Rachel knows, who locked him up.
After this we don't get to know, how did Harvey feel about the incident. What was he told? Did he know that it was Bruce who locked him up or did he think that it was Batman? If he knew that it was Bruce Wayne, how come he had no questions about that, because it was a very unusual act from Bruce Wayne. How can Bruce Wayne know that men are coming for Harvey and he needs to be protected.