IMO, they are simply making conversation to seem as if De Niro is a bumbling tourist. I don't recall the scene exactly, but his contact is then able to pull him aside briefly to have their private conversation and then conclude it by loudly directing De Niro to the post office.
As for the post office being used for different things, the content of the conversation is quite irrelevant to the plot. In some countries, the post office can be used for doing additional things (besides posting letters) like paying bills and the like. Tourists from a country where post offices offer a variety of different services might expect (in this case) French post offices to do the same. This is presumably why the contact asks (the tourist) De Niro what he actually wants to do at the post office in order to guide him better.
(I have no idea which services French post offices offer/do not offer.)