Oliver is replaced off screen in 'The Trusty Servant' (where they meet 'Rev. Green' - the former cannabis dealer).
This is pre figured in the prologue where he becomes too drunk to continue the pub crawl from this point.
In the main body of the film, the entire group (including Andy, who drank all the shots in the previous bar) is pretty drunk by this time - Oliver especially.
However, after visiting the toilet in 'The Trusty Servant' Oliver reappears looking fine and sober. This is acknowledged later by Gary who is really impressed by his new found longevity and can be seen in Martin Freeman's performance, which becomes somewhat 'robotic' after this point and is also noticeable by his Bluetooth earpiece which continuously glows blue from here on to his reveal as a 'blank'.
EDIT: Whoops - forgot - Oliver's birthmark (which earns him the name 'Omen / Oman') has been removed by laser surgery by adulthood, but is again visible on his 'blank'