Magical performances comprising of spirits being summoned on stage where pretty common even in the 19th century. The effect was achieved indeed with the use of smoke and mirror. However, a typical tool in usage was the Fantascope aka Magic Lantern. This apparatus is also seen in the movie but can be easily overlooked as normal lanterns.
I found this interesting piece of trivia on IMDB:
The method for creating the ghosts as shown to inspector Uhl involved the projection of a pre-recorded image into a hazy background. Since the ghosts Eisenheim conjured could speak to and interact with the audience, he most likely used a different method popular among magicians at that time. A fantascope was used to illuminate a real person off stage. The image was reflected off of a mirror or glassplate, creating a ghosted image. The lanterns that Eisenheim tells his assistants to leave behind when they are packing up the workshop bear a strong resemblance to fantascopes.