The key is to watch Chaplin's legs and the rope.
As the rock goes off-screen and the rope is getting close to taut, his knees flex. At that point he clearly jumps into the water, with enough style (because he was an expert stuntman) to look as if he'd been pulled into the water by the rock. If you watch the rope too, you see that there's no point at which it goes taut, and actually when he jumps into the water it goes completely slack.
This is completely clear when you watch it back repeatedly and analyse exactly how it's done. For audiences though it happens fast enough that they don't see the knee-bend and jump, or think through the implications of the rope going slack. As a result they buy into the scene, and no doubt some of the original audience asked the same question you just have!
You can also be sure that the "rock" is nothing of the sort. For the stunt to be safe in water, the "rock" will almost certainly be balsa wood or something equally light, so that it doesn't genuinely sink afterwards! Notice that the "rock" goes off-screen to ensure you don't think about that - and notice that whilst Chaplin produces a very impressive splash from his pratfall, there's no splash from the rock, even though the rope could only let it go just off-screen.