First and foremost they remove it to get a cleaner shot.
Secondly is that the windscreen is sometimes removed, to kill reflection, when the camera is mounted on the bonnet [hood]. The mirror is often attached to the screen.
It's not always done, because some directors find it more distracting than working round it.
Movies featuring the cars and driving itself seem to be more likely to not remove it, but this isn't an absolute - in fact none of it is an absolute.
It's a production/directorial decision.
There are some image errors below, this was done from thumbnails on Google Images in about half an hour. The general idea is no less true despite those inaccuracies.
Quick composite images…
And, just for fun, while we're on the topic.
How do they get those driving and action shots?
Clever car trailers & clever camera rigs.
And, in fact, apologies to Edgar Wright - I was sure the mirror was always there in Baby Driver; it just wasn't obvious in the first shot I found.
To make amends, more from Baby Driver, including mirror & clever driving rigs.
And, some classic Blue Brothers…
Mirror in, mirror out, mirror in, mirror out