You mentioned two things right: the smell, which indicated the belonging of the poor family, and crossing the line.
The driver concluded a few times to Mr. Park that he really loved his wife. For Mr. Park it was entering his private territory, which he could not accept from a servant, because he thought of himself to be the upper class. At the party Mr. Park told the driver to do his job, which he was paid for. He didn't want to get close to him on a human level.
When the killing happened, the driver was grieving about his daughter, and Mr. Park didn't care about her and only wanted to save himself and his son. While it's right for the father, he should have thought of taking the wounded girl to the hospital as a human being, or saving others, but not running away. He didn't ask to drive him away, but rather demanded the keys from the driver, leaving everyone behind, and was disgusted by the smell.
In this critical situation he didn't care about his manners and showed his arrogance and ignorance. He still considered himself to be supreme. The driver lost his daughter and decided not to let the rich guy get away. He has assumed the role of the God and killed him.
So the main parasite (the driver) eventually kills his host, Mr. Park. It refers to the title of this film, "Parasite".