His motives are partly revealed in the book, but his true underlying motives will all be revealed later.
This is from his wiki page:
Petyr was born to a small and insignificant house, spending most of
his adult life trying to rise above his lowly birthright and become
something greater, craving for power and status. He possesses
considerable skill in commerce and coin, and is a master of court
intrigue, his brilliance matched only by his ambition and his gift for
political improvisation. He is often underestimated by many high lords
due to his low birth (a mistake that often comes back to haunt those
who make this assessment).
The key thing here is that due to his birthright he craves power. Whereas Lord Varys is, in his own way, looking out for "the kingdom", Baelish only cares about himself.
Now, from the episode you are describing:
Petyr then kisses Sansa, but before she pulls away from him, Lysa
sees them. She later summons Sansa and confronts her
near the Moon Door. Lysa grabs Sansa, and pushes her down to her knees
near the opening of the door, telling her that all her enemies have
gone through it. Petyr arrives and tells Lysa to release her,
promising to send Sansa away. Lysa releases Sansa, and Petyr goes to
her, but he admits that he only ever truly loved Catelyn, and pushes
Lysa through the Moon Door to her death.
This clearly only shows a minimal amount of reason behind his decision. As always, the books provide more insight:
When dying, she reveals that Petyr had convinced her to poison her previous husband, Lord Jon Arryn, claiming it was to prevent him from sending her young son Robert away to be fostered. Lysa, who was fiercely protective of her son, did his bidding. Sansa also hears that after Jon was killed, Petyr convinced Lysa to write a letter to her sister Catelyn stating that Lord Arryn had been poisoned and that the Lannisters were responsible. The killing is blamed on the singer Marillion.
So whilst that describes what happened, this is the reasons (that we know of so far) for why it happened:
Petyr manipulated Lysa into the killing of Jon Arryn that led to Ned going to King's Land and "started" the entire saga. He knew he could rely on Lysa, by playing on her love for him. It's clearly revealed in the book that he is playing on this love. He never truly wanted her. Her outburst made her too dangerous to live. She revealed his participation in the death of Jon Arryn and the fact she saw him kiss Sansa meant Lysa couldn't be trusted to act as Baelish wanted her to. What if she were to reveal to someone what he did? What if she were to reveal to someone his current whereabouts, in a fit of rage? What if she were to hurt Sansa, a woman who could be wed to either Baelish or someone of his choosing and as a result of the death of Robb would inherit Winterfell? No - there was too much risk and so he had to kill her. We don't know what his final goal is, but its clear that Baelish has lofty ambitions and is playing in the Game of Thrones.
Finally, this is an interesting piece of trivia about her death:
The book shows that Marillion is blamed for the death (and is tortured until he confesses). Whilst he wasn't present in the TV series, he was the character who was caught earlier in the series singing derogatory songs about Robert Baratheon. Joffrey gave him a choice between losing his tongue or his hands. He begged for mercy and his tongue was cut out. Whilst this does happen in the books, it happens to an unnamed singer, not Marillion.
