Update - A more direct answer to your question.
Your question implies that you missed a key point where Littlefinger was being tricked by the Stark siblings. Sansa specifically acted like everything was normal in the conversation with Littlefinger, so that he wouldn't leave Winterfell before they could convene the trial.
Sansa pretends like she's putting Arya to trial, because it makes Littlefinger think that he is getting what he wants: a divided Stark family. Littlefinger attends the trial because he thinks that he has gotten away with it, completely unaware that he is actually attending his own trial.
The full answer about what happened.
There have been many things that happened which did not make sense (thus implicitly hinting that they were a facade for something else, i.e. trapping Littlefinger). Though we can't know for certain where Sansa, Arya and Bran started working together, we can piece together the most likely turn of events that led to this scheme.
This list is not necessarily in chronological order, though I've tried to arrange the events in a way that explains how the Stark siblings started scheming (which should be somewhat chronological per chapter)
Note
I'm aware that you're asking who convinced Sansa, and I'm responded with how the scheme started, but the two are connected. The Stark siblings started scheming together because they started to understand the truth.
Chaos is a ladder.
Bran pretty much called LF a liar to his face. When given the dagger, he immediately asks if LF knows who it belonged to.
Why would that be important? That is not an obvious question to ask. The only likely reason that Bran asks this question, is because he already knows that LF has lied before about who the dagger belongs to.
LF lies to Bran and says that he doesn't know. LF tries to distract the conversation by talking about Bran's experiences, but Bran promptly brings the topic up again when he says "Chaos is a ladder", another one of LF's quotes (while it's not a lie, it's a clear indication of how LF approaches the game of thrones, an indirect admission that he sows chaos to reap the rewards).
Most likely, Bran talked to Sansa soon after, and they started working together. Bran may have taken longer to talk to her (his three-eyed-ravenness gets in the way of his humanity from time to time).
If Bran was already scheming with Sansa at the time, then it was a mistake to blatantly call LF out on a lie, since they were trying to trap him.
The best assumption we have is that Bran was not aware of a scheme against Littlefinger yet. However, Littlefinger's lies in this conversation may have put fuel to the fire, and caused Bran to proactively seek out Sansa and talk to her.
Tangentially, Littlefinger should have understood from this conversation that Bran knows his secret, and likely knows more than what he is saying. Littlefinger should've hightailed it out of Winterfell at this point, aware of the sword of Damocles that Bran has blatantly pointed out to him. It feels like bad writing that LF goes on with his business in Winterfell unhindered.
Though it's not impossible that it shows a flaw in character, that he takes the risks because he still wants more power.
Sansa has seen behind Littlefinger's mask.
Sansa has known LF for some time. He has acted like a friend to the family, offered Sansa a way out of King's Landing, implicated Sansa in Joffrey's murder, killed Ser Dontos (a well meaning oaf who Sansa had saved from death before) in front of her, makes her join in his web of lies, tells her how much he loved her mother and then inappropriately kisses her, kills aunt Lysa (to be fair, she did threaten to kill Sansa but she was clearly an unhinged woman), and marries her off to Ramsay.
It is true that Sansa was a naive girl for most of these experiences, so she never saw LF's manipulations for what they were (at the time). However, Ramsay was the straw that broke the camel's back. Since then, Sansa has changed and matured considerably. She can now look back on the past events, and possibly see them for what they actually were.
LF made some slipups in Winterfell too. During their time spent together (both in the Vale and Winterfell) LF has revealed a lot of his MO to Sansa. He has taught her to lie, he has made her a partner in crime, he has taught her how to think the worst of people. She knows how he works.
Most viewers assumed that LF loves Sansa because he sees her as the next best thing to Catelyn. But given the fact that LF has explained what he knows, it's possible that he sees her as a pseudo-daughter, the daughter him and Catelyn should have had.
It's somewhat of a trope that those who consider themselves a parent (sub)consciously end up teaching their (perceived) child their "skills of life", to pass on their knowledge.
One of the bigger slipups occurred when he tried to suggest to Sansa that Arya wants to kill her to become Lady of Winterfell. Anyone who knows Arya, knows that she doesn't want to be a lady.
If Sansa even just considers Littlefinger being dishonest, withough being certain that he is in fact lying, that can bring back all her past memories of him, and very quickly an image starts to form. The image of a man who lies and schemes and manipulates, and has no issue whatsoever with throwing people under the bus even if they're friendly to him (Ser Dontos, Aunt Lysa, Sansa herself when he married her off to Ramsay).
It's equally likely that Sansa could have approached Bran. However, this makes less sense, if you consider that Bran has been absent from the events for a long time.
The only reason Sansa would've approached Bran (instead of the other way around) is if Sansa is looking for confirmation ("checking the video footage"). Which is of course also possible, though it seems a bit cheap (narratively speaking) for characters to start running to Bran for confirmation about their suspicions. It gets too close to a deus ex machina.
Arya joins the plot only after settling some grudges with Sansa.
Bran and Sansa both have been eye witness to LF's past shady dealings (in different ways). Any conversation between them would very quickly cement the actual truth.
This is similar to how Sam and Bran figure out Jon's legitimate birth together. Sam suggests a marriage between Rhaegar and Lyanna, and Bran "checks the video footage". Bran hasn't seen everything that happens everywhyere, but he has access to it. He only needs to know where to look (Sam and Sansa can give him that direction).
However, I think that Arya wasn't part of the scheme initially. The first conflict out on the wooden walkway seems genuine. Arya was bringing up things that she has carried from her childhood, and actively calls Sansa out on undermining Jon, even if only subconsiously.
But then we get to the scene with the faces and the dagger. A quick rundown of the scene.
- Sansa finds the faces.
- Arya explains why she uses them, also revealing a scary and dark side to her character.
- Sansa is visibly upset and scared by this information.
- Arya wants to play the game of faces and tries to focus on Sansa undermining Jon. But Sansa doesn't respond (she's too busy talking about the faces) and Arya doesn't get a straight answer.
- Arya picks up the dagger and mentions becoming Lady of Winterfell by cutting Sansa's face off.
- Arya then gives Sansa the dagger and walks off.
The big question here is whether Arya was already part of the scheme.
If she has been part of the scheme, then the only way that this scene makes sense is that it is staged, intended for Littlefinger (who is probably spying) to think that there is real animosity between the sisters.
But if they are expecting Littlefinger to spy on them, why would Arya give the dagger to Sansa? They're trying to give Littlefinger ammunition to accuse Arya of treason; and Arya giving Sansa the dagger is basically Arya saying "I'm not your enemy". This doesn't make sense. If Sansa and Arya were already in cahoots, there's no need for a show of faith/trust.
If she had not been part of the scheme, this scene makes more sense. Arya gets to reveal something extraordinary (shapeshifting) that she hasn't been able to talk to anyone about (in Westeros). Sansa learns about the changes to Arya's character.
Arya hints at cutting Sansa's face off with the intention of becoming Lady of Winterfell (not just petty revenge), as a way to probe Sansa. If Sansa had taken the bait, that indicates that Sansa believes that the position of power comes before family (thus somewhat proving that she is willing to undermine Jon).
But Sansa does not take the bait. She looks hurt and distraught. Sansa responds to her sister telling her something hurtful, not someone taking her power. To understand the distinction between these options, imagine how Cersei would've responded to the same threat. Her response would've focused on her power being taken away from her, she would call it treason. Her focus (power, not family) reveals her priorities.
Arya takes this as sufficient proof that Sansa is not intent on betraying family for a seat of power.
Note that before Arya tried probing Sansa, she tried it more directly, trying to make Sansa play the game of faces:
Back in Braavos, before I got my first face, there was I game I used to play.
The game of faces.
It's simple: I ask you a question about yourself and you try to make a lie sound like the truth.
If you fool me, you win.
If I catch a lie, you lose.
How do you feel about Jon being king? Is there someone else you feel should rule the North instead of him?
But Sansa was much too distracted by the faces, and also did not understand the point of the game, so she did not partake in it. Arya then changed gears and decided to use a more indirect approach (unsettling Sansa in order to find out what's important to her).
Arya then gives Sansa the dagger. It's a way of saying "you passed the test", Arya is no longer suspicious of Sansa. And this has another consequence (likely unforeseen by Arya): it also telegraphs to Sansa that Arya is not trying to do the same (kill Sansa for her seat of power).
Arya giving Sansa the dagger is likely the turning point here, which puts an end to both sisters' suspicions about eachother. Sansa would then be likely to include Arya in the scheme to trap Littlefinger.
Somewhat obviously, she would not have included Arya when she was still suspicious of Arya wanting to take up Sansa's seat of power, because that means that Arya could possibly be working with Littlefinger.
And so the scheme begins
Bran, Sansa and Arya are now working together. When you combine their information, it paints a clear image of Littlefinger and his past misdeeds.
- Bran has seen Littlefingers betrayals in King's Landing (let's assume he only knows what he has shown that he knows, to prevent Bran from becoming the deus ex machina).
- Sansa has seen Littlefinger's manipulations and murders in King's Landing, the Vale and Winterfell.
- Arya found the note, which proves that Littlefinger was trying to start an argument between the Stark sisters.
edit Small addition to finish my thought and more directly answer your questions.
What the Stark siblings know is not enough to conclude that Littlefinger is guilty, at least not in the eyes of third parties (anyone can argue that Bran is lying).
This is why the trial is held. It gives Littlefinger the chance to defend himself (which he fails at), and it gives third parties the chance to intervene on Littlefinger's behalf (if there is evidence to the contrary, but we as the viewer of course already know that LF is definitely guilty).
Your question implies that you missed a key point where Littlefinger was bring tricked by the Stark siblings. Sansa specifically acted like everything was normal in the conversation with Littlefinger, so that he wouldn't leave Winterfell before they could convene the trial.
Sansa pretends like she's putting Arya to trial, because Littlefinger thinks that he is getting what he wants: a divided Stark family.
This is why Sansa suddendly shifts her accusations to Littlefinger. He did not see it coming, and it caught him off guard.
I'm a slow learner it's true but I do learn... thank you for your many lessons Lord Baelish.
any treason on his part
during that scene. The way I interpret that conversation is that Sansa was trying to figure out what Littlefinger would try to get her to do - once she heard Littlefinger was essentially proposing for Arya to be executed, she'd heard enough - she even lists this as one of his crimes:What's the worst reason you have for turning me against my sister? That's what you do, isn't it? Turn family against family, turn sister against sister
.she was declaring his crimes, one by one, as if she was there when he did them.
That's because for many that she listed, she was. She was there when Lysa explained that she killed Jon Arryn at Littlefingers command. And she was there when he pushed her out the moon door. Having seen that, once she decided he was doing the same to her it wouldn't take much convincing for her to believe he did the other things as well.