When Molly is running the game on behalf of Dean, he tells the players:
Hey! Tip Molly if you want to get invited back next week.
Which establishes the trend of people tipping to be invited back.
After she steals the game form Dean, she says:
The game was mine now. I incorporated and Molly Bloom Event Planning was born. I paid taxes and 1099'd my employees. I never became romantically or sexually involved with any of the players. The game would have fallen apart. This was back when I was still making good decisions. And I went to a lawyer to make absolutely sure all this was legal.
Conversation with the lawyer:
Lawyer: Are you taking a rake?
Molly: No
Lawyer: Then you're not breaking the law. Now can I give you some advice?
Molly: Please
Lawyer: There's a saying in my business: Don't break the law when you're breaking the law.
Molly: What do you mean?
Lawyer: No drugs, no prostitutes, no muscle to collect debts.
Molly: Oh, I don't do anything like that. But you just said that I'm not breaking the law.
Lawyer: Keep it that way, because you don't want to break the law when you're breaking the law.
Molly: Am I breaking the law?
Lawyer: Not really.
Molly: We're able to find out for sure, aren't we? Laws are written down.
Lawyer: You're not taking a percentage of the pot?
Molly: No.
Lawyer: Then you're running a square game.
She follows up saying:
My game had a tricky ecosystem. It was built around escapism and exclusivity. These guys could buy their way into anything and anyone but here in this room you couldn’t
buy your win. You couldn’t buy me, you couldn’t buy the girls and you couldn’t buy a seat at the table.
Which I take to mean that the players are incentivised to tip well in order to be able to keep playing. We find out an example tip via "Bad Brad":
Every week he came to the game, lost a quarter of a million dollars and tipped me $5,000 so he could play the next week.
Molly has this conversation with Player X where we learn that all the profit she makes is from tips:
Player X: You know who the biggest winner in this game is?
Molly: You
Player X: You know who the second biggest winner is? It's you. What are you taking home, ten thousand a night now?
Molly: That is my business. Literally.
Player X: Between you, the dealers, and the servers, you're taking a lot of money out of this game.
Molly: Not as much as I'm bringing to it.
The fact that she gives credit to her players drives the drama for the second half of the movie. The $1.2 million mentioned in the question ends up being covered by Player X, which causes Molly to accuse him of cheating. This results in her losing the LA game and she moves to New York and starts her own game.
However, despite vetting the players, she still ends up with some who are unable to pay up. The dealer, B, tells Molly she needs to start taking a rake (a percentage of the pot from each hand that the gamerunner keeps) so that she's not taking such a big risk.
B: You gotta do it. Not doing it is insanity. You must be able to see that. You've been stiffed four or five times. You've got hundreds of thousands on the street, that's money you're never gonna see again.
After another incident when a player can't pay Molly:
B: You're exposure's crazy. It's not if, it's when. You're gonna get blown up. Your risk is nuts.
Molly: If I took a rake, this would no longer be legal.
B: And if you can't cover, this game will no longer exist. You're the bank now. You're guaranteeing the game. If you see a hand you don't want to carry, just look at me, flash me a number, and I'll take it off the table. Most runners cap it at 5%.
The next game we see ends up with a $3 million pot.
Molly (V.O.): She was right, I was extending credit, big numbers. And it’s not like Harlan Eustice hadn’t already put the fear of God into me. If I couldn’t pay, one time, that’d be the end of the game. I was the house.
Molly holds up 2 fingers, signalling a 2% rake.
Molly (V.O., cont) That's how quickly I made the decision. And just as quickly, B calculated 2% of the pot and took it off the table.
B takes the chips out of the pot, and taps them so everyone can clearly see she's taken them.
You're right, it's highly unusual to start taking a rake in the middle of play. Casinos and online poker sites list the rake upfront and / or they take a fee based on time (e.g. $x per hour). However, none of the players made a fuss, so it's implied they're OK with it. Or they didn't want to rock the boat and risk not being invited back. And as mentioned by B, other places are taking 5%, so they might think it's still good value.
Molly's game was profitable when run only on tips, but only if all the players were able to fulfill their credit obligations. Which is why Molly started taking a rake: to cover her losses when players couldn't pay up. It was naive for Molly not to have any way to recover money from players, but it is a sign of her character - she doesn't want people to be hurt.
Charlie (her new lawyer): You said you left ten-times that much on the street. Back at my office. I said it’s a $250,000 retainer and you said you left ten times that on the street.
Molly: Yeah.
Charlie: You were extending credit. You’re destitute and you left two and a half million dollars on the street?
Molly: I had to.
Charlie: Didn’t anyone try to buy your debt sheet?
Molly: Everyone tried to buy my debt sheet, is this the right time to--
Charlie: Why didn’t you sell it like you sold your clothes?
Molly: I couldn't.
Charlie: Why?
Molly: I couldn't be sure how they were gonna collect.