First of all, let us remember that this is a movie, and not a documentary. As Jessica Coon puts it,
If they’d done justice to all that goes into understanding an unknown language, you’d just be watching a series of TED talks.
You mention that there is no common frame of reference, but you are forgetting something. Movements and reality are common to us all. If I point or show or somehow indicate to you any random object and I pronounce its name enough times, you will eventually associate the object to the sound.
Louise essentially tries to establish connections between sounds and letters against objects and actions.
From the LA Times,
The professor said that [that approach] would essentially be the approach field workers take too.
She shows them words and reads them out loud for the aliens to learn how one related to the other (tbh, my own opinion is that they knew english all along, but that's is off-topic). Then she points to what the words actually mean (herself and Ian, for example).
In a way, she proceeded the way a linguist would proceed in doing field research. Get very basic concepts, get the person to understand that we want to get individual words. You’ll usually start with things like body parts—okay, point to your arm, what’s their word for arm—and you build up from there. And she did that.
The aliens did the same as she asked them questions. The idea was to get them to give answers with similar words, meanings, links, and pick up on the patterns of what they meant. However, the entire process is not shown (on the film at least, I'm not sure about the book).
From Slate (once more),
I would have preferred if they spent a little more time on the early aspects of her working on the language. Once she had these splotches, they jumped pretty quickly from “OK, that’s how they communicate a concept” to “I’ve now got a mini-dictionary of a bunch of concepts,” right? They show you some cryptic images of her making measurements of different chunks of the splotches. But they don’t really show the process of how she got from there to understand what chunks mean what.