In Thor: Ragnarok moments before Odin's death he says
My Life is all that held her back
How did Odin keep Hela away from Asgard?
...although where or how is not stated.
Her violent appetites grew beyond my control. I couldn’t stop her, so I imprisoned her. Locked her away. She draws her strength from Asgard...and once she gets there, her powers will be limitless.
The implication is that she was imprisoned by Odin's magic in some fashion which, on his passing, failed and she could be freed.
Reasoning, based on the quotation, is that Hela's own magic was related to distance from Asgard so it seems likely that whatever "prison" was used was sufficiently far away from Asgard to weaken it so that she couldn't override Odin's.
The script also suggests that the "prison" was another dimension
A piercing scream cuts the air as a figure is hurtled out of the portal, crashing out of this dimensional rift is HELA.
Odin had banished her to some other realm (which isn't defined in the movie, but in Norse mythology, and incidentally also the Thor comics, Hela is the Goddess of Death, who rules over the domain of Hel, the equivalent of the underworld, where the dead go).
She was unable to return so long as Odin lived to keep the banishment in place. When he died, his magics faded with him and so she could (and did, practically immediately) return, right beside the place Odin died. After that, it was a simple matter of hitching a lift via the Bifrost back to Asgard after Loki called for it and she was "home".