The film depends heavily on the events of X-Men 3: The Last Stand. In that film, Wolverine does something he regrets heavily, and it is the basis for his character arc in The Wolverine. So to fully appreciate this film, you should watch X-Men 3.
But to fully appreciate X-Men 3, you should see X-Men and X-Men 2, since it's a sequel to those, and a continuation of those stories.
You can skip X-Men: First Class, as it has no bearing on The Wolverine. And you should skip X-Men Origins: Wolverine for the many reasons critics described back when it was released (it also doesn't have much bearing on The Wolverine).
Alternatively, if you're familiar with the X-Men from the comics, and are aware of some of the key storylines that occurred there, you can read this spoiler to get up to speed:
In X-Men 3, Jean Grey is taken over by the Phoenix. She goes on a rampage, allies with Magneto and begins a mutant vs. human war. Wolverine is forced to kill her because she's completely out of control.